Shinkansen - CLI File Preprocessor

A fast, cross-platform command-line file preprocessor built with Rust that uses MiniJinja templates to transform files with dynamic variables.

Features

• Flexible Input: Process single files, multiple files, directories (with optional recursion), or stdin
• Smart Output: Write to stdout, single files, or directories while preserving structure
• Multiple Config Formats: Load variables from JSON, YAML, or TOML configuration files
• Variable Precedence: Layer variables from environment (explicit), config files, and CLI arguments
• Environment Control: Load specific environment variables with --env flag
• Cross-Platform: Builds on Linux, macOS, and (future) Windows
• Powered by MiniJinja: Full access to MiniJinja's powerful template syntax and filters

Installation

From Source

git clone https://github.com/bbeardsley/shinkansen
cd shinkansen
cargo build --release
# Binary will be at ./target/release/shinkansen

Using Just

just install

This installs the binary to your Cargo bin directory (typically ~/.cargo/bin).

Usage

Add the binary to your PATH or use it directly:

./target/release/shinkansen --help

Usage Examples

Basic Usage

Process a single template file with CLI variables:

shinkansen template.txt -D name="John" -D app="MyApp" -o -

Using Configuration Files

JSON Config (config.json):

{
  "name": "Alice",
  "version": "1.0.0",
  "environment": "production"
}

shinkansen template.txt -c config.json -o output.txt

YAML Config (config.yaml):

name: Bob
version: "2.0.0"
debug: true

shinkansen template.txt -c config.yaml -o -

TOML Config (config.toml):

name = "Charlie"
version = "3.0.0"
features = ["auth", "api"]

shinkansen template.txt -c config.toml -o output.txt

Processing from Stdin

echo "Hello, {{ name }}!" | shinkansen - -D name="World"
# Output: Hello, World!

# Explicit stdout with -o -
echo "Hello, {{ name }}!" | shinkansen - -D name="World" -o -

Variable Precedence

Variables are layered with increasing precedence:

1. Environment variables (only when specified with --env, lowest)
2. Config file variables
3. CLI arguments (highest)

export GREETING="Hi"
shinkansen template.txt -c config.yaml --env="GREETING" -D GREETING="Hello" -o -
# CLI value "Hello" wins

Environment Variable Control

Load specific environment variables:

shinkansen template.txt --env="PATH,HOME,USER" -o -

Escaping Special Characters

When using -D flag or environment variables, you can escape special characters:

• \\ - escaped backslash
• \, - escaped comma
• \= - escaped equals
• \[ - escaped left bracket
• \] - escaped right bracket
• \{ - escaped left curly brace
• \} - escaped right curly brace

Examples:

# Escape curly brackets in command line
shinkansen template.txt -D "message=Hello\{world\}"

# Escape multiple characters
shinkansen template.txt -D "data=key\=value\,more\\data"

# Environment variables with escaping
export VAR="test\{curly\}brackets"
shinkansen template.txt --env VAR

Environment variables are not loaded automatically. You must explicitly specify which variables to load using the --env flag. Multiple variables can be specified as a comma-separated list.

Directory Processing

Process all files in a directory (non-recursive):

shinkansen templates/ -o output/ -D env="prod"

Process directory recursively:

shinkansen templates/ -r -o output/ -c config.json

Directory structure is preserved in the output:

templates/
  ├── app.conf
  └── services/
      └── api.conf

→ output/
  ├── app.conf
  └── services/
      └── api.conf

Multiple Files

shinkansen file1.txt file2.txt file3.txt -o output_dir/ -D version="1.0"

Output Options

To stdout (default for single input):

shinkansen template.txt -D var="value"
# Or explicitly with -o -
shinkansen template.txt -D var="value" -o -

To a specific file:

shinkansen template.txt -D var="value" -o result.txt

To a directory:

shinkansen template.txt -D var="value" -o output/
# Creates output/template.txt

Template Syntax

Shinkansen uses MiniJinja templates. Here are some common patterns:

Variables

Hello, {{ name }}!
Version: {{ version }}

Filters

{{ name | upper }}
{{ email | default(value="no-email") }}
{{ price | round(precision=2) }}

Conditionals

{% if debug %}
Debug mode is ON
{% else %}
Production mode
{% endif %}

Loops

{% for item in items %}
- {{ item }}
{% endfor %}

Comments

{# This is a comment and won't appear in output #}

For complete MiniJinja syntax documentation, see: https://docs.rs/minijinja/latest/minijinja/

Input/Output Rules

Common Use Cases

Configuration Management

Generate configuration files for different environments:

# Development
shinkansen app.conf.template -c dev.yaml -o config/dev/app.conf

# Production
shinkansen app.conf.template -c prod.yaml -o config/prod/app.conf

Code Generation

shinkansen api_template.rs -c schema.json -o src/generated/api.rs

Documentation

shinkansen README.template.md -D version="1.2.3" -D date="2024-01-15" -o README.md

Batch Processing

shinkansen templates/ -r -c project.yaml -o output/

Building for Different Platforms

The project is designed to be cross-platform.

Linux:

cargo build --release

macOS:

cargo build --release

Windows:

cargo build --release

Cross-compilation:

# For Windows from Linux/macOS
cargo build --release --target x86_64-pc-windows-msvc

Error Handling

Shinkansen provides helpful error messages for common issues:

• Missing required template variables
• Invalid template syntax
• File/directory access errors
• Invalid input/output combinations
• Config file parsing errors

Requirements

• Rust 2024 edition or later
• Cargo
• Optional: just for build tasks

Dependencies

• clap - Command-line argument parsing with derive macros
• clap_complete - Shell completion generation
• minijinja - Template engine
• serde - Serialization framework
• serde_json - JSON support
• serde_yaml - YAML support
• toml - TOML support
• walkdir - Directory traversal
• tempfile - Temporary file handling (dev and runtime)

Examples

See the examples/ directory for complete, working examples:

• project.md.template - Project README template with conditionals and loops
• nginx.conf.template - Nginx configuration template
• config.json - Sample JSON configuration
• EXAMPLES.md - Detailed usage examples

Run examples directly:

# Simple template with CLI variables
./target/release/shinkansen examples/project.md.template \
  -D project_name="My Project" \
  -D version="2.0.0" \
  -o -

# Using config file
./target/release/shinkansen examples/project.md.template \
  -c examples/config.json \
  -o -

License

MIT License - see repository for details.

Troubleshooting

Variable Not Found Error

If you see Variable 'x' not found in context, ensure:

1. The variable is defined via -D, config file, or environment
2. The variable name matches exactly (case-sensitive)
3. Use the default filter for optional variables: {{ var | default(value="fallback") }}

Multiple Inputs Require Directory

When processing multiple files, specify an output directory:

shinkansen file1.txt file2.txt -o output_dir/