#make #task #runner #javascript #web

app chompbuild

Make-like parallel task runner with a JS extension system

22 releases

Uses new Rust 2021

0.2.5 Aug 4, 2022
0.2.4 Jul 24, 2022
0.2.3 Jun 18, 2022
0.2.2 Apr 15, 2022
0.1.9 Feb 24, 2022

#114 in Web programming

Download history 8/week @ 2022-04-21 63/week @ 2022-04-28 137/week @ 2022-05-05 308/week @ 2022-05-12 80/week @ 2022-05-19 109/week @ 2022-05-26 183/week @ 2022-06-02 2/week @ 2022-06-09 25/week @ 2022-06-16 3/week @ 2022-06-23 5/week @ 2022-06-30 41/week @ 2022-07-07 22/week @ 2022-07-14 98/week @ 2022-07-21 18/week @ 2022-07-28 46/week @ 2022-08-04

204 downloads per month

GPL-3.0 license

205KB
5K SLoC

CHOMP

Crates.io Discord

'JS Make' - parallel task runner for the frontend ecosystem with a JS extension system.

Install

Make sure you have Cargo installed, then run:

cargo install chompbuild

If you don't have access to Cargo, there is an npm wrapper available:

npm install -g chomp

Note: npm scripts add over 100ms to the script run time.

Common platform binaries are also available for all releases.

To quickly setup Chomp in a GitHub Actions CI workflow, see the Chomp GitHub Action.

Documentation

Getting Started

Migrating from npm Scripts

To convert an existing project using npm "scripts" to Chomp, run:

$ chomp --init --import-scripts
 chompfile.toml created with 2 package.json script tasks imported.

or the shorter version:

$ chomp -Ii
 chompfile.toml created with 2 package.json script tasks imported.

Then use chomp <name> instead of npm run <name>, and enjoy the new features of task dependence, incremental builds and parallelism!

Hello World

chomp works against a chompfile.toml TOML configuration in the same directory as the chomp command is run.

Chomp builds up tasks as trees of files which depend on other files, then runs those tasks with maximum parallelism.

For example, here's a task called hello which builds hello.txt based on the contents of name.txt, which itself is built by another command:

chompfile.toml

version = 0.1

[[task]]
target = 'name.txt'
run = '''
  echo "No name.txt, writing one."
  echo "World" > name.txt
'''

[[task]]
name = 'hello'
target = 'hello.txt'
dep = 'name.txt'
run = '''
  echo "Hello $(cat name.txt)" > hello.txt
'''

with this file saved, the hello command will run all dependency commands before executing its own command:

$ chomp hello

🞂 name.txt
No name.txt, writing one.
 name.txt [4.4739ms]
🞂 hello.txt
 hello.txt [5.8352ms]

$ cat hello.txt
Hello World

Finally it populates the hello.txt file with the combined output.

Subsequent runs use the mtime of the target files to determine what needs to be rerun.

Rerunning the hello command will see that the hello.txt target is defined, and that the name.txt dependency didn't change, so it will skip running the command again:

chomp hello

 name.txt [cached]
 hello.txt [cached]

Changing the contents of name.txt will then invalidate the hello.txt target only, not rerunning the name.txt command:

$ echo "Chomp" > name.txt
$ chomp hello

 name.txt [cached]
  hello.txt invalidated by name.txt
🞂 hello.txt
 hello.txt [5.7243ms]

$ cat hello.txt
Hello Chomp

Array deps can be defined for targets, whose targets will then be run first with invalidation based on target / deps mtime comparisons per the standard Makefile approach.

In Windows, Powershell is used and Bash on posix systems. Since both echo and > are defined on both systems the above works cross-platform (Powershell is automatically put into UTF-8 mode for > to work similarly).

Note that && and || are not supported in Powershell, so multiline scripts and ; are preferred instead.

JS Tasks

Alternatively we can use engine = 'node' or engine = 'deno' to write JavaScript in the run function instead:

chompfile.toml

version = 0.1

[[task]]
target = 'name.txt'
engine = 'node'
run = '''
  import { writeFile } from 'fs/promises';
  console.log("No name.txt, writing one.");
  await writeFile(process.env.TARGET, 'World');
'''

[[task]]
name = 'hello'
target = 'hello.txt'
deps = ['name.txt']
engine = 'node'
run = '''
  import { readFile, writeFile } from 'fs/promises';
  const name = (await readFile(process.env.DEP, 'utf8')).trim();
  await writeFile(process.env.TARGET, `Hello ${name}`);
'''

Tasks are run with full parallelism permitted by the task graph, which can be controlled via the -j flag to limit the number of simultaneous executions.

Using the --watch flag watches all dependencies and applies incremental rebuilds over invalidations only.

Or using chomp hello --serve runs a static file server with watched rebuilds.

See the task documentation for further details.

Monorepos

There is no first-class monorepo support, but some simple techniques can achieve the use cases.

For example, consider a monorepo where packages/[pkgname]/chompfile.toml defines per-package tasks.

A base-level chompfile.toml could run the test task of all the sub-packages with the following chompfile.toml:

[[task]]
name = 'test'
dep = 'packages/#/chompfile.toml'
run = 'chomp -c $DEP test'

chomp test will then use task interpolation to run the multiple sub-package test tasks in parallel. A similar approach can also be used for a basic unit testing.

Adding serial = 'true' the interpolation can be made to run in series rather than in parallel.

Cross-project dependencies are not currently supported. Instead if packages/a/chompfile.toml's build task depends on packages/b/chompfile.toml's build task to run first, then packages/a/chompfile.toml might look like:

[[task]]
name = 'build'
run = 'cargo build'
dep = 'build:deps'

[[task]]
name = 'build:deps'
run = 'chomp -c ../a build'

This would still be fast so long as packages/a/chompfile.toml's build task has its targets and dependencies properly configured to do zero work if the all target mtimes are greater than their dependencies.

Extensions

Extensions are able to register task templates for use in Chompfiles.

Extensions are loaded using the extensions list, which can be any local or remote JS file:

version = 0.1
extensions = [
  "./local.js",
  "https://remote.com/extension.js"
]

A core extensions library is provided with useful templates for the JS ecosystem, with the short protocol chomp:ext, a shorthand for the @chompbuild/extensions package contents.

A simple example is included below.

See the @chompbuild/extensions package for extension descriptions and examples.

Example: TypeScript with SWC

To compile TypeScript with the SWC template:

version = 0.1
extensions = ['chomp@0.1:swc']

[[task]]
name = 'build:typescript'
template = 'swc'
target = 'lib/##.js'
deps = ['src/##.ts']

In the above, all src/**/*.ts files will be globbed, have SWC run on them, and output into lib/[file].js along with their source maps.

The ## and # interpolation syntax are special because unlike glob dependencies (which are also supported), they must be a 1-1 relation from dependency to target.

Only files not existing or whose src mtimes are invalidated (or SWC itself is updated) will be rebuilt.

Specific files or patterns can be built directly by name as well, skipping all other build work:

chomp lib/main.js lib/dep.js

🞂 lib/dep.js
🞂 lib/app.js
 lib/dep.js [317.2838ms]
 lib/app.js [310.0831ms]

Patterns are also supported for building tasks by name or filename (the below two commands are equivalent):

$ chomp lib/*.js
$ chomp :build:*

To remove the template magic, run chomp --eject to convert the chompfile.toml into its untemplated form:

$ chomp --eject

 chompfile.toml template tasks ejected

Resulting in the updated chompfile.toml:

version = 0.1

[[task]]
name = 'build:typescript'
target = 'lib/##.js'
dep = 'src/##.ts'
stdio = 'stderr-only'
run = 'node ./node_modules/@swc/cli/bin/swc.js $DEP -o $TARGET --no-swcrc --source-maps -C jsc.parser.syntax=typescript -C jsc.parser.importAssertions=true -C jsc.parser.topLevelAwait=true -C jsc.parser.importMeta=true -C jsc.parser.privateMethod=true -C jsc.parser.dynamicImport=true -C jsc.target=es2016 -C jsc.experimental.keepImportAssertions=true'

License

GPLv3

For licensing discussion, see https://github.com/guybedford/chomp/issues/62.

Dependencies

~82MB
~1.5M SLoC