Salmo

A database migration system.

Currently it's not proof of concept stage. It's usable for experimentation, but I'd recommend waiting before using in production.

Motivation

Everyone and their mother has written a database migration system. A simple one is not hard. However they tend to have one of 2 issues: either they have no good development story (they are immutable and only intended for production deployment), or have a good development story but they are have a number of traps that can be problematic in production. Salmo is intended specifically to be usable during local development, but then the local changes are commited into an immutable form for production deployment.

It is currently only targeting postgres, but if it's successful there's no reason other databases cannot be supported. It can be used with any framework, and is intended to be easy to install and use in a wide variety of environments.

Migration format

salmo will search for a directory with a Salmo.toml file to use as the base for all commands. This Salmo file has a number of configuration values, including the directory that the migrations are in.

The default file structure is

dir/
  Salmo.toml
  migrations/
    committed.toml
    create-users/
      migrate.sql
      revert.sql

Usage

salmo init: create a Salmo.toml in the current directory, if one does not exist

salmo new [migration-id]: create a new migration directory with the given id

salmo migrate: execute all pending (committed, but not executed) migrations against the current database

salmo status: will show which migrations are what state. The states are:

• uncommitted: This migration is in development and will not be executed in salmo migrate.
• tried: This migration is uncommitted, but is being tested locally with salmo try
• pending: This migration has been committed, but not executed against the current database.
• complete: This migration has been commited and executed.

by default only uncommitted or pending migrations are shown.

salmo commit [migration-id]: commit the migration. The hash of the migration will be calculated so it can't be changed any more. The migration id can be omitted if only one uncommitted migration exists

salmo try [migration-id]: execute an uncommitted migration. If only one uncommitted migration exists, then that will be executed.toml

salmo retry [migration-id]: revert and re-execute a tried migration

salmo untry [migration-id]: revert a tried migration

salmo reset: revert all tried migrations

A note on evironments

Salmo requires (possibly multiple) environments to be configured (though it can defer the actual connection params to environment variables). Each environment has a is_production flag. If the environment is production, then salmo will not execute salmo try or salmo reset (since especially reset can be dangerous since salmo will undo execute schema changes).

Configuring salmo

Salmo is configured via the Salmo.toml file.

migrations_directory: [optional] the directory which contains the migration files envrionments: [required] an array of different environments with their connection parameters environments.name: [required] the name of the environment. This must be unique between all environments environments.connection_params: [optional] a table of connection params. Each one can be either a string or a table of the form {env: 'ENV_VAR_NAME' environments.connection_url: [optional] the string url to connect to the database with. It can be either a string or a table of the form {env: 'ENV_VAR_NAME'} environments.is_production: a boolean, if the environment is considered to be production default_environments: [optional] an array of active environment names. These environments will be active if no others are specified.

The active environment(s) can be set via the param --environments (or -e), or the SALMO_ENV environment variable. If multiple environments are active, commands will be executed in each. This makes keeping development and testing databases in sync easy.

committed.toml

Committed.toml does not need be manually edited, but it can be inspected if needed. It lists the execution order of migrations with their hash.

Recommended workflow

• create a migration with salmo new
• edit the newly created file
• salmo try to try to execute the migration
• if the migration fails, fix it and salmo try again
• develop code that uses the migration
• possibly realize that something is missing from the migration, or it could be better. In this case do a salmo reset, edit the migration and salmo try again
• possibly get code review
• possibly even test in a staging environment
• once the whole feature is ready to be deployed, run salmo commit
• when the new feature is deployed, salmo migrate to run the migration.