62 releases (11 breaking)
new 0.12.10 | Dec 4, 2023 |
---|---|
0.12.9 | Nov 30, 2023 |
0.12.7 | Oct 19, 2023 |
0.11.0 | Jul 3, 2023 |
#95 in Database interfaces
1,809 downloads per month
Used in 2 crates
155KB
3K
SLoC
schematic
derive(Config)
Schematic is a light-weight, macro-based, layered serde configuration and schema library, with built-in support for merge strategies, validation rules, environment variables, and more!
- Supports JSON, TOML, and YAML based configs via serde.
- Load sources from the file system or secure URLs.
- Source layering that merge into a final configuration.
- Extend additional files through an annotated setting.
- Field-level merge strategies with built-in merge functions.
- Aggregated validation with built-in validate functions (provided by garde).
- Environment variable parsing and overrides.
- Beautiful parsing and validation errors (powered by miette).
- Generates schemas that can be rendered to TypeScript types, JSON schemas, and more!
This crate was built specifically for moon, and many of the design decisions are based around that project and its needs. Because of that, this crate is quite opinionated and won't change heavily.
Usage
Define a struct and derive the Config
trait.
use schematic::Config;
#[derive(Config)]
struct AppConfig {
#[setting(default = 3000, env = "PORT")]
port: usize,
#[setting(default = true)]
secure: bool,
#[setting(default = vec!["localhost".into()])]
allowed_hosts: Vec<String>,
}
Then load, parse, merge, and validate the configuration from one or many sources. A source is either a file path, secure URL, or code block.
use schematic::{ConfigLoader, Format};
let result = ConfigLoader::<AppConfig>::new()
.code("secure: false", Format::Yaml)?
.file("path/to/config.yml")?
.url("https://ordomain.com/to/config.yaml")?
.load()?;
result.config;
result.layers;
The format for files and URLs are derived from the trailing extension.
Configuration
The bulk of schematic is powered through the Config
trait and the associated derive macro. This
macro helps to generate and automate the following:
- Generates a partial configuration struct, with all field values wrapped in
Option
. - Provides default value and environment variable handling.
- Implements merging and validation logic.
- And other minor features, like metadata.
The struct that derives Config
represents the final state, after all partial layers
have been merged, and default and environment variable values have been applied. This means that all
fields (settings) should not be wrapped in Option
, unless the setting is truly optional (think
nullable in the config file).
#[derive(Config)]
pub struct ExampleConfig {
pub number: usize,
pub string: String,
pub boolean: bool,
pub array: Vec<String>,
pub optional: Option<String>,
}
This pattern provides the optimal developer experience, as you can reference the settings as-is, without having to unwrap them, or use
match
orif-let
statements!
Partials
A powerful feature of schematic is what we call partial configurations. These are a mirror of the
derived configuration, with all settings wrapped in Option
, are prefixed with
Partial
, and have common serde and derive attributes automatically applied.
For example, the ExampleConfig
above would generate the following partial struct:
#[derive(Clone, Debug, Default, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
#[serde(default, deny_unknown_fields, rename_all = "camelCase")]
pub struct PartialExampleConfig {
#[serde(skip_serializing_if = "Option::is_none")]
pub number: Option<usize>,
#[serde(skip_serializing_if = "Option::is_none")]
pub string: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub boolean: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
pub array: Option<Vec<String>>,
#[serde(skip_serializing_if = "Option::is_none")]
pub optional: Option<String>,
}
So what are partials used for exactly? Partials are used for the entire parsing, layering, extending, and merging process, instead of the base/final configuration.
When deserializing a source with serde, we utilize the partial config as the target type, because not all fields are guaranteed to be present. This is especially true when merging multiple sources together, as each source may only contain a subset of the final config. Each source represents a layer to be merged.
Partials are also beneficial when serializing, as only settings with values will be written to the source, instead of everything! A common complaint of serde's strictness.
As stated above, partials also handle the following:
- Defining default values for settings.
- Inheriting environment variable values.
- Merging partials with strategy functions.
- Validating current values with validate functions.
- Declaring extendable sources.
Nested
Configuration can easily be nested within other configuration using the
#[setting(nested)]
attribute. Child configuration will be deeply merged and validated alongside
the parent.
#[derive(Config)]
pub struct ChildConfig {
// ...
}
#[derive(Config)]
pub struct ParentConfig {
#[setting(nested)]
pub nested: ChildConfig,
#[setting(nested)]
pub optional_nested: Option<ChildConfig>,
}
The #[setting(nested)]
attribute is required, as the macro will substitute the config struct with
its partial struct variant.
Nested configuration can also be wrapped in collections, like
Vec
andHashMap
. However, these are tricky to support and may not work in all situations!
Contexts
Context is an important mechanism that allows for different default values, merge strategies, and validation rules to be used, for the same configuration struct, depending on context!
To begin, a context is a struct with a default implementation.
#[derive(Default)]
struct ExampleContext {
some_value: bool,
another_value: usize,
}
Context must then be associated with a configuration through the context
attribute field.
#[derive(Config)]
#[config(context = ExampleContext)]
pub struct ExampleConfig {
// ...
}
And then passed to the ConfigLoader.load_with_context
method.
let context = ExampleContext {
some_value: true,
another_value: 10,
};
let result = ConfigLoader::<ExampleConfig>::new()
.url(url_to_config)?
.load_with_context(&context)?;
Refer to the default values, merge strategies, and validation rules sections for more information on how to use context.
Metadata
Configuration supports basic metadata for use within error messages through the
#[config]
attribute. Right now we support a name, derived from the struct name or the serde
rename
attribute field.
Metadata can be accessed with the META
constant.
ExampleConfig::META.name;
Serde support
By default the Config
macro will apply
#[serde(default, deny_unknown_fields, rename_all = "camelCase")]
to the
partial configuration. The default
and deny_unknown_fields
ensure proper parsing
and layer merging.
The rename_all
field can be customized, and we also support the rename
field, both via the
top-level #[config]
attribute.
#[derive(Config)]
#[config(rename = "ExampleConfig", rename_all = "snake_case")]
struct Example {
// ...
}
The
rename
field will also update the metadata name.
Configuration enums
Configurations typically use enums to handle value variations of a specific setting. To
simplify this process, we offer a ConfigEnum
macro/trait that can be derived for enums with
unit-only variants.
#[derive(ConfigEnum)]
enum LogLevel {
Info,
Error,
Debug,
Off
}
This enum will generate the following implementations:
- Provides a static
variants
method, that returns a list of all variants. Perfect for iteration. - Implements
FromStr
andTryFrom
for parsing from a string. - Implements
Display
for formatting into a string.
The string value/format is based on the variant name, and is converted to kebab-case by default.
This can be customized with the #[serde(rename_all = "kebab-case")]
attribute, which keeps
consistency with serde's handling.
Fallback variant
Although ConfigEnum
only supports unit variants, we do support a catch-all variant known as the
"fallback variant", which can be defined with #[variant(fallback)]
. Fallback variants are
primarily used when parsing from a string, and will be used if no other variant matches.
However, this pattern does have a few caveats:
- Only 1 fallback variant can be defined.
- The fallback variant must be a tuple variant with a single field.
- The field type can be anything and we'll attempt to convert it with
try_into()
. - The fallback inner value is not casing formatted based on serde's
rename_all
.
#[derive(ConfigEnum)]
enum Value {
Foo,
Bar,
Baz
#[variant(fallback)]
Other(String)
}
Common derives
Furthermore, all enums (not just unit enums) typically support the same derived traits, like
Clone
, Eq
, etc. To reduce boilerplate, we offer a derive_enum!
macro that will apply these
traits for you.
derive_enum!(
#[derive(ConfigEnum, Default)]
enum LogLevel {
Info,
Error,
Debug,
#[default]
Off
}
);
This macro will inject the following attributes:
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
#[serde(rename_all = "kebab-case")]
Settings
Settings are the individual fields/members of a configuration struct, and can be annotated with the
optional #[setting]
attribute.
Default values
Structs only.
In schematic, there are 2 forms of default values:
- The first is on the partial configuration, is defined with the
#[setting]
attribute, and is the first layer of the configuration to be merged. - The second is on the final configuration itself, and uses
Default
to generate the final value if none was provided. This acts more like a fallback.
This section will talk about the #[setting]
attribute and default
. The default
attribute field
is used for declaring primitive values, like numbers and booleans. It can also be used for array and
tuple literals, as well as function (mainly for from()
) and macros calls.
#[derive(Config)]
struct AppConfig {
#[setting(default = "/")]
base: String,
#[setting(default = 3000)]
port: usize,
#[setting(default = true)]
secure: bool,
#[setting(default = vec!["localhost".into()])]
allowed_hosts: Vec<String>,
}
#[derive(Config)]
enum Host {
#[setting(default)]
Local,
Remote(HostConfig),
}
Enums only support
#[setting(default)]
, which denotes that variant as the default. It does not support setting values for the variant itself, or its inner tuple fields.
If you need more control or need to calculate a complex value, you can pass a reference to a
function to call. This function receives the context as the first argument (use ()
or
generics if you don't have context), and can return an optional value. If None
is returned, the
Default
value will be used instead.
fn find_unused_port(ctx: &Context) -> Option<usize> {
let port = do_find();
Some(port)
}
#[derive(Config)]
struct AppConfig {
#[setting(default = find_unused_port)]
port: usize,
}
Environment variables
Structs only.
Settings can also inherit values from environment variables via the env
attribute field. When
using this, variables take the highest precedence, and are merged as the last layer.
#[derive(Config)]
struct AppConfig {
#[setting(default = 3000, env = "PORT")]
port: usize,
}
Container prefixes
If you'd prefer to not define env
for every setting, you can instead define a prefix on the
containing struct using the env_prefix
attribute field. This will define an environment variable
for all non-nested fields in the struct, in the format of "env prefix + field name" in
UPPER_SNAKE_CASE.
For example, the environment variable below is now APP_PORT
.
#[derive(Config)]
#[config(env_prefix = "APP_")]
struct AppConfig {
#[setting(default = 3000)]
port: usize,
}
Parsing values
We also support parsing environment variables into the required type. For example, the variable may be a comma separated list of values, or a JSON string.
The parse_env
attribute field can be used, which requires a path to a function to handle the
parsing, and receives the variable value as a single argument.
#[derive(Config)]
struct AppConfig {
#[setting(env = "ALLOWED_HOSTS", parse_env = schematic::env::split_comma)]
allowed_hosts: Vec<String>,
}
We provide a handful of built-in parsing functions in the
env
module.
When defining a custom parse function, you should return an error with ConfigError::Message
if
parsing fails. A None
value can also be returned, which will fallback to the previous or default
value.
use schematic::ConfigError;
pub fn custom_parse(var: String) -> Result<Some<ReturnValue>, ConfigError> {
do_parse()
.map(|v| Some(v))
.map_err(|e| ConfigError::Message(e.to_string()))
}
Extendable
Structs only.
Configs can extend other configs, generating an accurate layer chain, via the extend
attribute
field. Extended configs can either be a file path (relative from the current config) or a secure
URL. For example:
extends:
- "./another/file.yml"
- "https://domain.com/some/other/file.yml"
When defining extend
, we currently support 3 types of patterns. The first is with a single string,
which only allows a single file to be extended.
#[derive(Config)]
struct AppConfig {
#[setting(extend, validate = schematic::validate::extends_string)]
extends: Option<String>,
}
The second is with a list of strings, allowing multiple files to be extended. This is the YAML example above.
#[derive(Config)]
struct AppConfig {
#[setting(extend, validate = schematic::validate::extends_list)]
extends: Option<Vec<String>>,
}
And lastly, supporting both a string or a list, using our built-in enum.
#[derive(Config)]
struct AppConfig {
#[setting(extend, validate = schematic::validate::extends_from)]
extends: Option<schematic::ExtendsFrom>,
}
We suggest making this field optional, so that extending is not required by consumers!
Merge strategies
A common requirement for configuration is to merge multiple sources/layers into a final result. By
default schematic will replace the previous value with the next value if the next value is Some
,
but sometimes you want far more control, like shallow merging or deep merging collections.
This can be achieved with the merge
attribute field, which requires a path to a function to call.
#[derive(Config)]
struct AppConfig {
#[setting(merge = schematic::merge::append_vec)]
allowed_hosts: Vec<String>,
}
#[derive(Config)]
enum Projects {
#[setting(merge = schematic::merge::append_vec)]
List(Vec<String>),
// ...
}
We provide a handful of built-in merge functions in the
merge
module.
When defining a custom merge function, the previous value, next value, and context are passed as
arguments, and the function must return an optional merged result. If None
is provided, neither
value will be used.
Here's an example of the merge function above.
pub fn append_vec<T, C>(mut prev: Vec<T>, next: Vec<T>, context: &C) -> Result<Option<Vec<T>>, ConfigError> {
prev.extend(next);
Ok(Some(prev))
}
Validation rules
What kind of configuration crate would this be without built-in validation? As such, we support it as a first-class feature, with built-in validation rules provided by garde.
In schematic, validation does not happen as part of the serde parsing process, and instead happens for each partial configuration to be merged.
Validation can be applied on a per-setting basis with the validate
attribute field, which requires
a path to a function to call.
#[derive(Config)]
struct AppConfig {
#[setting(validate = schematic::validate::alphanumeric)]
secret_key: String,
#[setting(validate = schematic::validate::regex("^\.env"))]
env_file: String,
}
Or on a per-variant basis when using an enum.
#[derive(Config)]
enum Projects {
#[setting(validate = schematic::validate::min_length(1))]
List(Vec<String>),
// ...
}
We provide a handful of built-in validation functions in the
validate
module. Furthermore, some functions are factories which can be called to produce a validator.
When defining a custom validate function, the value to check is passed as the first argument, the
current partial as the second, and the context as the third. The ValidateError
type
must be used for failures.
use schematic::ValidateError;
fn validate_string(
value: &str,
partial: &PartialAppConfig,
context: &Context
) -> Result<(), ValidateError> {
if !do_check(value) {
return Err(ValidateError::new("Some failure message"));
}
Ok(())
}
If validating an item in a vector or collection, you can specifiy the nested path when erroring. This is extremely useful when building error messages.
use schematic::PathSegment;
ValidateError::with_segments(
"Some failure message",
// [i].key
[PathSegment::Index(i), PathSegment::Key(key.to_string())]
)
Serde support
The rename
and skip
attribute fields are currently supported and will apply a #[serde]
attribute to the partial setting.
#[derive(Config)]
struct Example {
#[setting(rename = "type")]
type_of: SomeEnum,
}
Generators
Schematic provides a schema modeling layer that defines the shape of types, which all configuration and enums implement. These schemas can then be passed to a generator, which renders the schema into a specific format, and writes the result to a file.
use schematic::{schema, renderers};
fn main() {
let mut generator = schema::SchemaGenerator::default();
generator.add::<ConfigOne>();
generator.add::<ConfigTwo>();
generator.add::<EnumThree>();
generator.add::<OtherWithSchemas>();
}
Added types will recursively add all nested schemas, so you only need to add the root types, and not everything!
JSON schemas
- Enabled with the
json_schema
feature. - The last schema to be added to the generator will be the root document, while all previous schemas will be definitions/references.
generator.generate(
output_dir.join("schema.json"),
schema::json_schema::JsonSchemaRenderer::default(),
);
TypeScript types
- Enabled with the
typescript
feature. - Each schema added to the generator will be
export
ed as a type.
generator.generate(
output_dir.join("types.ts"),
schema::typescript::TypeScriptRenderer::default(),
);
Features
The following Cargo features are available:
config
(default) - Enables configuration support (all the above stuff).url
(default) - Enables loading, extending, and parsing configs from URLs.
Parsing
json
(default) - Enables JSON.toml
- Enables TOML.yaml
- Enables YAML.
Validation
valid_email
- Enables email validation with theschematic::validate::email
function.valid_url
- Enables URL validation with theschematic::validate::url
andurl_secure
functions.
Schema generation
-
schema
- Generates schemas for schematic types and built-in Rust types. -
json_schema
- Enables JSON schema generation. -
typescript
- Enables TypeScript types generation. -
type_chrono
- Implements schematic for thechrono
crate. -
type_regex
- Implements schematic for theregex
crate. -
type_relative_path
- Implements schematic for therelative-path
crate. -
type_rust_decimal
- Implements schematic for therust_decimal
crate. -
type_semver
- Implements schematic for thesemver
crate. -
type_serde_json
- Implements schematic for theserde_json
crate. -
type_serde_toml
- Implements schematic for thetoml
crate. -
type_serde_yaml
- Implements schematic for theserde_yaml
crate. -
type_url
- Implements schematic for theurl
crate. -
type_version_spec
- Implements schematic for theversion_spec
crate. -
type_warpgate
- Implements schematic for thewarpgate
crate.
Dependencies
~2–19MB
~288K SLoC