shm-rs

A Rust crate which allows to dynamically configure and read the schema based configuration files and also serialize data into JSON or binary files so later it could be deserialized into structs directly.

This crate also capable to construct a Rust structs and enumerators from the static schema files.

A statis scheme file is a file which contains a description of:

• how to read user provided config file
• how to store collected data
• how to serialize it (in which order)

A dynamic scheme file is a file which contains configurations which is required to read and serialize.

Features:

• Serialize or deserialize
• Data structures and collections
• Structure/Enums back to scheme (human readable format)

For the syntax see /docs/.

Version Log

V0.10.2
- Fixed generaion of code structure, added std::ops to Range and RangeInclusive.
- Cleaned old changelog.
- Moving towards the release V1.0.0

V0.10.(0-1)
- Added a multiline comment section ;[your comment here];
- Reimplemented error reporting for dynamic scheme.
- Added a config loader, collector

V0.9.2
Fixed problem with deserialization to scheme of lognint and longuint. (Patch had not applied correctly and the maintainer did not noticed it).

V 0.9.1 erased.

V0.9.1
Fixed problem with longint and longuint serialization and translation problem.

V 0.9.0 erased.

V0.9
Removed generics from the Lexer. Now default int, uint types are i64 and u64 which 
should be deserialized into equal sized or lower types.
The default long int and long uint are i128 and u128.
This was specially made to bind the data types to avoid any sort of incompatability.
Most of the systems are 64bits and I don't think that there is a need in 32bit mode.

V0.8.4
Fixed issue with arg-enums. 
Added static check which tests if enum item has same identifier name as define.

V0.8.3
Fixed problem with escape charater when converting back to the schema from structures.
Added list of escape sequences in Lexer.

V0.8.2
Fixed misprint title of the function in RustCode.
Fixed problem when argenum options were ordered in arbitrary order. An IndexSet
is now used which should store items in hashset in sequence.

V0.8.1
Fixed error missing enum bind name for f/argenum in f/vector.

V0.8
Added functions to lib.rs

Added:
(arg-enum-bind)

Version numbering

Until V1.0.0 this library is purely experimental and its code and syntax is subject for major changes.

Minor version number idicates changes in syntax of config files or code of the crate.

Build number is a bugfix. No changes to the code should be done.

Sequence

┌───────────────────┐              ┌───────────────────────┐
│ static_scheme.shm │              │  dynamic_scheme.shm   ◄─────────────┐
└────────┬──────────┘              └──────────┬────────────┘             │
         │                                    │                          │
         │                                    │                          │
         │                                    │                          │
 ┌───────▼─────────┐                 ┌────────▼─────────┐                │
 │  File / Buffer  │                 │  File / Buffer   │                │
 └───────┬─────────┘                 └────────┬─────────┘                │
         │                                    │                          │
         │ text                               │ text                     │
         │                                    │                          │
 ┌───────▼─────────┐                  ┌───────▼─────────┐                │
 │     Lexer       │                  │     Lexer       │                │
 └───────┬─────────┘                  └───────┬─────────┘                │
         │                                    │                          │
         │                                    │   data                   │
         │                                    │                          │
 ┌───────▼─────────┐                  ┌───────▼──────────┐               │
 │                 │      structure   │                  │               │
 │  StaticScheme   ├──────────┬───────►  DynamicScheme   │               │
 │                 │          │       │                  │               │
 └───────┬─────────┘          │       └───────┬──────────┘               │
         │  STRUCTURED DESCR  │               │                          │
         │                    │               │ Collected from scheme    │
         │                    │               │                          │
   ┌─────▼──────┐             │       ┌───────▼──────────┐               │
   │  RustCode  │             │       │ Structured Data  │               │
   └─────┬──────┘             │       └───────┬──────────┘               │
         │                    │               │                          │
         │                    │               │                          │
    ┌────▼──────┐             │       ┌───────▼──────────┐               │
    │Generated  │             └───────►  Serializator    │               │
    │Rust Code  │                     └───────┬──────────┘               │
    │Structures │                             │                          │
    │Enums      │                             │                          │
    │Consts     │                     ┌───────▼──────────┐               │
    └────┬──────┘                     │  Serialized Data │               │
         │                            │                  │               │
         │structure                   │  Binary/JSON     │               │
         │                            └──┬───────────────┘               │
         │                               │                               │
         │                               │                               │
     ┌───▼───────────────┐    data       │                               │
     │ Serde Deserialize ◄───────────────┘                               │
     └────────┬──────────┘                                               │
              │                                                          │
              │       data+structs  ┌──────────────────┐                 │
              │          ┌──────────► Serde Serialize  │                 │
              │          │          └───────┬──────────┘                 │
              │          │                  │                            │
     ┌────────▼──────────┤       ┌──────────▼──────────┐                 │
     │  Structures       │       │    Composer         ├─────────────────┘
     │  Enums            │       │    Struct2scheme    │
     └───────────────────┘       └─────────────────────┘

Example

Example of dynamic scheme file which aka DynamicScheme

(machine
    (address "01:40/5010-A3F.0") ; router's address (to access admin panel)
    (address "01:40/5010-A40.0") ; another adress (to access admin panel)
)

;; A 100MBit interface uplink from <another company> 
;; Clients from floor 2 using only this connection
(interface "wan0" 
    (phys-location "00:00:01")
    (address "01:40/5010-A3F")
    (mtu 2048u) ; 2 kiB
    (address-mask "01:40/5010-(A3F-A4A)")
)

;; A 1Gbit interface uplink from <title>
(interface "wwan1"
    (phys-location "00:00:02")
    (address "01:40/5010-A3F")
    (mtu 2048u)
    (address-mask "01:40/5010-(A3F-A4A)")
)

;; All client machines connected to router (floor 1)
(interface "lan0"
    (phys-location "00:01:01")
    (address "01:40/5010-A3F")
    (mtu 2048u)
    (address-mask "01:40/5010-A3F.(0-FFFF)")
)

;; All client machines connected to router (floor 2)
(interface "lan1"
    (phys-location "00:01:02")
    (address "01:40/5010-A40")
    (mtu 2048u)
    (address-mask "01:40/5010-A40.(0-FFFF)")
)

;; Route announcement
;; Clients on lan0 would receive:
;; to: 01:40/5010-(A3F-A40) metric: 1
;; to: 01:40/5010-any metric 1
;; to: any metric 1
(route-table "global"
    (interface-alias "lan0" ; announce the following routes on interface lan0
        (route-to "01:40/5010-(A3F-A40).*"
            (metric 1u)
        )
        (route-to "01:40/5010-*" 
            (via "wan0")
            (metric auto)
        )
        ; routing globally to whole internet
        (route-to "*" 
            (via "wwan1")
            (metric auto)
        )
    )

    (interface-alias "lan1"
        (route-to "01:40/5010-(A3F-A40).*"
            (metric 1u)
        )
        (route-to "*" 
            (via "wan0")
            (metric auto)
        )
        ; do not route to global internet via wwan1
    )

    ;; inbound routing
    ;; announce to uplink that we have something on our network
    ;; if there are any machines that is local only then don't announce the 
    ;; adress to uplink, so no route to host would be returned by uplink for the
    ;; remote host.
    (interface-alias "wan0"
        (route-to "01:40/5010-A3F.*" 
            (via "lan0")
            (metric 1u)
        )

        (route-to "01:40/5010-A40.*"
            (via "lan1")
            (metric 1u)
        )
    )

    (interface-alias "wwan1"
        (route-to "01:40/5010-A3F.*" 
            (via "lan0")
            (metric 1u)
        )

        ; no route to lan1 members via this interface
    )
)

For the above the following static scheme StaticScheme is required in order to read and serialize:

(version 1000)

(serializator "networking"
    (define "auto" 0u '("metric"))

    (procedure "address"
        (arg "a_addr" string)
    )

    ;(struct "NetAddr" "address"
    ;    (field "net_addr" (f/string '("a_addr")))
    ;)

    ; -- MACHINE
    (procedure "machine"
        (proc "p_address" '("address") (proc-allow '(collection)))
    )

    (struct "NetMachine" "machine"
        ;(field "nm_addrs" (f/vector (f/struct '("p_address"))))
        (field "nm_addrs" (f/vector (f/string '("p_address" "a_addr"))))
    )

    ; -- COMMON
    (procedure "phys-location"
        (arg "a_phys_loc" string)
    )

    (procedure "mtu"
        (arg "a_mtu" uint)
    )

    (procedure "address-mask"
        (arg "a_addr_mask" string)
    )

    ; -- INTERFACE

    (procedure "interface"
        (arg "a_if_alias" string)
        (proc "p_phys_loc" '("phys-location"))
        (proc "p_addr" '("address"))
        (proc "p_mtu" '("mtu") (proc-allow '(optional)))
        (proc "p_address_mask" '("address-mask"))
    )

    (struct "NetInterface" "interface"
        (field "ni_ifalias" (f/string '("a_if_alias")))
        (field "ni_ifloc" (f/string '("p_phys_loc" "a_phys_loc")))
        (field "ni_addr" (f/string '("p_addr" "a_addr")))
        (field "ni_ifmtu" (f/optional) (f/uint '("p_mtu" "a_mtu")))
        (field "ni_ifaddr_mask" (f/string '("p_address_mask" "a_addr_mask")))
    )

    ; -- ROUTE-TABLE

    (procedure "via"
        (arg "a_via" string)
    )

    (procedure "metric" 
        (arg "a_metric" symbol uint)
    )

    (procedure "route-to"
        (arg "a_route_addr" string)
        (proc "p_via" '("via") (proc-allow '(optional)))
        (proc "p_metric" '("metric"))
    )

    (struct "NetRouteTo" "route-to"
        (field "rt_dest" (f/string '("a_route_addr")))
        (field "rt_via" (f/optional) (f/string '("p_via" "a_via")))
        (field "rt_metric" (f/uint '("p_metric" "a_metric")))
    )

    (procedure "interface-alias"
        (arg "a_route_title" string)
        (proc "p_route_to" '("route-to") (proc-allow '(collection)))
    )

    (struct "NetInterfaceAlias" "interface-alias"
        (field "if_alias" (f/string '("a_route_title")))
        (field "if_routes_to" (f/vector (f/struct '("p_route_to"))))
    )

    (procedure "route-table"
        (arg "a_route_title" string)
        (proc "p_route_tables" '("interface-alias") (proc-allow '(collection optional)))
    )

    (struct "NetRouteTable" "route-table"
        (field "rt_title" (f/string '("a_route_title")))
        (field "rt_tbls" (f/optional) (f/vector (f/struct '("p_route_tables"))))
    )

    ; -- ROOT

    (rootprocedure
        (proc "p_machine" '("machine"))
        (proc "p_interface" '("interface") (proc-allow '(collection)))
        (proc "p_route_tables" '("route-table") (proc-allow '(collection)))
    )

    (rootstruct "Network"
        (field "net_machine" (f/struct '("p_machine")))
        (field "net_ifs" (f/vector (f/struct '("p_interface"))))
        (field "net_rt_tbls" (f/vector (f/struct '("p_route_tables"))))
    )
)

So the above config is serialized into:

{
  "net_machine": {
    "nm_addrs": [
      "01:40/5010-A3F.0",
      "01:40/5010-A40.0"
    ]
  },
  "net_ifs": [
    {
      "ni_ifalias": "wan0",
      "ni_ifloc": "00:00:01",
      "ni_addr": "01:40/5010-A3F",
      "ni_ifmtu": 2048,
      "ni_ifaddr_mask": "01:40/5010-(A3F-A4A)"
    },
    {
      "ni_ifalias": "wwan1",
      "ni_ifloc": "00:00:02",
      "ni_addr": "01:40/5010-A3F",
      "ni_ifmtu": 2048,
      "ni_ifaddr_mask": "01:40/5010-(A3F-A4A)"
    },
    {
      "ni_ifalias": "lan0",
      "ni_ifloc": "00:01:01",
      "ni_addr": "01:40/5010-A3F",
      "ni_ifmtu": 2048,
      "ni_ifaddr_mask": "01:40/5010-A3F.(0-FFFF)"
    },
    {
      "ni_ifalias": "lan1",
      "ni_ifloc": "00:01:02",
      "ni_addr": "01:40/5010-A40",
      "ni_ifmtu": 2048,
      "ni_ifaddr_mask": "01:40/5010-A40.(0-FFFF)"
    }
  ],
  "net_rt_tbls": [
    {
      "rt_title": "global",
      "rt_tbls": [
        {
          "if_alias": "lan0",
          "if_routes_to": [
            {
              "rt_dest": "01:40/5010-(A3F-A40).*",
              "rt_via": null,
              "rt_metric": 1
            },
            {
              "rt_dest": "01:40/5010-*",
              "rt_via": "wan0",
              "rt_metric": 0
            },
            {
              "rt_dest": "*",
              "rt_via": "wwan1",
              "rt_metric": 0
            }
          ]
        },
        {
          "if_alias": "lan1",
          "if_routes_to": [
            {
              "rt_dest": "01:40/5010-(A3F-A40).*",
              "rt_via": null,
              "rt_metric": 1
            },
            {
              "rt_dest": "*",
              "rt_via": "wan0",
              "rt_metric": 0
            }
          ]
        },
        {
          "if_alias": "wan0",
          "if_routes_to": [
            {
              "rt_dest": "01:40/5010-A3F.*",
              "rt_via": "lan0",
              "rt_metric": 1
            },
            {
              "rt_dest": "01:40/5010-A40.*",
              "rt_via": "lan1",
              "rt_metric": 1
            }
          ]
        },
        {
          "if_alias": "wwan1",
          "if_routes_to": [
            {
              "rt_dest": "01:40/5010-A3F.*",
              "rt_via": "lan0",
              "rt_metric": 1
            }
          ]
        }
      ]
    }
  ]
}

Raw result without formatting:

{"net_machine":{"nm_addrs":["01:40/5010-A3F.0","01:40/5010-A40.0"]},"net_ifs":[{"ni_ifalias":"wan0","ni_ifloc":"00:00:01","ni_addr":"01:40/5010-A3F","ni_ifmtu":2048,"ni_ifaddr_mask":"01:40/5010-(A3F-A4A)"},{"ni_ifalias":"wwan1","ni_ifloc":"00:00:02","ni_addr":"01:40/5010-A3F","ni_ifmtu":2048,"ni_ifaddr_mask":"01:40/5010-(A3F-A4A)"},{"ni_ifalias":"lan0","ni_ifloc":"00:01:01","ni_addr":"01:40/5010-A3F","ni_ifmtu":2048,"ni_ifaddr_mask":"01:40/5010-A3F.(0-FFFF)"},{"ni_ifalias":"lan1","ni_ifloc":"00:01:02","ni_addr":"01:40/5010-A40","ni_ifmtu":2048,"ni_ifaddr_mask":"01:40/5010-A40.(0-FFFF)"}],"net_rt_tbls":[{"rt_title":"global","rt_tbls":[{"if_alias":"lan0","if_routes_to":[{"rt_dest":"01:40/5010-(A3F-A40).*","rt_via":null,"rt_metric":1},{"rt_dest":"01:40/5010-*","rt_via":"wan0","rt_metric":0},{"rt_dest":"*","rt_via":"wwan1","rt_metric":0}]},{"if_alias":"lan1","if_routes_to":[{"rt_dest":"01:40/5010-(A3F-A40).*","rt_via":null,"rt_metric":1},{"rt_dest":"*","rt_via":"wan0","rt_metric":0}]},{"if_alias":"wan0","if_routes_to":[{"rt_dest":"01:40/5010-A3F.*","rt_via":"lan0","rt_metric":1},{"rt_dest":"01:40/5010-A40.*","rt_via":"lan1","rt_metric":1}]},{"if_alias":"wwan1","if_routes_to":[{"rt_dest":"01:40/5010-A3F.*","rt_via":"lan0","rt_metric":1}]}]}]}

And the following Rust code will be generated using generator.rs:

use serde::{Serialize, Deserialize};

#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct NetMachine 
{ 
    pub nm_addrs: Vec<String>
}


#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct NetInterface 
{ 
    pub ni_ifalias: String,
    pub ni_ifloc: String,
    pub ni_addr: String,
    pub ni_ifmtu: Option<u64>,
    pub ni_ifaddr_mask: String
}


#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct NetRouteTo 
{ 
    pub rt_dest: String,
    pub rt_via: Option<String>,
    pub rt_metric: u64
}


#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct NetInterfaceAlias 
{ 
    pub if_alias: String,
    pub if_routes_to: Vec<NetRouteTo>
}


#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct NetRouteTable 
{ 
    pub rt_title: String,
    pub rt_tbls: Option<Vec<NetInterfaceAlias>>
}


#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct Network 
{ 
    pub net_machine: NetMachine,
    pub net_ifs: Vec<NetInterface>,
    pub net_rt_tbls: Vec<NetRouteTable>
}