#orm #charybdis #automatic #tool #migration #model

app charybdis-migrate

Automatic Migration Tool for Charybdis ORM

19 unstable releases (3 breaking)

new 0.5.2 Apr 26, 2024
0.4.18 Apr 20, 2024
0.4.10 Mar 23, 2024
0.2.8 Dec 27, 2023
0.2.5 Nov 24, 2023

#309 in Database interfaces

Download history 5/week @ 2024-02-02 30/week @ 2024-02-09 265/week @ 2024-02-16 304/week @ 2024-02-23 194/week @ 2024-03-01 63/week @ 2024-03-08 75/week @ 2024-03-15 306/week @ 2024-03-22 31/week @ 2024-03-29 271/week @ 2024-04-05 356/week @ 2024-04-12 355/week @ 2024-04-19

1,013 downloads per month

MIT license

89KB
2K SLoC

Breaking changes

As of 0.4.13 UDT fields must be in the same order as they are in the database. This is due to scylla driver limitation that does not support named bind values. Earlier versions would automatically order fields by name, but this is no longer the case as ORM could not work with exiting UDTs.

Automatic migration Tool:

charybdis-migrate tool that enables automatic migration to database without need to write migrations by hand. It iterates over project files and generates migrations based on differences between model definitions and database.

Installation

  cargo install charybdis-migrate

Usage

migrate --hosts <host> --keyspace <your_keyspace> --drop-and-replace (optional)

Automatic migration

  • charybdis-migrate enables automatic migration to database without need to write migrations by hand. It expects src/models files and generates migrations based on differences between model definitions and database. It supports following operations:

    • Create new tables

    • Create new columns

    • Drop columns

    • Change field types (drop and recreate column --drop-and-replace flag)

    • Create secondary indexes

    • Drop secondary indexes

    • Create UDTs

    • Create materialized views

    • Table options

        #[charybdis_model(
            table_name = commits,
            partition_keys = [object_id],
            clustering_keys = [created_at, id],
            table_options = #r"
                WITH CLUSTERING ORDER BY (created_at DESC) 
                AND gc_grace_seconds = 86400
            ";
        )]
        #[derive(Serialize, Deserialize, Default)]
        pub struct Commit {...}
      

      ⚠️ If table exists, table options will result in alter table query that without CLUSTERING ORDER and COMPACT STORAGE options.

      Model dropping is not added. If you removed model, you need to drop table manually.

  • Running migration

    ⚠️ If you are working with existing datasets, before running migration you need to make sure that your **model ** definitions structure matches the database in respect to table names, column names, column types, partition keys, clustering keys and secondary indexes so you don't alter structure accidentally. If structure is matched, it will not run any migrations. As mentioned above, in case there is no model definition for table, it will not drop it. In future, we will add modelize command that will generate src/models files from existing data source.

  • Global secondary indexes

    If we have model:

    #[charybdis_model(
        table_name = users,
        partition_keys = [id],
        clustering_keys = [],
        global_secondary_indexes = [username]
    )]
    

    resulting query will be: CREATE INDEX ON users (username);

  • Local secondary Indexes

    Indexes that are scoped to the partition key

    #[charybdis_model(
        table_name = menus,
        partition_keys = [location],
        clustering_keys = [name, price, dish_type],
        global_secondary_indexes = [],
        local_secondary_indexes = [dish_type]
    )]
    

    resulting query will be: CREATE INDEX ON menus((location), dish_type);

Define Tables

 use charybdis::macros::charybdis_model;
 use charybdis::types::{Text, Timestamp, Uuid};
 
 #[charybdis_model(
     table_name = users,
     partition_keys = [id],
     clustering_keys = [],
     global_secondary_indexes = [username],
     local_secondary_indexes = [],
 )]
 pub struct User {
     pub id: Uuid,
     pub username: Text,
     pub email: Text,
     pub created_at: Timestamp,
     pub updated_at: Timestamp,
     pub address: Address,
 }

Define UDT

 use charybdis::macros::charybdis_udt_model;
 use charybdis::types::Text;
 
 #[charybdis_udt_model(type_name = address)]
 pub struct Address {
     pub street: Text,
     pub city: Text,
     pub state: Option<Text>,
     pub zip: Text,
     pub country: Text,
 }

Define Materialized Views

use charybdis::macros::charybdis_view_model;
use charybdis::types::{Text, Timestamp, Uuid};

#[charybdis_view_model(
    table_name=users_by_username,
    base_table=users,
    partition_keys=[username],
    clustering_keys=[id]
)]
pub struct UsersByUsername {
    pub username: Text,
    pub id: Uuid,
    pub email: Text,
    pub created_at: Timestamp,
    pub updated_at: Timestamp,
}

Resulting auto-generated migration query will be:

CREATE MATERIALIZED VIEW IF NOT EXISTS users_by_email
AS SELECT created_at, updated_at, username, email, id
FROM users
WHERE email IS NOT NULL AND id IS NOT NULL
PRIMARY KEY (email, id)

Dependencies

~14–28MB
~367K SLoC