#sql #airflow #cli-tool #dbt #data-engineer

app badass

badass is a cli tool inspired by dbt and airflow

3 releases

0.1.2 Mar 22, 2024
0.1.1 Mar 22, 2024
0.1.0 Mar 22, 2024

#653 in Database interfaces

Download history 350/week @ 2024-03-18 36/week @ 2024-03-25 49/week @ 2024-04-01

55 downloads per month

Apache-2.0

25KB
472 lines

BADASS

Badass is a CLI tool inspired by DBT and Airflow. Mainly a playground for me to become more familiar with Rust.

Configuration

Badass uses config-rs to search for a badass config (.toml, .json, .yaml or .ini) in the current workin directory. It is also possible to override settings with an environment variable (prefixed with BADASS_)

BADASS_output_compiled=/tmp/compiled badass settings
The settings are: 

Settings {
    models: Models {
        location: "./demo/models",
    },
    output: Output {
        compiled: "/tmp/compiled",
        materialized: "./target/materialized",
    },
    query_engine: QueryEngine {
        params: "host=localhost user=tim",
    },
}

Features

Compile SQL templates

badass compile

We leverage minijinja to generate SQL files.

{% set payment_methods = ["bank_transfer", "credit_card", "gift_card"] %}
select
    order_id,
    {%- for payment_method in payment_methods %}
    sum(case when payment_method = '{{payment_method}}' then amount end) as {{payment_method}}_amount,
    {%- endfor %}
    sum(amount) as total_amount
from app_data.payments
group by 1

Is compiled into the following:

select
    order_id,
    sum(case when payment_method = 'bank_transfer' then amount end) as bank_transfer_amount,
    sum(case when payment_method = 'credit_card' then amount end) as credit_card_amount,
    sum(case when payment_method = 'gift_card' then amount end) as gift_card_amount,
    sum(amount) as total_amount
from app_data.payments
group by 1

View (compiled) SQL template query results

badass show demo
.------------------------.
| Tim   | Van Wassenhove |
| Tiebe | Van Wassenhove |
| Amber | Van Wassenhove |
| Evy   | Penninckx      |
'------------------------'

Materialize SQL templates

badass materialize

Use the (compiled) SQL templates to build database artifacts (tables, views, ...)

Currently we only render CTAS, eg:

SELECT * FROM foo

Becomes

CREATE TABLE xxx AS SELECT * FROM foo

Dependencies

~14–25MB
~439K SLoC