2 releases
0.3.1 | Mar 10, 2023 |
---|---|
0.3.0 | Feb 13, 2023 |
0.2.1 |
|
0.2.0 |
|
0.1.0 |
|
#1945 in Rust patterns
7KB
136 lines
Datafet
Few utility functions for our mostly data engineering applications.
EID
Few functions to generate consistent hash based [external][1] ids.
pub fn get_external_id_str(name: &str, prefix: &str) -> String {
let mut spooky_hasher = SpookyHasher::new(1337, 7331);
spooky_hasher.write(name.as_bytes());
let hash_uint64 = spooky_hasher.finish();
let hash_hex = format!("{:x}", hash_uint64);
let base32 = base32::encode(Alphabet::RFC4648 { padding: false }, hash_hex.as_bytes());
format!("{}-{}", prefix, base32.to_ascii_lowercase())
}
There are few kinds we have in most of our applications:
pub fn get_db_eid(database_name: &str) -> String {
get_external_id_str(database_name, "db")
}
pub fn get_table_eid(table_name: &str) -> String {
get_external_id_str(table_name, "table")
}
pub fn get_job_eid(job_name: &str) -> String {
get_external_id_str(job_name, "job")
}
pub fn get_query_eid(query_name: &str) -> String {
get_external_id_str(query_name, "query")
}
[1]: meaning that components outside of the his codebase can hold onto a particular id and it means the same underlying entity all the time.
Time
There is only one small function in this to get the string version of Utc::now().
pub fn utc_now() -> String {
let now: DateTime<Utc> = Utc::now();
now.to_rfc3339()
}
Dependencies
~2MB
~23K SLoC