EnvPath

A library for parsing and deserialising paths with special rules.

The format is similar to ["$proj(com.xy.z): data ? cfg", "$const: os", "$val: rand-16"]

Maybe I should change it to deserializing.
Never mind all the details, let's get started!

中文

preface

Note: This readme contains a lot of non-technical content.

Before the official start, if it's not too much trouble, could you provide me with an answer to my query?

How have we been solving the problem of cross-platform path configuration?

Assume the following configuration.

[dir]
data = "C:\\Users\\[username]\\AppData\\Roaming\\[dirname]"

Perhaps we would create a new Map (e.g. HashMap<String, Dirs>) for the configuration file, allowing different platforms to use different configurations.

[dir.linux]
data = "/home/[username]/.local/share/[appname]"
cache = "/home/[username]/.cache/[app]"

[dir.macos]
data = "/Users/[username]/Library/Application Support/x.y.z"

[dir.xxxbsd]

[dir.your-os-name]

This is a good approach, but is there a more universal method?

So people thought of using environment variables.

I guess you're thinking of the XDG specification. Since it's so useful, why don't we use $XDG_DATA_HOME/[appname]/ on all platforms?

Unfortunately, not all platforms support it, so we choose the more universal $HOME.
Sadly, on early versions of Windows, there might not be %HOME%, but only %userprofile%.
So, what should we do? And how do we do it?
We can use external crates to automatically retrieve paths for different platforms or manually write different directory mapping relationships for different platforms.
Great, it seems like we have solved the cross-platform issue, but we may have forgotten one thing. That is, the path separators on different platforms may be different.
We can generate paths for different platforms, but the format generated may not be very universal.

• Windows: C:\path\to\xxx
• Unix-like: /path/to/xxx

Path Separator

The following are some additions.

The path separator in the Macintosh operating system has undergone several changes throughout its history. In the early versions of the Macintosh operating system, the path separator was a forward slash (/). However, with the introduction of the Hierarchical File System (HFS) in 1985, the path separator was switched to a colon (:).
With the release of the macOS operating system in 2001, the HFS+ file system was introduced, and the path separator remained a colon (:). However, as of macOS Catalina (10.15), Apple has introduced a new read-only file system called APFS, which uses a forward slash (/) as the path separator.
According to the Apple Technical Note TN1150: HFS Plus Volume Format, the use of the colon as the path separator in the HFS file system was intended to make the Macintosh operating system more user-friendly by allowing users to easily navigate through directories. The switch to the forward slash in APFS is likely due to its compatibility with other Unix-based systems.
Source: Apple Technical Note TN1150

The following is a table of separators.

Note: some operating systems may change their file path separators between versions. I can't guarantee that the table above is exactly correct, so if something goes wrong, report an issue and let me change it.

Since different platforms use different path separators, how can we make them look the same?

The answer is to use an array (or vector).

However, its disadvantages are quite obvious. For ordinary users, this format may be harder to read than a string (although developers may prefer the former). Please note that user configuration files are for users to see, not just for deserialization, so readability is crucial.

For example, ["C:", "\\", "Users", "Public"] is equivalent to C:\Users\Public. There's a small detail that's easy to overlook, which is that the second element is "\".

EnvPath (raw) also uses an array structure (actually a vector), but with special rules.

For example, ["$dir: dl ? doc"] specifies the Downloads directory (which has different paths on different platforms), and if the Downloads directory does not exist, it will use the Documents directory instead. Note: A single "?" and a double "?" are different, as we will mention later.

After saying a lot of irrelevant things, let's get started!

Quick Start

Before we start, please make sure that the Rust version is not too old. Because this library uses some relatively new syntax, such as let-else (which requires 1.65+).

Basic guide

First, we need to add the dependency.

cargo add envpath --no-default-features --features=dirs,consts,project

Then add the following content to our main() or test function.

use envpath::EnvPath;

let v = EnvPath::from(["$dir: data", "$env: test_qwq", "app"]).de();
dbg!(v.display(), v.exists());

This is a simple example, and there are more features and concepts that we haven't mentioned here.

Don't worry, take it step by step.

It will then output something like the following.

[src/lib.rs:74] v.display() = "/home/m/.local/share/$env: test_qwq/app"
[src/lib.rs:74] v.exists() = false

We can see that $env: test_qwq was not resolved successfully.

So what happened? Is it malfunctioning?

No, it's not. This is a deliberate design decision. When EnvPath was designed, it was intentionally made compatible with regular paths.

If one day, EnvPath adds a feature that requires the prefix $recycle and includes the keyword bin to resolve to a specific directory. And on your system disk, there happens to be a $RECYCLE:BIN folder, and unfortunately, the file system on that disk has not enabled case sensitivity. When there is a collision with a same-named path, it will first try to resolve it, and if it fails, it will assume that the same-named path exists and return it.

The probability of collision with the same-named path exists, but with a little bit of skill, most collision events can be avoided.

Trick: Use whitespace characters (spaces, line breaks, tabs, etc.) and use ?(will be introduced below)

For example, $env: test_qwq can be written as $env ： test-QwQ

Although many spaces have been added, if successful, they will be resolved to the same value. Using the posix sh on unix to describe the above expression is: $TEST_QWQ (i.e. all lowercase letters are changed to uppercase, and all - are changed to _)

Although you may find this approach difficult to accept, it is customary for global system environment variables, and I have not created any new rules.

Since the resolution failed, why not return an empty directory?

Let's take an example with the env command in posix sh!

Assuming the directory you want to access is $XDG_DATA_HOME/app, if the relevant env is empty, then what you are accessing is /app, which is different from the expected result. (I want to go home, but I bought the wrong train ticket 🎫

You may argue: I can use ${ENV_NAME:-FALLBACK} to specify the fallback! It's clearly because you're just not smart enough.

However, sometimes a careless mistake can lead to a big problem. I think complaining less will make life more beautiful.

At this point, you may have forgotten where the error occurred earlier: $env: test_qwq.
So how do we solve it? You can try changing it to $env: test_qwq ? user ? logname, or add more question marks and valid environment variable names.

I won't explain the function of ? here, go explore it yourself, often you will discover more fun.

Going back to the code we mentioned earlier, let's simplify it a bit. EnvPath::from(["$dir: data"]).de();

As is well known, [] is an array. But what exactly is .de()?

In Chinese, if we use "de" to refer to a country, it means Germany. Written in Chinese characters, it is "德". If used to describe a person, it can mean that he has "noble character".

Ohhhh! I got it. This function took a trip to Germany (de), so it changed and became a function with noble character.

Anyway, I think you're very smart, and this function did indeed change.
But it only converts a structure like $env: QuQ? ?? qwq-dir? AwA-home into another value.

Serialization and deserialization

If you want to serialize/deserialize a configuration file, you need to enable the serde feature of envpath and add serde, as well as other related dependencies.

Next, we will add a ron dependency (You can actually use formats such as yaml or json, but you need to add the relevant dependencies instead of using ron.)

cargo add envpath --features=serde
cargo add serde --features=derive
cargo add ron

Serialization

Now let's try serialization.

        use serde::{Deserialize, Serialize};
        use envpath::EnvPath;

        #[derive(Debug, Default, Serialize, Deserialize)]
        #[serde(default)]
        struct Cfg<'a> {
            dir: Option<EnvPath<'a>>,
        }

        let dir = Some(EnvPath::from([
            "$env: user ?? userprofile ?? home",
        ]));

        let ron_str = ron::to_string(&Cfg { dir }).expect("Failed to ser");
        println!("{ron_str}");

        std::fs::write("test.ron", ron_str)
            .expect("Failed to write the ron cfg to test.ron");

The output result is: (dir: Some(["$env: user ?? userprofile ?? home"]))

It looks like the structure is the same as before serialization, except for the additional dir key.

Yes, after serialization, it looks like that.

This path format is suitable for cross-platform use.

Since environment variables and other things may be dynamically changed.

Keeping the raw format during serialization and obtaining its true path during deserialization is reasonable.

Deserialization

Next, let's try deserialization!

        use envpath::EnvPath;
        use serde::{Deserialize, Serialize};
        use std::fs::File;

        #[derive(Debug, Default, Serialize, Deserialize)]
        #[serde(default)]
        struct Cfg<'a> {
            dir: Option<EnvPath<'a>>,
        }

        let cfg: Cfg = ron::de::from_reader(
            File::open("test.ron").expect("Failed to open the file: text.ron"),
        )
        .expect("Failed to deser ron cfg");

        dbg!(&cfg);

        if let Some(x) = cfg.dir {
            if x.exists() {
                println!("{}", x.display())
            }
        }

The output result of the above function is:

[src/lib.rs:116] &cfg = Cfg {
    dir: Some(
        EnvPath {
            raw: [
                "$env: user ?? userprofile ?? home",
            ],
            path: Some(
                "/home/m",
            ),
        },
    ),
}
/home/m

The ? operator checks if a value exists. If it doesn't exist, continue checking. If it exists, use that value.

On the other hand, the ?? operator requires both the value and the path to exist.

For example, consider $env: user ? userprofile. Let's assume that the value of user is m, and userprofile is empty. Since the value of user exists, the expression returns m.

If we change it to $env: user ?? userprofile ? home, even though the value of user exists, its path does not. So we continue checking. Then, since the value of userprofile does not exist, we continue checking until the condition is satisfied.

? and ?? have different functions, and adding ?? does not mean that you can discard ?. For values that are normal strings, such as $const: os, rather than paths, ? is more useful than ??. Each one has an important role to play.

That concludes the basic guide. The above describes some basic features.

project_dirs has more advanced features. Here are some simple introductions. For example, $proj(com.macro-hard.app-name): data will generate a data directory for this project (It does not create it automatically, just generates its value).

M$, his smile was as wide as the Grand Canyon, but behind it lurked a simmering rage that could rival a volcano as he approached me and asked a question as sweet as honey on a summer day.

Sorry, please forgive me.

Now, it is $proj(com. x. y): data.

• On Android, it is /data/data/com.x.y
• On macOS, it is /Users/[username]/Library/Application Support/com.x.y

After learning the basic usage, we will continue to introduce and supplement more content.

• The simplest: consts
• Common standard directories: dirs
• Advanced project directories: project

In the following text, we will introduce what their functions are and what values they all have.

Features

env

In the previous text, we have learned about the basic usage. Here are a few more things to explain.

"env" refers to environment variables. $env:home is used to obtain the value of the HOME environment variable. $env:xdg-data-home is equivalent to $XDG_DATA_HOME.

As for the use of "?", you can refer to the previous text.
When you understand the purpose of $env:userprofile ?? QwQ-Dir ? LocalAppData ? home, then congratulations, you have learned how to use env!

consts

Use $const:name (e.g. $const:arch) or $const:alias (e.g. $const:architecture) to obtain constant values. These values are obtained at compile time rather than runtime.

deb-arch

The following table shows the possible output values for $const:deb-arch:

For example, if you compile a package for armv7, the value obtained by $const:arch would be arm, while $const:deb-arch could be armhf.

value

The value feature needs to be enabled.

Use $val:name (e.g. $val: rand-16) to obtain the values. Unlike $const:, most of the values here are obtained at runtime.

$val: rand-[usize] syntax requires the rand feature to be enabled.

rand is used to obtain random content, and currently only supports strings.

remix

example

["
    $const: empty ??
        env * home ?
        env * HOME
",
    "test"
]

env* can be used for fallback, but unlike $env:, it does not automatically convert lowercase letters to uppercase, and it does not automatically convert - to _.

• env * home retrieves $home, not $HOME.
• $env: home => $HOME
• env * xdg-data-home => $xdg-data-home, not $XDG_DATA_HOME
• $env: xdg-data-home => $XDG_DATA_HOME

Note: If the $env: expression contains a *, the automatic conversion feature will also be disabled.

The following syntax is currently supported:

• $const: exe_suffix ? env * HOME ? env * XDG_DATA_HOME ? env * EXE_SUFFIX
• $env: home ? xdg-data-home ? exe_suffix ? const * exe_suffix

Not supported:

• $const: exe_suffix ? $env: home ? xdg-data-home ? exe_suffix

If it is supported, the parsing may become complicated and there could be confusion between $env: exe_suffix and $const: exe_suffix.

dirs

These are some base-dirs, or you could say standard directories.
Use $dir:name (e.g. $dir:dl) or $dir:alias (e.g. $dir:download) to obtain the directory.
Many of these contents are obtained from dirs, but there are also some additions.

Linux

first_path refers to the first $PATH variable, while last_path refers to the last one. If PATH is /usr/local/bin:/usr/bin, then /usr/local/bin is the first_path, and /usr/bin is the last_path.

Regarding tmp and temp:

• tmp: First, get the value of $env:tmpdir. If it exists, use that value. If not, use env::temp_dir() to obtain the directory path and check if it is read-only. If it is, use ["$dir:cache", "tmp"].
  • On some platforms, the tmp directory may be read-only for regular users, such as /data/local/tmp.
• temp: Use env::temp_dir() to obtain the directory path, without performing any checks.
• tmp-rand: Generate a random temporary directory, rand feature needs to be enabled.

Android

• var:

  • sd = "/storage/self/primary"

For items not listed, use Linux data.

Windows

• var:
  • ms_dir = $home\AppData\Roaming\Microsoft

macOS

project

Most of the data is obtained from directories.

Use $proj(qualifier.organization.application):name (e.g. $proj(org.moz.ff):data) or $proj(com.company-name.app-name):alias to obtain the project directory.

These directories will vary depending on the operating system and the specific configuration.

Assuming the project is (org.moz.ff), here's an example:

Linux

Android

• var:

  • sd = "/storage/self/primary"

Windows

macOS

"??" in project

The ? syntax supported by $proj is slightly more complex than other types, because it has () while others don't.

Don't worry, if you have already mastered the core syntax, then you can quickly master the ?? syntax of $proj in a few minutes.

Assuming there are three projects:

• (org. moz. ff)
• (com. gg. cr)
• (com. ms. eg)

The first example is:

["
    $proj (org . moz . ff ): runtime ? data ?? state ?
    (com . gg . cr): cfg ?? cache ?
    (com . ms . eg): local-data ? data
"]

Let's start parsing the runtime of the ff project, unfortunately, it does not exist.

Next, we parse the data! Great, we find that its value exists.

Because there are double ?, we also need to check if the file path exists.

Unfortunately, the data directory does not exist.

The next one is state.

Unfortunately, it did not pass either because its value does not exist.

By now, all members of the ff camp have been defeated, and none have been successfully parsed.

So, we continue to parse the cr project, luckily, it succeeded on the first try.

The value of cr's cfg not only exists, but the path also exists.

The final return value is the cfg directory of the cr project!

The second example is:

["
    $proj (org . moz . ff )：runtime ？ data ？？ state ？
    (com . gg . cr)： cfg ？？ cache ？
    (com . ms . eg)： local-data ？ data
"]

Q: Why don't I see any difference from the first example?

A: It allows you to use full-width symbols (colon and question mark) as separators, but this is limited.

It depends on the first symbol that appears. That is to say, if the first separator is a half-width "?"(\u{3F}) instead of a full-width "？"(\u{FF1F}), then the rest should also be expressed in half-width.