libzettels

Libzettels is a library intended as a backend for applications which implement Niklas Luhmann's system of a "Zettelkasten".

• If you want to develop a Zettelkasten application, continue reading here.
• If you already develop software using this library, you probably want to look directly at the API-docs.
• If you're looking for a Zettelkasten application, have a look at Zettels.
• If you have no idea what a Zettelkasten is, have a look at Zettels' README, in particular the section "What the heck is a Zettelkasten?"

Code and package

Libzettels is still in alpha stage and probably buggy. Expect the API to change until version 1.0.0. Libzettels is written in Rust. See code and crate here:

• Gitlab repository
• crates.io

Information for application developers

Again, see below for details on how a Zettelkasten works and what Libzettel's approach is.

In short, Libzettel takes a directory (your Zettelkasten's root directory) containing Markdown files (which may be in sub-directories) with a YAML metadata block (as defined by pandoc).

To implement a Zettelkasten, libzettels bundles the information about the relations between these files (your zettels) in a queryable index. To populate this index, libzettels does two things:

1. It inspects the fields title, keywords and followups of the YAML metadata block.

---
title:  'Example Zettel'
keywords: [example, question]
followups: [file.md, subdir/anotherfile.md, ../yetanotherfile.md]
foo: 'Potentially more data ignored by libzettels.'
...

2. It parses the inline-style links (like [example](afile.md)) of the markdown document body and extracts the targets of these links.

For this latter task, libzettels offers three methods:

1. using the UNIX command line tool grep.
2. using ripgrep.
3. a method native to libzettels. It works, but is in no way optimized for speed and thus (at least for a large numer of files) probably much slower than the other methods.

Since it is available on all platforms, the native method is the default.
However, application developers using this library should consider whether and how to offer their users to choose their prefered method. For instance, grep is available out of the box for most platforms out there (like GNU/Linux, macOS, diverse flavours of BSD and other UNIX-variants) and should thus be an easy option.

What does libzettels do?

Libzettels was designed to deal with two kinds of zettel files:

• Markdown-based zettel files
• Image-based zettel files

but it is able to handle a lot of different formats, as long as the YAML-header is present and readable (see README).

Markdown-based zettel file

A markdown-based zettel file is a markdown file with a YAML-header as defined by pandoc). Most of the information concerning the interrelations with other Zettels is read from the YAML-header. An exception is the field links, which is generated by parsing markdown links in the file (only of the "inline" syntax).
The YAML-metadata may contain additional information (like author, e.g.). However, such additional data is ignored, here.

Example

---
title:  'Some Zettel'
keywords: [example]
followups: [file2.md, file3.md]
...

Here begins the Zettel's actual content, possibly containing links to 
[other Zettels](file2.md). These links are used for internal links within 
the Zettelkasten. External links can be achieved by [reference style][id] 
links. These are ignored by the Zettelkasten. Lorem ipsum…

[id]: https://daringfireball.net/projects/markdown/syntax#link

Image-based zettel file

Image-based zettel files are a way to integrate scans of handwritten notes into the Zettelkasten. To do this, a user copies the image file to the root directory of the Zettelkasten and creates an accompanying text file containing the YAML-metadata about the interrelation to other zettels and some sort of reference to the image file. For example, such a text file could be a markdown file with a YAML-header and an image link to the image file. Because the links field can not be automatically filled with meaningful data, it needs to be set in the YAML (or it will be a empty list).

Image files (all non-text formats like png, jpg etc., plus svg) are ignored by the Zettelkasten, so they can be safely kept in the same folder as the accompanying text files and other zettel files.

Example

---
title:  'An image-based Zettel'
keywords: [example]
followups: [file2.md, file3.md]
links: [file1.md]
...
![](imagefile.png)

Other files as zettel files

Other files can be used, too. In fact, the two standard file types described above, are just examples of two possible approaches:

1. Files with a YAML-header (like markdown-based zettel files)
2. Files with an accompanying YAML-metadata file (like image-based zettel files)

Files with a YAML-header

This approach, has the following requirements:

• It must be possible to apply the method lines() of the BufRead-Trait from Rust's standard library to the file.
• The file should contain a valid YAML-document beginning with --- and ending with either ... oder ---. If the file contains more than one YAML-document, libzettels will ignore all but the first one.

If you prefer to write the actual contents of a zettel in some other format than markdown (e.g. LaTeX, HTML, dokuwiki etc.), that will work just fine. However the links field will not be automatically filled for non-markdown files, so – just as with image-based zettels – you will have to designate links in that field of the YAML, manually.

---
title: 'A Zettel with LaTeX-Markup'
keywords: [example]
followups: [another-zettel.tex]
links: [yet-another-zettel.tex]
...

\section{A thought}
The actual \emph{content} of the Zettel is in LaTeX. Maybe I'll just paste
it into a LaTeX-document, later...

Another concern to think about is your further toolchain. Libzettels will accept a LaTeX-file with a YAML-header, but your LaTeX-engine probably won't. But still, as far as libzettels is concerned, it is entirely possible to have a working Zettelkasten with the Zettels' contents written in your preferred markup.

Speaking of toolchains: If you're reading this section, probably pandoc is the tool to look at. It might give you a way to convert a mix of YAML and your favourite markup language into a pure file of your favourite markup language. For example, the YAML-LaTeX-mix above can be converted into pure LaTeX by running:

pandoc -f markdown -o output.tex input.tex

Did you notice that we told pandoc to treat it's input as markdown? Yeah, that works. If the output is LaTeX, pandoc will keep all LaTeX markup in place. In fact, the author of libzettels writes his own zettels in a crazy mixture of markdown and LaTeX and it works fine.

One last note: Please note that the requirement above says "should contain", not "must". If no such document is present, libzettels will still write an entry for that file to the index, but it will use default values, which are "empty" values:

---
title: 'untitled'
keywords: []
followups: []
links: []
...

This will not make much sense, because you have no way to change these. But files without a YAML-header can be endpoints of a zettel-to-zettel-relationship. Other Zettels can link to it and can declare it as a followup. But that's it.

Files with an accompanying YAML-metadata file

This approach, has the following requirements:

• Libzettels must ignore your actual zettel file and not try to index it. For non-text file types (i.e. everything that the method lines() of the BufRead-Trait from the Rust standard library cannot be used for), you don't have to do anything. Libzettels will ignore it by default. The same is true for SVG-files. Other XML-based formats need to be filtered out via the ignore file (see here)
• An accompanying YAML-metadata file (like for the image-based zettels) needs to be there, so that libzettels can index it. Also, you'll probably want to have some sort of reference to your actual zettel file (like the image link for image-based zettels). Such a reference could just be another field in the YAML. This is no problem, because libzettels will ignore all YAML-fields but its own ones.

For example, let's say your Zettel of the format foo zettel1.foo is accompanied by zettel1.yaml. Then the contents of zettel1.yaml could look like this:

---
title: 'My Zettel'
keywords: [example]
followups: [another-zettel.yaml]
links: [yet-another-zettel.yaml]
zettel: zettel1.foo
...

Note that links and followups would point to other YAML-metadata files, not to the actual foo files.