#dwarf-fortress #json-parser #parser #json

dfraw_json_parser

Library which parses Dwarf Fortress raw files into JSON

38 releases (15 breaking)

0.17.5 May 12, 2024
0.16.0 Dec 9, 2023
0.15.1 Nov 29, 2023
0.12.0 Jan 24, 2023
0.2.1 Aug 9, 2022

#2 in #dwarf-fortress


Used in dfraw_json_parser-cli

MIT license

1MB
20K SLoC

Rust 18K SLoC // 0.0% comments TypeScript 2K SLoC // 0.1% comments

dfraw_json_parser

There are generated typescript type definitions in the bindings directory.

Existing Projects

These are projects which use this library.


lib.rs:

dfraw_json_parser provides a way to turn raw files from Dwarf Fortress into JSON. It's currently useful for getting some basic information from from a limited set of raw types:

  • creatures
  • plants
  • inorganics (rocks, ores, etc)
  • materials
  • syndromes
  • graphics (sprites and tile pages, some layer support)
  • limited template support, creature variations support, and select XX support

How It Works

It functions using a regular expression to break apart the tokens and then checks the key in the token against a long list of ones it knows about. Any matches are utilized to build its own representation of the raw. Optionally you can turn this result into JSON using the serde_json library. Or I guess turn it into anything serde supports.

Tauri Support

This library was built to provide raw parsing for another project, Overseer's Reference Manual, which creates a tauri app that lets the user look at the raws on their machine in a searchable and filterable manner. The "tauri" feature flag enables functions which will emit parsing progress back to the tauri window.

Glossary of Terms

Raw

A raw is a definition used by Dwarf Fortress to create creatures, things and objects in the world. They can be used to also define how things interact. They are composed of a variety of tokens inside a raw file, starting with an identifier token of [#type#:#identifier#]. Raws can also select and modify other raws using a [SELECT:#identifier#] token.

Raw File

A raw file is a single text file (with .txt extension) which has the first line as the raw file identifier, and then defines the type of raw file using the [OBJECT:#type#] token. The file can contain any number of raws of that type.

Raw File Module, Raw Module

Since Dwarf Fortress 50, raws are organized into directories. (Previously they were in a flat structure.) This was done to facilitate the steam workshop.

Raw Module Location, Module Location

Raw file module folders are located in 3 locations that this library recognizes:

  • $df_game_dir/data/vanilla: the vanilla raws are here
  • $df_game_dir/data/installed_mods: the workshop mods which have been used in generating a world are here. They are considered installed because they are in use by at least one world. (Note: these are never removed even if you delete all worlds which use them.)
  • $df_game_dir/mods: the workshop mods are stored here to be available when creating a world

Info Text File

In each raw module, there is an info.txt file which contains information about that module, like author, version, earliest compatible version (for updating in existing saves), and information for the steam workshop if it is a mod downloaded from the steam workshop.

Dependencies

~9–50MB
~795K SLoC