49 unstable releases
0.34.2 | Mar 4, 2023 |
---|---|
0.34.0 | Oct 31, 2022 |
0.33.1 | Sep 26, 2022 |
0.32.1 | Jun 15, 2022 |
0.17.0 | Nov 22, 2020 |
#91 in HTTP server
167 downloads per month
3MB
12K
SLoC
atomic-server

Status: alpha. Breaking changes are expected until 1.0.
Atomic-server is a graph database server for storing and sharing Atomic Data. Demo on atomicdata.dev
- ð Fast (less than 1ms median response time on my laptop), powered by actix-web and sled
- ðŠķ Lightweight (8MB download, no runtime dependencies)
- ðŧ Runs everywhere (linux, windows, mac, arm)
- âïļ Dynamic schema validation / type checking using Atomic Schema.
- ð Embedded server with support for HTTP / HTTPS / HTTP2.0 and Built-in LetsEncrypt handshake.
- ðïļ Browser GUI included powered by atomic-data-browser. Features dynamic forms, tables, authentication, theming and more.
- ðū Event-sourced versioning / history powered by Atomic Commits
- ð Synchronization using websockets: communicates state changes with a client.
- ð§° Many serialization options: to JSON, JSON-AD, and various Linked Data / RDF formats (RDF/XML, N-Triples / Turtle / JSON-LD).
- ð Full-text search with fuzzy search and various operators, often <3ms responses. Powered by tantivy.
- ð Pagination, sorting and filtering queries using Atomic Collections.
- ð Authorization (read / write permissions) and Hierarchical structures powered by Atomic Hierarchy
- ðē Invite and sharing system with Atomic Invites
- ð File management: Upload, download and preview attachments.
- ðĨïļ Desktop app: Easy desktop installation, with status bar icon, powered by tauri.
- ð Libraries: Javascript / Typescript, React, Svelte
Powered by Rust, atomic-lib and more.
https://user-images.githubusercontent.com/2183313/139728539-d69b899f-6f9b-44cb-a1b7-bbab68beac0c.mp4
Table of contents
- Table of contents
- When should you use this
- When not to use this
- Installation & getting started
- Initial setup and configuration
- Usage
- FAQ & Troubleshooting
- Can / should I create backups?
- I lost the key / secret to my Root Agent, and the
/setup
invite is no longer usable! What now? - How do I migrate my data to a new domain?
- How do I reset my database?
- How do I make my data private, yet available online?
- Items are missing in my Collections / Search results
- I get a
failed to retrieve
error when opening - Can I embed Atomic-Server in another application?
- Where is my data stored on my machine?
When should you use this
- You want a powerful, lightweight, fast and easy to use CMS or database with live updates, editors, modelling capabilities and an intuitive API
- You want to build a webapplication, and like working with using React or Svelte.
- You want to make (high-value) datasets as easily accessible as possible
- You want to specify and share a common vocabulary / ontology / schema for some specific domain or dataset. Example classes here.
- You want to use and share linked data, but don't want to deal with most of the complexities of RDF, SPARQL, Triple Stores, Named Graphs and Blank Nodes.
- You are interested in re-decentralizing the web or want want to work with tech that improves data ownership and interoperability.
When not to use this
- High-throughput numerical data / numerical analysis. Atomic-Server does not have aggregate queries.
- If you need high stability, look further (for now). This is beta sofware and can change.
- You're dealing with very sensitive / private data. The built-in authorization mechanisms are relatively new and not rigorously tested. The database itself is not encrypted.
- Complex query requirements. We have queries with filters and features for path traversal, but it may fall short. Check out NEO4j, Apache Jena or maybe TerminusDB.
Installation & getting started
You can run atomic-server
in five ways:
- Using docker (probably the quickest):
docker run -p 80:80 -p 443:443 -v atomic-storage:/atomic-storage joepmeneer/atomic-server
- Install a desktop build (macOS only as of now)
- From a published binary
- Using Cargo from crates.io:
cargo install atomic-server
- Manually from source
When you're running atomic-server
, go to Initial setup and configuration.
If you want to run this locally as a developer / contributor, check out the Contributors guide.
1. Run using docker
- Run:
docker run -p 80:80 -p 443:443 -v atomic-storage:/atomic-storage joepmeneer/atomic-server
Thedockerfile
is located in the project root, above thisserver
folder. - If you want to make changes (e.g. to the port), make sure to pass the relevant CLI options (e.g.
--port 9883
). - If you want to update, run
docker pull joepmeneer/atomic-server
and docker should fetch the latest version.
2. Install desktop build (macOS only)
We automatically build .dmg
installers for MacOS. You can download them from the releases page.
3. Run pre-compiled binary
Get the binaries from the releases page and copy them to your bin
folder.
4. Install using cargo
# Install from source using cargo, and add it to your path
# If things go wrong, check out `Troubleshooting compiling from source:` below
cargo install atomic-server --locked
# Check the available options and commands
atomic-server --help
# Run it!
atomic-server
5. Compile from source
git clone git@github.com:atomicdata-dev/atomic-data-rust.git
cd atomic-data-rust/server
cargo run
If things go wrong while compiling from source:
# If cc-linker, pkg-config or libssl-dev is not installed, make sure to install them
sudo apt-get install -y build-essential pkg-config libssl-dev --fix-missing
Initial setup and configuration
- You can configure the server by passing arguments (see
atomic-server --help
), or by setting ENV variables. - The server loads the
.env
from the current path by default. Create a.env
file from the default template in your current directory withatomic-server generate-dotenv
- After running the server, check the logs and take note of the
Agent Subject
andPrivate key
. You should use these in theatomic-cli
and atomic-data-browser clients for authorization. - A directory is made:
~/.config/atomic
, which stores your newly created Agent keys, the HTTPS certificates other configuration. Depending on your OS, the actual data is stored in different locations. See use theshow-config
command to find out where, if you need the files. - Visit
http://localhost:9883/setup
to register your first (admin) user. You can use an existing Agent, or create a new one. Note that if you create alocalhost
agent, it cannot be used on the web (since, well, it's local).
Running using a tunneling service (easy mode)
If you want to make your atomic-server available on the web, but don't want (or cannot) deal with setting up port-forwarding and DNS, you can use a tunneling service. It's the easiest way to get your server to run on the web, yet still have full control over your server.
- Create an account on some tunneling service, such as tunnelto.dev (which we will use here). Make sure to reserve a subdomain, you want it to remain stable.
tunnelto --port 9883 --subdomain joepio --key YOUR_API_KEY
atomic-server --domain joepio.tunnelto.dev --custom-server-url 'https://joepio.tunnelto.dev' --initialize
HTTPS Setup on a VPS (static IP required)
You'll probably want to make your Atomic Data available through HTTPS on some server. You can use the embedded HTTPS / TLS setup powered by LetsEncrypt, acme_lib and rustls.
You can do this by passing these flags:
Run the server: atomic-server --https --email some@example.com --domain example.com
.
You can also set these things using a .env
or by setting them some other way.
Make sure the server is accessible at ATOMIC_DOMAIN
at port 80, because Let's Encrypt will send an HTTP request to this server's /.well-known
directory to check the keys.
The default Ports are 9883
for HTTP, and 9884
for HTTPS.
If you're running the server publicly, set these to 80
and 433
: atomic-server --https --port 80 --port-https 433
.
It will now initialize the certificate.
Read the logs, watch for errors.
HTTPS certificates are automatically renewed when the server is restarted, and the certs are 4 weeks or older.
They are stored in your .config/atomic/
dir.
HTTPS Setup using external HTTPS proxy
Atomic-server has built-in HTTPS support using letsencrypt, but there are usecases for using external TLS source (e.g. Traeffik / Nginx / Ingress).
To do this, users need to set these ENVS:
ATOMIC_DOMAIN=example.com
# We'll use this regular HTTP port, not the HTTPS one
ATOMIC_PORT=80
# Disable built-in letsencrypt
ATOMIC_HTTPS=false
# Since Atomic-server is no longer aware of the existence of the external HTTPS service, we need to set the full URL here:
ATOMIC_SERVER_URL=https://example.com
Using systemd
to run Atomic-Server as a service
In Linux operating systems, you can use systemd
to manage running processes.
You can configure it to restart automatically, and collect logs with journalctl
.
Create a service:
nano /etc/systemd/system/atomic.service
Add this to its contents, make changes if needed:
[Unit]
Description=Atomic-Server
#After=network.targetdd
StartLimitIntervalSec=0[Service]
[Service]
Type=simple
Restart=always
RestartSec=1
User=root
ExecStart=/root/atomic-server
WorkingDirectory=/root/
EnvironmentFil=/root/.env
[Install]
WantedBy=multi-user.target
# start / status / restart commands:
systemctl start atomic
systemctl status atomic
systemctl restart atomic
# show recent logs, follow them on screen
journalctl -u atomic.service --since "1 hour ago" -f
Usage
There are three ways to interact with this server:
- GUI: Use the
atomic-data-browser
JS frontend by visitinglocalhost:9883
. - API: We have a subset of the API documented using Swagger / OpenAPI. Check out ./_requests.http for various HTTP requests to the server. Also, read the Atomic Data Docs. You can also try the react boilerplate to build your own front-end app using @tomic/lib and @tomic/react.
- CLI: The
atomic-cli
terminal app
Using Atomic-Server with the browser GUI
Open your server in your browser.
By default, that's http://localhost:9883
.
Fun fact: ⚛
is HTML entity code for the Atom icon: â.
The first screen should show you your Drive. You can think of this as your root folder. It is the resource hosted at the root URL, effectively being the home page of your server.
There's an instruction on the screen about the /setup
page.
Click this, and you'll get a screen showing an Invite.
Normally, you could Accept as new user
, but since you're running on localhost
, you won't be able to use the newly created Agent on non-local Atomic-Servers.
Therefore, it may be best to create an Agent on some other running server, such as the demo Invite on AtomicData.dev.
And after that, copy the Secret from the User settings
panel from AtomicData.dev, go back to your localhost
version, and press sign in
.
Paste the Secret, and voila! You're signed in.
Now, again go to /setup
. This time, you can Accept as {user}
.
After clicking, your Agent has gotten write
rights for the Drive!
You can verify this by hovering over the description field, clicking the edit icon, and making a few changes.
You can also press the menu button (three dots, top left) and press Data view
to see your agent after the write
field.
Note that you can now edit every field.
You can also fetch your data now as various formats.
Try checking out the other features in the menu bar, and check out the collections
.
Use atomic-cli
as client
atomic-cli
is a useful terminal tool for interacting with atomic-server
.
It makes it easy to query and edit Atomic Data from the command line.
Check it out.
API
You can fetch individual items by sending a GET request to their URL.
# Fetch as JSON-AD (de facto standard for Atomic Data)
curl -i -H "Accept: application/ad+json" https://atomicdata.dev/properties/shortname
# Fetch as JSON-LD
curl -i -H "Accept: application/ld+json" https://atomicdata.dev/properties/shortname
# Fetch as JSON
curl -i -H "Accept: application/json" https://atomicdata.dev/properties/shortname
# Fetch as Turtle / N3
curl -i -H "Accept: text/turtle" https://atomicdata.dev/properties/shortname
Check out ./example_requests.http for more things that you can do. We have a subset of the API documented using Swagger / OpenAPI. Also, read the Atomic Data Docs to learn more about Collections, Commits, JSON-AD and other concepts used here.
FAQ & Troubleshooting
Can / should I create backups?
You should. Especially before installing a newer Atomic-Server version, as it might be imcompatible with the previous database model and could corrupt the database.
Run atomic-server export
to create a backup in your ~/.config/atomic/backups
folder.
Import them using atomic-server import -p ~/.config/atomic/backups/${date}.json
.
I lost the key / secret to my Root Agent, and the /setup
invite is no longer usable! What now?
You can run atomic-server --initialize
to recreate the /setup
invite. It will be reset to 1
usage.
How do I migrate my data to a new domain?
There are no helper functions for this, but you could atomic-server export
your JSON-AD, and find + replace your old domain with the new one.
This could especially be helpful if you're running at localhost:9883
and want to move to a live server.
How do I reset my database?
Remove the db
folder in your atomic
config: rm -rf ~/.config/atomic/db
.
How do I make my data private, yet available online?
You can press the menu icon (the three dots in the navigation bar), go to sharing, and uncheck the public read
right.
See the Hierarchy chapter in the docs on more info of the authorization model.
Items are missing in my Collections / Search results
You might have a problem with your indexes.
Try rebuilding the indexes using atomic-server --rebuild-index
.
Also, if you can, recreate and describe the indexing issue in the issue tracker, so we can fix it.
I get a failed to retrieve
error when opening
Try re-initializing atomic server atomic-server --initialize
.
Can I embed Atomic-Server in another application?
Yes. This is what I'm doing with the Tauri desktop distribution of Atomic-Server.
Check out the desktop
code for an example!
Where is my data stored on my machine?
It depends on your operating system, because some data is temporary, others are configuration files, and so forth. Run atomic-server show-config
to see the used paths. You can overwrite these if you want, see --help
.
Dependencies
~44â81MB
~1.5M SLoC