#postgresql #gtfs #feed #real-time #import #back-end

bin+lib catenary-backend

import and backend for gtfs data

2 releases

0.1.1 Mar 9, 2024
0.1.0 Mar 6, 2024

#907 in Database interfaces

Custom license and AGPL-3.0

460KB
9K SLoC

Rust 8K SLoC // 0.0% comments Python 1K SLoC // 0.1% comments Shell 57 SLoC // 0.2% comments Bitbake 55 SLoC // 0.4% comments

Contains (Zip file, 18KB) transitland-atlas-vscode-extension-0.0.2.vsix

Install Dependencies

sudo apt install protobuf-compiler build-essential gcc pkg-config libssl-dev postgresql unzip wget

Loading in Data

Loading in data into the Postgres database is a multistep process. Ensure your postgres database is working and your password is set correctly.

Download the Transitland repo

Transitland acts as an initial source of knowledge for Catenary-Backend, and associates static feeds and realtime feeds together. Download and Update it via:

git submodule init && git submodule update

If you already have it, remember to git pull / merge changes To do this, cd into the folder transitland-atlas and run git pull

Download GTFS static data

This downloads the world's GTFS Static Data. This step may take a while, so go play some Minecraft / touch grass and come back when it's all finished!

cargo run --release --bin transitlanddownload

Unzip and format the zip files

./src/ingest_gtfs_schedule/unzip-statics.sh

Import data into the postgres database

cargo run --release --bin import -- --postgres "host=localhost user=postgres password=correcthorsebatterystaple" --threads 25 --startfresh true --isprod false

This command writes to gtfs_stage. Omit startfresh if you would want to wipe the staging directory.

For safety reasons, you are unable to wipe the gtfs schema, which is the production database, from this version.

You can also write to production, especially loading in a single agency, like this.

cargo run --release --bin import -- --postgres "host=localhost user=postgres password=correcthorsebatterystaple" --threads 25 --startfresh false --limittostaticfeed f-9q9-caltrain --isprod true

Moving staging to be the new production database.

Moving the gtfs_stage set of tables to gtfs is really simple

cargo run --bin move_to_prod -- --postgres "host=localhost user=postgres password=correcthorsebatterystaple"

Move to prod deletes the current production database, renames the staging and then commits the change.

You're all done! Data is fully ready for serving to users!

Running the Application

Install Systemd Service

sudo cp transitbackend.service /etc/systemd/system/transitbackend.service
sudo systemctl daemon-reload
sudo systemctl enable --now transitbackend.service

Example endpoints

http://localhost:5401/getroutesperagency?feed_id=f-9mu-orangecountytransportationauthority

http://localhost:5401/gettrip?feed_id=f-9mu-orangecountytransportationauthority&trip_id=10995882

For Contributors

For unix users, running git config core.hooksPath .githooks is required. Pull requests will not be merged without this.

No option exists for Windows users at the moment. Please try WSL Ubuntu for the moment. We're working on adding this.

Dependencies

~86MB
~1.5M SLoC