#postgis #duck-db #geospatial #data-transformation #data-file

bin+lib duckdb-postgis

A library for transforming geospatial data using DuckDB and ingesting it into a PostGIS database

10 releases

new 0.1.11 Mar 2, 2025
0.1.10 Mar 2, 2025
0.1.9 Jan 24, 2025
0.1.6 Dec 16, 2024
0.1.5 Oct 22, 2024

#444 in Database interfaces

Download history 5/week @ 2024-11-04 12/week @ 2024-11-11 1/week @ 2024-11-18 36/week @ 2024-12-02 2/week @ 2024-12-09 170/week @ 2024-12-16 15/week @ 2024-12-23 8/week @ 2024-12-30 132/week @ 2025-01-06 42/week @ 2025-01-13 212/week @ 2025-01-20 30/week @ 2025-01-27 23/week @ 2025-02-03

82 downloads per month

MIT license

3MB
537 lines

Rust Library for writing geospatial/non-geospatial data to Postgis using DuckDB

Crates.io

cargo add duckdb-postgis

This now supports python and will soon be available on PyPi.

Current Release v0.1.11

This Rust library does the following things

  • Reads in geospatial data file types (Geopackage, Shapefile, etc)
  • Reads in non geospatial data
  • Performs CRS transformation on the data if required - ensures the CRS is ESPG:4326
  • Loads the data into a PostGIS table with a correctly defined geometry column

Improvements for release 0.1.12

  • Handle raster data file format?
  • Discard rows where there may be errors in the geometry column / ensure the programme doesn't crash when a geometry error is encountered - skip over it and log it instead
  • There are still bugs for loading parquet files and handling some types of geometry columns due to how they are named - these will be fixed in the next release
  • Add flexibility for target CRS.

Example usage

mod duckdb_load;
use duckdb_load::launch_process_file;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    launch_process_file(
        "[add_file_path]",
        "[add_table]",
        "postgresql://admin:password@localhost:5432/[add_db_name]",
        "[add_schema]"
    )?;
    Ok(())
}

This can also be called in Python thanks to Pyo3.

import duckdb_postgis

def test_duckdb_postgis_import():
    """Test function to verify the duckdb_postgis module is working correctly."""
    try:

        # Attempt to process the test file
        duckdb_postgis.process_file(
            "[add_file_path]",
            "[add_table]",
            "postgresql://admin:password@localhost:5432/[add_db_name]",
            "[add_schema]"
        )
        print("File processing completed successfully")
        return True

    except Exception as e:
        print(f"An error occurred: {str(e)}")
        return False

if __name__ == "__main__":
    test_duckdb_postgis_import()

Dependencies

~40MB
~517K SLoC