PhotoBook

A rust command line tool to harvest, organize, download from clouds and backup your photos.
Packaged with an Elasticsearch database and Kibana as consultation interface.

Installation

• the git way

  • First Clone this repo.

  • Adapt the code and configurations as needed.

  • Then run

      $ cargo build --release
    

    The PhotoBook bin will be available under the target/release directory of the project.

  • Finally, don't forget to fulfill the config photobook.toml to get an operational tool (see below).

• the cargo way

  This tool is packaged and published on crates.io.
  Simply install it on your system with cargo !

    $ cargo install photobook
  

  Note : beware to copy your config file described below where your system attempts it.
  For example, on a linux system, the photobook.toml must land on /home/<your user>/.config/photobook/.

Features

PhotoBook proposes 4 main features:

• Harvest

  This command creates an inventory of all the pictures referenced in a source, as a filesystem or a flickr repository, and stores it in an elasticsearch database.
  Then you can consult, request those documents as you need, to create by your own albums, favorites, and more.

  Harvest command allows 3 operations:

  • initialize to (re)create a proper index based on the pictures from a source : $ photobook harvest -o initialize -s flickr

  • update to maintain your index up to date (the default operation)

      $ photobook harvest -o update -s flickr
      $ photobook harvest -s flickr // update is the default operation, so could be omit            
    

  • clean to remove from your index deleted pictures into the source

      $ photobook harvest -o clean -s flickr
    

  and 2 sources:

  • local to inventory from a path (mandatory) on your local filesystem

      $ photobook harvest -s local -p '/home/comtelabs/pictures/'
    

  • flickr connecting a flickr account and referencing all photos stored on its repository whatever their status is

      $ photobook harvest -o initialize -s flickr
    

• Organize

  This command performs a structured and supervised copy of pictures from a directory to another.
  Organize command thus expects 2 mandatory arguments:

  • from defining the source path
  • to defining the target path

  The process only operates pictures located into the source directory, filtering files by their type.
  It orders and sorts pictures by year, month and date, then creates a tree of directories based on this sorting architecture, and finally stores them according to these criteria.

  Command examples:

     $ photobook organize --from '/home/comtelabs/pictures/' --to '/data/pictures/'
     $ photobook organize -f '/home/comtelabs/pictures/' -t '/data/pictures/'
  

• Download

  This command is dedicated to cloud photo sources, and as the name suggests, download photo from a cloud to a local path.
  To use it, you will need to harvest first the cloud chosen: the PhotoBook database references all photos stored on the cloud, and follows photo lives.

  Download command proposes one source (Flickr at this time) and need a target path to save your photos. Note iCloud is on the road map !
  And allows 2 modes of download:

  • full to download all photos owned by the flickr user configured that have not been downloaded before whatever their flickr status is (the default feature).

      $ photobook -t '/home/comtelabs/pictures/'
    

  • server to download all photos stored on a defined flickr server, and that have not been downloaded before, whatever their flickr status is. Note you can find the server list in the dedicated index through a simple query on Kibana.

      $ photobook -t '/home/comtelabs/pictures/' -s 936
    

• Backup

  Not released yet

Requirements

PhotoBook works with an Elasticsearch database and Kibana as database consultation tool.
ELK stack offers powerful applications and tools for free, if you have simple needs.

2 major advantages for an individual developer of a tool dedicated to privacy usage are:

• easy deployment through dockers - but you can host an elastic server too if you want one
• ready to use consultation tool with Kibana - and with a large set of features and really huge capacities

How to get elastic ?

• For those who don't want to pass a lot of time to create their own database, and don't care about how it is stored, you will find a docker-compose.yml file on the root of this repo.
• For the others, feel free to mount your own elastic stack, on a local server, a VPS, or else !
  You will find here the elastic documentation. Note that PhotoBook only needs Elasticsearch and Kibana.

How to use elastic dockers ?

• ensure docker and docker-compose are installed on your machine

  • if not, please refer to the docker and docker-compose install documentation, according to your system.

• and then simply run and re-run each time you need it

    $ docker-compose down && docker-compose up -d 
  

• your elastic database will be listened to http://localhost:9200

• and especially kibana will be available from any browser at http://localhost:5601

Isn't it simply great ?

Configuration

Configurations are stored into 2 files:

• a constant.rs file you will have to create on the project root.
• a photobook.toml file that is create by default on the user config directory of your system.
  • For example, on a linux system, the photobook.toml must land on /home/<your user>/.config/photobook/.
  • If you don't know where the file should be stored, just launch the photobook command, it will create an empty default file that you will be able to find, searching about photobook.toml on your filesystem.
  • Do know more about this feature, please refer to the directories library documentation.

A constant_sample.rs is provided in the /src directory of this public repo to present expected configs:

• Elasticsearch

    pub const ELASTIC_URL: &str = "http://localhost:9200";
    pub const ELASTIC_PICTURES_INDEX: &str = "pictures";
    pub const ELASTIC_FLICKR_PHOTOS_INDEX: &str = "flickr_photos";
  

• Flickr

    pub const FLICKR_API_URL: &str = "https://api.flickr.com/services/rest";
    pub const FLICKR_NONCE: &str = "my_flickr_nonce";
  

A photobook_sample.toml is provided on the root of this repo, and will expect those keys:

 flickr_key = '<my flickr key>'
 flickr_secret = '<my flickr secret>'
 flickr_oauth_token = '<my oauth token>'
 flickr_oauth_secret = '<my oauth secret>'

Note : PhotoBook uses the last Flickr API based on OAuth 1.0.
Please refer to Flickr API documentation to get or create your tokens and keys.

Usage

Some common usages have been described above, widely using the short form of argument declaration.
For further explain please consult the CLI help for more: help is provided by a -h or --help mention after any command or sub-command.
For example:

$ photobook --help
$ photobook harvest -h  

Contribute

PhotoBook is an open source project, so feel free to use, re-use, amend, enrich it and more.
The major requirement, according to the CeCILL license (GNU compatible), is to share your evolution and modification with the author.
If so, please just send your code here !

Road map

• Backup feature
• Chain operation often played in a row, for example : update the harvest of a cloud source, download the new photos from this source, backup the directory.
• iCloud harvesting and downloading options
• Simple web interface to consult the pictures index and dig into

Licence note

Copyright Silvère Camponovo
Code and program published under CeCILL v2.1 license. Please read LICENCE file before any use.