A tool to create visual and interactive HTML reports from collections of CSV/TSV tables. Reports include automatically generated vega-lite histograms per column. Plots can be fully customized by users via a config file. These also allow the user to add linkouts to other websites or link between multiple tables. An example report can be viewed online with the corresponding config file.

Usage

datavzrd config.yaml --output results/

Configuring datavzrd

datavzrd allows the user to easily customize it's interactive HTML report via a config file. When generating large reports, templating yaml files can be a bit tricky. We advise using yte for easy yaml templating with python expressions.

name: My beautiful datvzrd report
datasets:
  table-a:
    path: "table-a.csv"
    links:
      gene details:
        column: gene
        view: "gene-{value}"
      gene expression:
        column: gene
        table-row: table-b/gene
        optional: true
  table-b:
    path: table-b.csv
    separator: ;
  gene-mycn:
    path: "genes/table-mycn.csv"
    headers: 2
    links:
      some expression:
        column: quality
        view: table-b
views:
  table-a:
    dataset: table-a
    desc: |
      # A header
      This is the **description** for *table-a*.

      ```latex
      \sum_{i=1}^n i^3 = \left( \frac{n(g(n)+1)} 2 \right) ^2
      ```
    render-table:
      columns:
        x:
          custom: |
            function(value, row) {
              // Access cell value or any other value of the same row (by row.<colname>)
              // to render custom content.
              return `<b>${value}</b>`;
            }
        y:
          link-to-url: 
            Ensembl: 
              url: 'https://www.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g={value}'
  table-b:
    dataset: table-b
    render-table:
      columns:
        significance:
          optional: true
          custom-plot:
            data: |
              function(value, row) {
                // Generate data for this cell, having access to the value and any other values from
                // the same row.
                return [{"significance": value, "threshold": value > 60}]
              }
            spec: |
              {
                "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
                "mark": "circle",
                "encoding": {
                  "size": {"field": "significance", "type": "quantitative", "scale": {"domain": [0,100]}},
                  "color": {"field": "threshold", "scale": {"domain": [true,false]}}
                },
                "config": {"legend": {"disable": true}}
              }
  gene-mycn:
    dataset: gene-mycn
    page-size: 40
    render-table:
      columns:
        frequency:
          plot:
            ticks:
              scale: linear
        quality:
          plot:
            heatmap:
              scale: linear
              range:
                - green
                - red
  gene-mycn-plot:
    dataset: gene-mycn
    render-plot:
      spec: |
        {
          "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
          "mark": "circle",
          "encoding": {
            "size": {"field": "significance", "type": "quantitative", "scale": {"domain": [0,100]}},
            "color": {"field": "threshold", "scale": {"domain": [true,false]}},
            "href": {"field": "some expression"}
          },
          "config": {"legend": {"disable": true}}
        }

name

name allows the user to optionally set a name for the generated report that will be heading all resulting tables and plots.

default-view

default-view allows the user to optionally specify a view for the generated report that will be shown when opening the index file.

max-in-memory-rows

max-in-memory-rows defines the threshold for the maximum number of rows in memory. If the given dataset exceeds the threshold the data will be split across multiple pages and their html files. Defaults to 1000 rows.

aux-libraries

aux-libraries allows to add one or more js libraries via cdn links for usage in render-html. The keyword expects a list of urls that link to the js libraries.

webview-controls

webview-controls allows to turn on sharing individual rows via a link with the data encoded into a url pointing to a webview that hosts a static version of datavzrd at https://datavzrd.github.io. Note that when using the link the row data can temporarily occur (in base64-encoded form) in the server logs of the given webview host. Defaults to false.

datasets

datasets defines the different datasets of the report. This is also the place to define links between your individual datasets.

views

views consists of all different CSV/TSV views (table or plot) that should be included in the resulting report. If neither render-table nor render-plot is present, datavzrd will render the given file as a table. Each item definition can contain these values:

render-table

render-table contains definitions for a table view

columns

columns contains individual configurations for each column that can either be adressed by its name defined in the header of the CSV/TSV file, its 0-based index (e.g. index(5) for the 6th column), a range expression (e.g. range(5, 10) for the 5th column to 9th column) or a regular expression (e.g. "regex('prob:.+')" for matching all columns starting with prob:):

add-columns

add-columns allows to generate new columns out of the existing dataset.

headers

headers contains definitions for additional header rows. Each row can be accessed with its index starting at 1 (0 is the first header row that can't be customized).

render-plot

render-plot contains individual configurations for generating a single plot from the given CSV/TSV file.

render-html

render-html contains individual configurations for generating a single custom view where a global variable data with the dataset in json format can be accessed in the given js file. The rendered view contains a <div id="canvas"> that can then be manipulated with the given script. By default, the div uses the full width and centers its contents. Of course, the divs CSS can be overwritten via Javascript. jQuery is already available, any other necessary Javascript libraries can be loaded via aux-libraries.

links

links can configure linkouts between multiple items.

custom-plot

custom-plot allows the rendering of customized vega-lite plots per cell.

link-to-url

link-to-url allows rendering a link to a given url with {value} replaced by the value of the table.

plot

plot allows the rendering of either a tick-plot for numeric values or a heatmap for nominal values.

ticks

ticks defines the attributes of a tick-plot for numeric values.

heatmap

heatmap defines the attributes of a heatmap for numeric or nominal values.

bars

bars defines the attributes of a bar-plot for numeric values.

color

color defines the attributes of a color scale definition for tick and bar plots

Authors

• Johannes Köster (https://koesterlab.github.io)
• Felix Wiegand