Robotap

Publish data as interactive pages, APIs and files, from definitions. A query and a page of plain HTML become an interactive, access-controlled web page, a CSV and an API, from the data you already have.

Open source (MIT). One small Node server. Version 0.x: the shape of the definitions is versioned and migrated, the code around it still changes.

The stretch between a database and a chart

There are excellent places to keep data: PostgreSQL, MongoDB, a folder of Parquet files that DuckDB reads in a blink. There are excellent ways to draw it: d3, Observable Plot, maps. Between them is a stretch nobody owns, and everybody who gets data from the one to the other meets the same list: which query runs where, who may see what, what happens when two people open the page at once, how a value in the address becomes a value in a query without becoming an attack, how a click on a chart narrows the table next to it, how to cache, to print, to embed, to put on a phone, to hand to a spreadsheet.

Each project writes that glue again, and a little worse each time. Robotap is that stretch, written once.

How it works

A dataset is a query: where rows come from and how they are shaped. A report says which datasets a page has, which choices a visitor can make and who may open it. A layout is a page of plain HTML5 with a slot for each dataset. A project is a folder of JSON and HTML, and a change shows on the next reload.

<div data-ds="sales" data-layout="ds/datatable"></div>

The person who knows the data writes the query; the person who knows the look writes HTML and CSS. There is no build step and no framework to learn. Charts and maps are plain JavaScript modules a project adds (a zoomable d3 sunburst is one module and one layout of the demo, with no change to the framework). DuckDB reads CSV and Parquet inside the server, so a file of nine million rows needs no database to host.

The same definition is also a CSV, a JSON endpoint, a print version and a block that embeds in any other web page, under the same access rules.

What it is for

• Watching a website from behindA log file is a data source like any other. A report shows errors, slow pages and the last hour to the people with the right role; no agent to install.
• Publishing research data privatelyA lab shares its data with its own people in a form better than a zip file: a report behind sign-in, the data as CSV, the exact query in the definition.
• Publishing a visualisationAn interactive chart, a searchable table and a map, with the data behind them as a file, embedded in an article with one element, on a server you control.

What it is not

It is not a business-intelligence tool: no drag and drop, no semantic layer, no ad-hoc exploration by people who do not write queries. It is for people who would rather write a definition than click a canvas, and who want the result reviewed, versioned and tested like code. It is not multi-tenant or a hosted service, it has no sign-in with a directory (sign-in is a signed cookie and a user table), and one process is one core: past that, a cache in front or more instances is the answer.

Work with it

Robotap is the software; hosting data solutions and designing data models and reports is the work around it. If you have data that should reach people (a research project, an admin view of a site, a story with a chart), write to hello@robotap.dev.