Documents

A document has:

• A read token, a 32-hexdigit token to access the document.
• A write token, a 32-hexdigit token to modify (write) the document.
• A revision, typically its last modification Unix timestamp in milliseconds since epoch.
• A JSON value.

Application data is typically stored as a series of documents, linked with each other. The application loads these documents by providing their read token, and saves them using their read and write tokens.

Documents are linked with each other by mentioning their read token as #r:

With only the read token, the document is linked read-only. To provide write access, store the corresponding write token as #w:

In both cases cases, the JSON dictionary may contain other keys as well.

All active documents must directly or indirectly be linked from the root document. Documents not linked are garbage collected.

Documents are client-side merged. To perform a transaction, an application downloads the current version of the document, applies the desired changes, and writes resulting new version back to the server. If the document has been modified on the server in the meantime, the server refuses the new version. The client may then retry, or abort the transaction.

Root Document and Garbage Collection

The server maintains a root document. Its read and write tokens are stored in a file called root in the data folder. The root document links all application-specific root documents, and through them the whole (active) data. Only the database administrator should know the root document, since access to it allows to modify or wipe the whole data.

From time to time, a garbage collector sweeps through the data, and removes all documents and blobs not linked to the root document, and not written recently.

If the root document contains a gc key linking a document, the garbage collector will write a short report to this document whenever it completes.

JavaScript

Using Backend.js, a document can be retrieved as follows:

const backend = new Backend('https://viereck.ch/backend');

backend.readDocument(readToken, onDone, onError);

function onDone(data, revision, request) {
	...
}

function onError(errorCode, request) {
	...
}
		

and stored as follows:

backend.writeDocument(readToken, writeToken, derivedFrom, revision, data, onDone, onError);

function onDone(request) {
	...
}

function onError(errorCode, request) {
	...
}
		

HTTP/REST Requests

Read a document

GET /backend/documents/READTOKEN

If-Newer-Than: REVISION

Retrieves a document.

Write a document

PUT /backend/documents/READTOKEN

Write-Token: WRITETOKEN

Derived-From: REVISION

Revision: REVISION

Stores a new version of an existing document, or creates a new document.

To create a new document, the read and write tokens must be picked at random, and the Derived-From revision must be 0. Read and write token cannot be changed.

Read multiple documents

GET /backend/documents?t=READTOKEN/IF-NEWER-THAN,READTOKEN/IF-NEWER-THAN,...

200 OK

READTOKEN ⭢ REVISION ⭢ DOCUMENT

READTOKEN ⭢ REVISION ⭢ DOCUMENT

...

Retrieves a list of documents in a single request. The amount of documents that can be retrieved in a single request depends on the maximum path length the web browser or web server can handle. Older browsers can handle URIs up to 2000 bytes (~40 documents), while newer browsers can handle much longer URIs. NginX is limited to about 8170 bytes (~160 documents) by default, and CDNs are typically have similar limits.

UDP Requests

If UDP is enabled, the following requests are available.

Read a document

Retrieves a document. If the document does not exist, the revision is 0.

If the document is longer than the maximum UDP packet length, the server sends multiple UDP packets. Each of these response packets specifies the document revision and length, as well as the offset of the current slice. With this information, the client can determine if the obtained value is complete. Missing parts may be requested by sending the same request, but specifying the missing range(s).

Write a document

Writes a document. The server responds with:

• "W" if the document was written successfully
• "C" if the derived-from revision was wrong (conflict)
• "F" if the write key was wrong (forbidden)

Note that the maximum document size is limited by the maximum UDP packet size of about 65000 bytes (using packet fragmentation). Larger documents must be written via HTTP.

Listen for changes

Listens for changes for about 2 minutes. To keep listening for changes, renew the request every 2 minutes or less.

Whenever the document changes to a revision newer than the known revision, the first 1200 bytes of the new value are sent to the client. If the document is larger, the client may retrieve the remaining bytes using "r" requests.