Hybrid Schemaless Databases

Created: August 4, 2020

TL;DR - Hydrid relational schemaless databases are quite awesome, with a little bit flowers in the garden.

True schemaless databases came into my radar with MongoDB, Redis and such. I have often admired the ability to alter schema on fly and enjoyed experimenting with the possibilities.

Enterprise solutions like Couchbase and Firebase have been big successes (even the now defunct Cparse).

Why?

The success of schemaless databases makes a lot of sense in part because of API consumption of JSON and mobile devices if you can get the synchronization part right.

At a startup I worked at, I experimented with a hybrid relational database and schemaless using JSON extensions for SQLite - the industry de facto mobile database.

This was quite exciting to enable serving queryable and dynamic data without changing to much of the architecture.

Using JSON as a single column for data has some interesting features for hashing to detect changes easier.

The JSON merge patch RFC makes it simple to keep the data integrity.

Recently

Not too long ago an article from Uber engineering described how they transitioned from postgres to a hybrid schemaless using MySQL and JSON.

Postgres also has JSON extensions for querying and storage, but I believe Uber went with MySQL for some replication or scalability features needed.

The JSON data types where introduced a while back and have been around for quite a few years. They could definitely be considered a stable feature.

Micrantha

This is very inline with what I am attempting at micrantha labs.

Garden as is is called, is currently a extension that makes it easier to perform basic CRUD using schemaless JSON on postgres (or any JSON extended database).

The Pathway is a mobile synchronization piece leveraging industry standard protocols in IoT and focusing on data.

The Nursery is the web frontend for users.

Onward and Upwards

I love the flexibility of schemaless with the tried and tested features of a relational database in one package.

Using IoT network protocols saves a lot of time with good features and experience. They do make sense from a mobile phone device (a thing with internet).

Even potentially moving towards progressive web apps for mobile, the storage api or indexedDB should be able to grow nicely the garden.

Disclaimer

The reasoning is frankly, is my day jobs typically do not allow me to flex technically. And because I am a hugely egoic when it comes to technology.

Have always preferred programming, data, protocols and algorithms to user interface. Which is so ironic to a UI focused career.

A complex and daunting project with little reward and a fun endeavour. Micrantha is just a facade to justify my pandemic time.

As they say in the music industry: “do not quit your day job!”


comments powered by Disqus