Technology1 min read

Distributed, high-performance blob storage on SQLite

By · Published by Everything Blog

In short

Kite engineered this custom distributed solution to phase out SSDB, an unmaintained single-node database that suffered from latency spikes on payloads exceeding one megabyte. The engineering team reviewed numerous alternatives, including PostgreSQL, ScyllaDB, RocksDB, and SeaweedFS, but each presented severe throughput, mapping, or compaction challenges. Additionally, standard object stores like MinIO and S3 were rejected due to high latencies on smaller items and excessive costs for write-heavy workloads. Blobasaur resolves these issues by delivering a horizontally scalable, disk-efficient key-value store optimized for large payloads.

Key points

  • Blobasaur delivers single-digit millisecond latency across thousands of reads and writes…: Blobasaur delivers single-digit millisecond latency across thousands of reads and writes per second on commodity hardware.
  • Traditional object storage options like MinIO and S3 proved unviable due to high latencie…: Traditional object storage options like MinIO and S3 proved unviable due to high latencies on small payloads and write costs.
  • The distributed platform was tailored to withstand volatile, bursty stock market traffic…: The distributed platform was tailored to withstand volatile, bursty stock market traffic through horizontal scalability.

Sources checked by our analysis pipeline: 1

TLDR; We built Blobasaur, a sharded, distributed, lightweight blob storage server on top of SQLite to solve our high-throughput blob read/write usecase. JSON blobs varying from a few KBs to MBs are now written and read thousands of times per second on commodity hardware while maintaining single-digit millisecond latency.

On Kite, our trading platform with ~20 million customers, users are able to customize their investment and trading screens heavily. They save multi-chart layouts, financial indicator templates, vector drawings, and various kinds of arbitrary UI preferences. In addition, we offer a Terminal mode that allows users to design their own workspaces by dragging, dropping, and customing widgets. Under the hood, our backend services store these user preferences and metadata as opaque blobs (mostly JSON) ranging from a few kilobytes to several megabytes.

Many years ago, we stored such blobs in CouchDB, which turned out to be ill-suited for the kind of throughput we were starting to see. We then moved to SSDB (ssdb), a C++ disk-backed store that speaks the Redis protocol. This made is very easy to integrate as well.…

Technology