Blobasaur is an open source blob storage server that Zerodha built on top of SQLite, and it now runs in production at the broker. Written in Rust and speaking the Redis protocol, it reads and writes JSON blobs from a few kilobytes to several megabytes thousands of times a second on ordinary hardware, with single-digit millisecond latency, according to Zerodha’s engineering blog.
The job it does
Zerodha’s trading platform Kite has around 20 million customers. People save multi-chart layouts, indicator templates, vector drawings and workspaces in its Terminal mode. Backend services store those preferences as opaque blobs, mostly JSON.
The team first used CouchDB, then SSDB, a C++ disk-backed store that speaks the Redis protocol. SSDB worked for years. Two problems pushed a replacement. It became unmaintained, and nobody on the team works in C++ to fork it. It also runs on a single node, which at one point forced a 1 MB cap on blob size because latency rose on larger payloads.

What they ruled out
The requirements were a simple key-value API, multi-megabyte blobs at thousands to tens of thousands of requests a second, horizontal scaling, compression and easy upkeep. The team tested or rejected several options:
- ScyllaDB had lower throughput and higher resource use for large blobs.
- PostgreSQL BYTEA columns performed poorly on speed and disk space.
- YugabyteDB and RocksDB weren’t built for high-throughput multi-megabyte writes.
- Ceph was too complex to even try.
- SeaweedFS came closest, but overwrites kept growing disk use before compaction, and it has no per-blob TTL.
- S3 and MinIO carry high per-request latency and cost, with writes at 50% of traffic.
- One file per blob meant reinventing atomic writes, batching and TTL cleanup.
SQLite’s documentation claims it is up to 35% faster than the filesystem for small blobs. The team also saw 99% of its compressed values land well under 100 KiB, where SQLite beats raw files.

How Blobasaur is built
SQLite in WAL mode allows many readers but only one writer, and writes are about half of Zerodha’s traffic. So Blobasaur splits data across several SQLite files called shards, mapped with multi-probe consistent hashing. Each shard has its own writer task that batches pending writes into one transaction, while reads go to a pool of read-only connections.
In a benchmark with 64 concurrent connections and 64 KiB JSON payloads, 16 shards raised write throughput 3.3x over one shard and cut median write latency from 8.2 ms to 2.1 ms. Reads stayed roughly the same. At peak it handled 25,324 writes a second on a 16-core laptop, about 1.66 GB/s of uncompressed JSON.
Zerodha chose Rust over Go because Go’s SQLite driver needs CGo, which complicates static builds and adds pressure when handling large buffers. The result is a single static binary.

Storage details
Plain keys sit in a blobs table, and hash keys in a table per namespace. Expiry lives on the same row, with a partial index so keys without a TTL cost nothing, and a background worker removes expired rows. Zstd compression runs in production. A 63.6 KiB payload shrinks to 5.5 KiB, which cuts the pages written from about 16 to about 2.
Durability is relaxed on purpose. The store holds UI preferences, not financial data, so a power loss might cost a user a recent chart tweak but never affects trading. Any shard can also be opened with the standard sqlite3 command-line tool.
The migration was also low risk for developers. Several Go and Python services already talked to SSDB over the Redis protocol, so Blobasaur supports the same GET, SET, HGET, HSET, DEL and EXPIRE commands and lets those services stay untouched. The team wanted the same latency for a 2 KB settings blob as for a 5 MB chart layout.
Why it matters for other teams
The lesson is a narrow one. When a workload is mostly small compressed blobs and a Redis-compatible interface is enough, SQLite shards can beat heavier systems. Zerodha’s constraint, that lost recent writes are acceptable, makes this trade possible, so teams storing critical data should weigh it carefully. Blobasaur is open source for anyone who wants to test it.
Source: zerodha.tech