The Tableland team has been doing deep research the past few months. We’re amidst some new protocol capabilities and recently wrote about how DIMO and WeatherXM plan to use it—check out the blog post we wrote for a little alpha! Throughout this process, we looked at a number of SQLite tools and wanted to pass along some high-level research as we found them quite interesting!

SQLite Tools

The following list defines the projects we came across and various notes about them, including consensus type (Raft, Consul, or N/A), features (sync, replication, recovery, hosted, publish, or GUI), whether or not it’s open sourced, and potential usage highlights/limitations for our use case. It’s not meant to be fully comprehensive, but it just might be useful to folks looking for database products/solutions.

LiteSync

LiteSync allows your applications to easily keep their SQLite databases synchronized.

• Features: replication, sync

• Open Source?: no

• Consensus: Raft

• Usage: Simple to integrate, does replication, sync, and encryption. Full sync between replicas. Directly operate via SQLite.

Marmot

Marmot is a distributed SQLite replicator.

• Features: replication, sync

• Open Source?: yes

• Consensus: Raft

• Usage: Change detection running on NATS (https://nats.io). Appears to support sharding. Uses RAFT via NATS for conflict resolution. Last writer wins. NATS appears to be something like LibP2P, but with connection and scalability tooling built in.

rqlite

rqlite is an easy-to-use, lightweight, distributed relational database, which uses SQLite as its storage engine.

• Features: replication, sync

• Open Source?: yes

• Consensus: Raft

• Usage: Full replication across replicas. Each replica can write, uses RAFT for consistency. However, the API wraps SQLite.

DQLite

Dqlite is a fast, embedded, persistent SQL database with Raft consensus that is perfect for fault-tolerant IoT and Edge devices.

• Features: replication, sync

• Open Source?: yes

• Consensus: Raft

• Usage: Supported by Canonical (folks behind Ubuntu). Wraps SQLite and includes networking and RAFT consensus. While it does sync, it isn’t bi-directional like other projects. This is purely for scaling and still uses a client-server architecture. However, the API wraps SQLite.

LiteFS

LiteFS is a FUSE-based file system for replicating SQLite databases across a cluster of machines.

• Features: replication, sync

• Open Source?:

• Consensus: Raft, Consul

• Usage: Replicating database state across a cluster of replicas. Uses Consul for leader election, and projects like litefs-raft provide things like RAFT.

Litestream

Litestream is a standalone disaster recovery tool for SQLite.

• Features: replication, recovery

• Open Source?: yes

• Consensus: N/A

• Usage: Litestream is a streaming replication tool for SQLite databases. It runs as a separate background process and continuously copies write-ahead log pages from disk to one or more replicas. It does not do sync.

SQLite3 Sync

SQLite3 Sync is a live SQLite3 database master-slave replication with sqlite3-rdiff using rsync over SSH.

• Features: replication, recovery, sync

• Open Source?: yes

• Consensus: N/A

• Usage: Live SQLite3 database master-slave replication using rsync over SSH.

Litereplica

Litereplica allows your applications to easily keep replicas of SQLite databases

• Features: replication, recovery

• Open Source?:

• Consensus: no

• Usage: Litereplica is a library that provides master-slave replication with automatic failover. Appears to be related to the Litesync project.

SymmetricDS

SymmetricDS is open source database replication software that focuses on features and cross platform compatibility.

• Features: replication, sync

• Open Source?: yes

• Consensus: N/A

• Usage: Generic database replication software that focuses on features and cross platform compatibility. This isn’t specific to SQLite per se, but does replicate SQLite as a feature.

Mycelial

Mycelial is the one-click data backbone for Edge AI/ML, Cloud ETL, and everything in between.

• Features: replication, sync

• Open Source?: yes

• Consensus: N/A

• Usage: SQLite extension that connects a remote hub that can sync many readers on your DB.

SQLite Cloud

SQLite Cloud is a is a distributed relational database system that ensures strong consistency across all nodes in a cluster.

• Features: replication, hosted

• Open Source?: no

• Consensus: N/A

• Usage: Fully hosted SQLite DBs with some auto-scaling, auto-replication things built into the dash.

DBHub

DBHub is a Cloud for SQLite databases.

• Features: publish

• Open Source?: yes

• Consensus: N/A

• Usage: Create private and public datasets. Integration for DB Browser for SQLite. CLI interface. Automatic alerts for data changes. Collaborative development.

SQLite Browser

SQLite Browser is a high quality, visual, open source tool to create, design, and edit database files compatible with SQLite.

• Features: GUI, hosted

• Open Source?: yes

• Consensus: N/A

• Usage: Online viewer that integrates with DB Hub. Simple visualization tool for SQLite databases.

Turso

Turso is an edge-hosted, distributed database based on libSQL, an open-source and open-contribution fork of SQLite.

• Features: replication, hosted

• Open Source?: yes

• Consensus: N/A

• Usage: Turso is an edge-hosted, distributed database based on SQLite. Designed to minimize query latency via intelligent geo-replication. A major selling feature is its integration with platforms like CloudFlare, Netlify, and Vercel. Some of it is open source.

Cloudflare D1

Cloudflare D1 is Cloudflare’s native serverless SQL database.

• Features: replication, hosted

• Open Source?: no

• Consensus: N/A

• Usage: D1 is Cloudflare’s native serverless database. It is based on SQLite, and integrates with Cloudflare Workers. (In fact, Tableland’s SDK is fully D1-compatible!)

Awesome Decentralized Database

Awesome Decentralized Database is a curated list of the best decentralized database solutions, tools, resources, and learning materials that empower developers to build data-driven applications on the decentralized web. (Note: if you see anything missing, PRs are welcome!)****

You can check out the repo linked above for the details, but a current snapshot of what we’ve put together is outlined below.

Relational Databases

Blockchain

• AergoLite - AergoLite: SQLite with Blockchain.

• Catena - Catena is a distributed database based on a blockchain, accessible using SQL.

• Daisy - Private proof of authority blockchain where blocks are SQLite databases, in Go.

• Tableland - Decentralized SQLite database that provides provides wallet-based authentication, row-level permissions, and high performance at a low cost.

IPFS

• go-bs-sqlite3 - IPFS blockstore.

• ipfs-sqlite-block-store - IPFS blockstore.

• ipfs-sqlite-cli - CLI utility to backup and restore a database from IPFS. Requires local instance of GO-IPFS running.

Peer-to-Peer

• CR-SQLite - CR-SQLite is a run-time loadable extension for SQLite and libSQL. It allows merging different SQLite databases together that have taken independent writes.

• IceFireDB - IceFireDB-SQLite database is a decentralized SQLite database.

Raft

• ChiselStore - ChiselStore is an embeddable, distributed SQLite for Rust, powered by Little Raft.

• ha-sqlite - High-availability sqlite database service based on raft.

• raft-sqlite - Raft backend using SQLite.

• ReSqlite - ReSqlite is an extension of Sqlite that aims to add basic replication functionality to Sqlite database.

• tqlite - Distributed SQL database with replication, fault-tolerance, tunable consistency and leader election.

Real-time Databases

Peer-to-Peer

• Fireproof - Fireproof is a decentralized realtime database that stores documents using prolly-trees.

• Redwood - Highly-configurable, distributed, realtime database that manages a state tree shared among many peers.

NoSQL Databases

Blockchain

• BigchainDB Blockchain database.

• Bluzelle - Decentralized database for Web 3.0 using blockchain and edge computing for greater performance and security.

• ComposeDB - Decentralized, composable graph database to build interoperable applications on Ceramic.

• The Graph - Indexing protocol for querying networks like Ethereum, IPFS, Polygon, and other blockchains.

Peer-to-Peer

• db3 - Open-source and decentralized firebase firestore alternative for building fully decentralized dApps quickly with minimal engineering effort.

• OrbitDB - Serverless, distributed, peer-to-peer database.

• WeaveDB - NoSQL database powered by Warp Contracts (SmartWeave) on the Arweave blockchain.

Encryption and Security

• FreeSQLiteEncryption - Add-on to the public domain version of SQLite that allows an application to read and write encrypted database files.

• Lit - Create and manage distributed cryptographic key-pairs for condition-based encryption and programmatic signing.

• Threshold Network - Access control with end-to-end encrypted data sharing and communication without the need to trust a centralized authority.

• sqlcipher - Fork of the SQLite database library that adds 256 bit AES encryption of database files and other security features.

• sqlite_codec - sqlite3 with built-in transparent encryption based on mbedtls library.

Learning

• Awesome Database Learning - Another awesome list for database learning.

• Learn GraphQL - Learn real world GraphQL, quickly.

• Learn SQL - Practice and learn SQL from the basics.

Let’s Chat

If you found this interesting, give out a follow on Twitter, check out our GitHub, and drop into our Discord. We’re always down to discuss the latest and greatest in database and blockchain tech! And soon, we’ll be launching our Studio web application to make it easier to develop with Tableland, so sign up for the waitlist if you’re interested in trying it out.