The State of Vector Search in SQLite

Making vector search fast, memory-efficient, and natural in SQLite.

I usually don’t like to reinvent the wheel, but sometimes the available tools don’t quite fit. Recently, while working with vector data in SQLite, I noticed that the current ecosystem doesn’t fully align with what most SQLite users actually need.

The typical requirements are not “billion-scale” datasets or cloud-only setups. Most developers want something that is:

• Memory-efficient (able to run comfortably on laptops, phones, and small servers).

• Fast enough to handle a few million multidimensional vectors.

• Simple to use within regular SQLite workflows.

When you look at the current offerings, there’s a gap.

Existing Options

• sqlite-vss (Alex Garcia)
  Built on Meta’s Faiss C++ library, but plagued by integration issues that ultimately led Alex to abandon it. He explains why here.

• sqlite-vec (Alex Garcia)
  A clean re-implementation in C, easier to maintain, exposing a brute-force search strategy. It works via virtual tables, which means vectors must live in separate tables and queries become more complex. (This is the one we first adopted and even sponsored.)

• libsql (Turso)
  Uses the HNSW DiskANN algorithm, a powerful choice, but the indexing phase can take hours. For many SQLite use cases, that indexing overhead is simply not practical. (It turns out that indexing is optional in libSQL, thanks https://substack.com/@penberg for the correction)

---

Why Try Something Different?

These projects are valuable experiments, but they didn’t solve the constraints we were facing: speed, memory limits, and natural SQL usage. Out of that frustration, we started prototyping an alternative, which eventually became sqlite-vector.

The idea was simple:

• Use a brute-force-like approach, but highly optimized.

• Implement hardware-specific distance functions (auto-selected at runtime).

• Support quantization, preloading into memory, and multiple data types (FLOAT32, FLOAT16, BFLOAT16, INT8, UINT8).

• Allow vectors to be stored in ordinary tables, with no special virtual tables required.

---

Benchmark Exploration

To understand the trade-offs, we ran tests with 100,000 vectors of dimension 384 (FLOAT32) on an Apple M1 Pro MacBook Pro (2021, 16GB RAM).

sqlite-vec (100,000 rows)

Insert-time:          1179.07 ms
Full-scan query time:   67.84 ms

sqlite-vector (100,000 rows)

Insert-time:          563.04 ms
Full-scan query time:  56.65 ms

8-bit quantization:   224.85 ms (one-time)
Quant-scan query:      17.44 ms
Quant-preload query:    3.97 ms
Quant-mem usage:       37.38 MB
Recall@20:             1.0000 (20/20) 

(We couldn’t complete the libsql test, as index creation ran for hours without finishing.)

All tests for sqlite-vector were run with a 30 MB memory cap.

---

Observations

• Insert time is ~50% faster than sqlite-vec.

• Plain query time is ~16% faster.

• With quantization, queries become 3× faster.

• With quantization + preload, queries run in under 4 ms, roughly 17× faster than sqlite-vec, while still achieving perfect recall.

• Memory usage stays low (37 MB for 100k vectors).

Perhaps most importantly, the queries do not need complex JOIN statements to retrieve all information bound to embeddings:

-- Create a regular table
CREATE TABLE images (
  id INTEGER PRIMARY KEY,
  embedding BLOB, -- store Float32/UInt8/etc.
  label TEXT
);

-- Run a nearest neighbor query on the quantized data
SELECT e.id, v.distance
FROM images AS e
JOIN vector_quantize_scan('images', 'embedding', '[12,22,11,243,...]', 20) AS v
  ON e.id = v.rowid;

---

Closing Thoughts

Vector search is becoming an essential feature in many modern applications; yet, the SQLite ecosystem lacks an extension that meets the everyday needs of its community. Most developers don’t need billion-scale indexes or hours-long indexing jobs. What they need is something lightweight, memory-efficient, and fast enough to handle a few million vectors, all while feeling like “regular SQLite.”

The SQLite ecosystem already has a few approaches to vector search, but none yet feel fully aligned with the everyday needs of SQLite developers: reasonable scale, fast enough queries, efficient memory use, and seamless SQL integration.

We believe the community could benefit from sqlite-vector, which is why we’ve made it entirely free for open-source projects.

If you’re experimenting with vector search in SQLite, I’d love to hear your experiences, whether with sqlite-vector, sqlite-vec, libsql, or your own experiments.

---

I’m the founder of SQLite AI, our mission is to make local-first sync with SQLite and AI on the Edge straightforward and reliable. If you’d like to experiment with a production-ready vector engine, the same kind of technology discussed in this article, you can explore it at: https://www.sqlite.ai/sqlite-vector