liteparser: a fast, embeddable SQLite parser
A complete SQLite parser you can embed anywhere.
While flying to San Francisco a few months ago, I decided to attempt something I had wanted for years: building a complete parser for SQLite:
The ability to fully understand a SQLite statement programmatically unlocks many interesting possibilities: query analysis, SQL transformations, tooling, and AI systems that interact with databases.
Unfortunately, building a SQLite parser is surprisingly difficult.
SQLite’s grammar has evolved for more than 20 years and includes many subtle rules around expression precedence, optional syntax, and compatibility quirks. The official parser is generated from a large parse.y file and is deeply integrated with the rest of the SQLite engine.
Over the years, I tried several approaches:
writing a parser from scratch
manually parsing specific statements like
CREATE TABLE(https://github.com/marcobambini/sqlite-createtable-parser)modifying SQLite’s
parse.ygrammar
None of these attempts worked well. Most projects ended up either incomplete or too fragile to maintain.
Recently I decided to try again.
With the help of modern coding assistants, I was finally able to push through the complexity and build a standalone parser derived from SQLite’s official grammar.
The result is liteparser.
liteparser is a feature-complete, embeddable, high-performance parser for SQLite statements written in C.
SQLite statements are parsed into a structured AST that can be inspected, analyzed, or transformed programmatically, making it possible to build tools that understand and manipulate SQL queries safely without executing them.
This capability enables a wide range of tooling around SQLite, including query linters, SQL formatters, query analyzers, schema migration tools, database IDEs, and AI systems that need to reason about SQL.
Key characteristics:
parses all SQLite statements
generates a complete AST
supports JSON representations of parsed queries
statements can be inspected, analyzed, or transformed programmatically
preserves SQLite operator precedence and grammar semantics
uses arena allocation to minimize memory overhead
works on native platforms and WASM
easily embeddable in other programming languages
Because it derives from SQLite’s official parse.y, it behaves consistently with SQLite itself.
The library has been extensively tested using:
the official SQLite test suite
millions of fuzz-testing iterations
As the founder of SQLite AI, I decided to release liteparser as open source under the MIT license, so anyone can freely use it.
If you’re building tools that need to understand SQLite queries like linters, query analyzers, developer tooling, or AI systems, this library might be extremely useful.
const char *sql = "SELECT name FROM users WHERE age > 18;";
liteparser_ast *ast = liteparser_parse(sql);
liteparser_print_json(ast);{
"type": "select",
"columns": ["name"],
"from": "users",
"where": {
"op": ">",
"left": "age",
"right": 18
}
}Project: