Using libsyntaqlite-parser from C
libsyntaqlite-parser is the part of libsyntaqlite that parses SQL. It is a fraction of libsyntaqlite's size, and because it is plain C, you can compile it into your project with no other tools. In this tutorial you use it to parse a few queries and print their syntax trees, see how the parser handles errors, and list the tokens it read.
If you also want to format or analyze SQL, use libsyntaqlite. Its parser functions are the same.
1. Get libsyntaqlite-parser
You can download libsyntaqlite-parser as a prebuilt library or as C source. The code in this tutorial is the same for both.
mkdir sql-parser && cd sql-parser
curl -L https://github.com/LalitMaganti/syntaqlite/releases/download/v0.14.0/libsyntaqlite-parser-0.14.0.tar.gz | tar xz
The header, plus a static and a shared library for each platform.
mkdir sql-parser && cd sql-parser
curl -L https://github.com/LalitMaganti/syntaqlite/releases/download/v0.14.0/libsyntaqlite-parser-0.14.0-src.tar.gz | tar xz
The header and libsyntaqlite_parser.c, which you compile with your program.
2. Write the program
Create parse.c:
#include "libsyntaqlite_parser.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
int main(int argc, char** argv) {
const char* sql = "SELECT id, name FROM users WHERE active = 1";
if (argc > 1) {
sql = argv[1];
}
// Create a parser for the SQLite dialect.
SyntaqliteParser* p = syntaqlite_parser_create(NULL);
// Feed the source text.
syntaqlite_parser_reset(p, sql, strlen(sql));
// Parse each statement (a source string can contain multiple).
int stmt = 0;
for (;;) {
int32_t rc = syntaqlite_parser_next(p);
if (rc == SYNTAQLITE_PARSE_DONE)
break;
stmt++;
if (rc == SYNTAQLITE_PARSE_ERROR) {
fprintf(stderr, "error in statement %d: %s\n",
stmt, syntaqlite_result_error_msg(p));
continue;
}
// Print the AST.
uint32_t root = syntaqlite_result_root(p);
char* dump = syntaqlite_dump_node(p, root, 0);
printf("--- statement %d ---\n%s\n", stmt, dump);
free(dump);
}
syntaqlite_parser_destroy(p);
return 0;
}
3. Compile and run
If you downloaded the prebuilt library, link the static library for your platform:
cc -o parse parse.c -I. linux-x64/libsyntaqlite_parser.a -ldl
If you downloaded the source, compile it with your program:
cc -O2 -o parse parse.c libsyntaqlite_parser.c -ldl
-ldl is only needed on Linux systems with glibc older than 2.34. For macOS,
Windows, and the shared library, see
Installing the C libraries.
Run it with the default query:
./parse
--- statement 1 ---
SelectStmt
flags: (none)
columns:
ResultColumnList [2 items]
ResultColumn
flags: (none)
alias: (none)
alias_as: FALSE
expr:
ColumnRef
column: "id"
table: (none)
schema: (none)
...
from_clause:
TableRef
table_name: "users"
...
where_clause:
BinaryExpr
op: EQ
...
Pass your own SQL:
./parse "CREATE TABLE t(x INTEGER PRIMARY KEY, y TEXT NOT NULL)"
4. Handle errors
The parser recovers from errors and keeps going. Try invalid SQL:
./parse "SELECT FROM; SELECT 1"
error in statement 1: syntax error near 'FROM'
--- statement 2 ---
SelectStmt
flags: (none)
columns:
ResultColumnList [1 items]
ResultColumn
flags: (none)
alias: (none)
alias_as: FALSE
expr:
Literal
literal_type: INTEGER
source: "1"
...
The first statement fails, but the parser carries on and parses the second.
5. Access tokens
To see the tokens the parser read, turn on token collection before parsing:
syntaqlite_parser_set_collect_tokens(p, 1);
syntaqlite_parser_reset(p, sql, strlen(sql));
syntaqlite_parser_next(p);
uint32_t count;
const SyntaqliteParserToken* tokens =
syntaqlite_result_tokens(p, &count);
for (uint32_t i = 0; i < count; i++) {
printf("token %u: type=%u offset=%u len=%u\n",
i, tokens[i].type,
tokens[i].offset, tokens[i].length);
}
For SELECT id FROM users, this prints:
token 0: type=161 offset=0 len=6
token 1: type=40 offset=7 len=2
token 2: type=127 offset=10 len=4
token 3: type=40 offset=15 len=5
Next steps
- libsyntaqlite for C: format and analyze SQL as well as parse it
- Installing the C libraries: linking on macOS and Windows
- libsyntaqlite C API reference: all parser and tokenizer functions