sql-splitter
SQL dump toolkit at 600+ MB/s
Split, merge, convert, validate, sample, shard, diff, redact, and query SQL dump files. MySQL, PostgreSQL, SQLite, and MSSQL. Written in Rust.
Why sql-splitter?
Working with large SQL dumps is painful. Importing a 10GB file takes forever, and you often just need a few tables. sql-splitter lets you split the dump into individual files so you can import only what you need.
Blazing Fast
600+ MB/s throughput
Memory Efficient
~50MB constant usage
Multi-Dialect
MySQL, PostgreSQL, SQLite, MSSQL
12 Powerful Commands
splitSplit dump into per-table filesmergeCombine files back into one dumpanalyzeView table statisticsvalidateCheck integrity and syntaxconvertTransform between dialectssampleCreate reduced datasetsshardExtract tenant datadiffCompare two dumpsredactAnonymize PII datagraphGenerate ERD diagramsorderTopological FK orderingquerySQL analytics with DuckDB
Quick Start
Get up and running in seconds.
1
Install
curl
curl -fsSL https://sql-splitter.dev/install.sh | shbrew
brew install helgesverre/tap/sql-splittercargo
cargo install sql-splitterWindows, pre-built binaries, and more in the install guide.
2
Split your dump
sql-splitter split database.sql --output=tables3
Work with individual tables
# Each table gets its own .sql file
ls tables/
# users.sql orders.sql products.sql ...Common Workflows
Database Migration
# Convert MySQL to PostgreSQL
sql-splitter convert mysql.sql --to postgres -o postgres.sql
# Stream directly to database
sql-splitter convert mysql.sql.gz --to postgres -o - | psql "$PG_CONN"Create Dev Dataset
# Sample 10% with FK integrity preserved
sql-splitter sample prod.sql -o dev.sql --percent 10 --preserve-relations
# Anonymize sensitive data
sql-splitter redact dev.sql -o safe.sql --hash "*.email" --fake "*.name"CI/CD Validation
# Validate all dumps with JSON output
sql-splitter validate "dumps/*.sql" --strict --json --fail-fast