Benchmarking

sql-splitter includes a Docker-based benchmarking suite to compare performance against other SQL dump splitting tools.

Tools Tested

Tool	Language	Stars	Notes
sql-splitter	Rust	-	Multi-dialect, streaming I/O
mysqldbsplit	PHP	101	Fastest competitor; selective extraction
mysqldump-splitter	Rust	1	Hierarchical output, gzip support
mysql-dump-splitter	Go	0	Include/exclude tables
mysqldumpsplit	Go	~40	Buffers in memory, has deadlock bug*
mysqldumpsplitter	Bash/awk	540+	Most popular shell-based tool
mysql_splitdump	Bash/csplit	93	Uses GNU coreutils csplit
mysqldumpsplit	Node.js	55	Requires Node 10 (gulp 3.x)
mysql-dump-split	Ruby	77	Archived project
extract-mysql-dump	Python	~5	Multi-database extraction, Python 3.3+

*Original Go tool has a deadlock bug with non-interleaved dumps; benchmarks use a patched fork.

Running Benchmarks

Benchmarks run inside Docker for reproducibility across different machines.

Quick Start

# Build the Docker image (first time only)
just docker-build

# Run benchmark with generated 100MB test file
just docker-bench

# Run with custom size (e.g., 200MB)
./docker/run-benchmark.sh --generate 200

Options

Flag	Description
--generate SIZE	Generate test data of SIZE MB
--runs N	Number of benchmark runs (default: 3)
--warmup N	Warmup runs before timing (default: 1)
--export FILE	Export results to markdown file
--list	Show installed tools
--test	Test which tools work with file

Examples

# Generate 500MB test and run 5 iterations
./docker/run-benchmark.sh --generate 500 --runs 5

# Export results to markdown
./docker/run-benchmark.sh --generate 100 --export results.md

# List available tools
./docker/run-benchmark.sh --list

# Test which tools work with a specific file
./docker/run-benchmark.sh --test /data/dump.sql

Benchmark Results

Hardware: Apple M2 Max, 32GB RAM, Docker Desktop (linux/arm64)

100MB Generated Test File

Tool	Mean	σ	Throughput	Relative
sql-splitter (Rust)	64 ms	±5	1605 MB/s	1.00 (fastest)
mysqldbsplit (PHP)	79 ms	±3	1304 MB/s	1.23x slower
mysqldumpsplit (Go)*	193 ms	±46	536 MB/s	3.00x slower
mysql-dump-splitter (Go/Bekkema)	196 ms	±53	526 MB/s	3.05x slower
mysql_splitdump (csplit)	237 ms	±16	435 MB/s	3.69x slower
mysqldump-splitter (Rust/Scoopit)	271 ms	±35	381 MB/s	4.21x slower
mysqldumpsplit (Node.js)	427 ms	±20	242 MB/s	6.64x slower
mysql-dump-split (Ruby)	979 ms	±16	105 MB/s	15.2x slower
mysqldumpsplitter (Bash)	1027 ms	±25	100 MB/s	16.0x slower
extract-mysql-dump (Python)	1655 ms	±122	62 MB/s	25.7x slower

1GB Generated Test File

Tool	Mean	σ	Throughput	Relative
sql-splitter (Rust)	0.42s	±0.01	2453 MB/s	1.00 (fastest)
mysqldbsplit (PHP)	0.80s	±0.10	1289 MB/s	1.90x slower
mysql-dump-splitter (Go/Bekkema)	0.94s	±0.01	1097 MB/s	2.24x slower
mysqldump-splitter (Rust/Scoopit)	1.17s	±0.02	884 MB/s	2.78x slower
mysqldumpsplit (Go)*	1.50s	±0.06	691 MB/s	3.55x slower
mysql_splitdump (csplit)	2.43s	±0.13	425 MB/s	5.77x slower
mysqldumpsplit (Node.js)	2.91s	±0.03	356 MB/s	6.89x slower
mysqldumpsplitter (Bash)	9.37s	±0.36	110 MB/s	22.2x slower
mysql-dump-split (Ruby)	9.92s	±0.63	104 MB/s	23.5x slower
extract-mysql-dump (Python)	13.75s	±0.12	75 MB/s	32.6x slower

5GB Stress Test (disk-backed /tmp, 3 runs)

Tool	Mean	σ	Throughput	Relative
sql-splitter (Rust)	10.8s	±8.1	480 MB/s	1.00 (fastest)
mysql-dump-splitter (Go/Bekkema)	12.1s	±0.7	427 MB/s	1.12x slower
mysqldbsplit (PHP)	12.8s	±1.5	404 MB/s	1.19x slower
mysqldump-splitter (Rust/Scoopit)	22.8s	±9.3	227 MB/s	2.11x slower
mysqldumpsplit (Node.js)	23.8s	±5.1	218 MB/s	2.20x slower
mysqldumpsplit (Go)*	35.6s	±5.0	146 MB/s	3.29x slower
mysqldumpsplitter (Bash)	49.8s	±1.1	104 MB/s	4.60x slower
mysql-dump-split (Ruby)	68.3s	±1.3	76 MB/s	6.32x slower
extract-mysql-dump (Python)	90.6s	±1.5	57 MB/s	8.38x slower
mysql_splitdump (csplit)	92.5s	±15.8	56 MB/s	8.55x slower

At 5GB the container’s disk write bandwidth dominates and compresses the margins — sql-splitter stays fastest, with the top three tools all effectively I/O-bound (~400–480 MB/s).

Key Findings

Speed vs Memory Tradeoffs

• sql-splitter (Rust) is the fastest at every size — 1.6 GB/s on 100MB, 2.4 GB/s on 1GB — since the v1.14.1+ write-path rework (parallel pipelined writers) and allocation-lean parsing.
• mysqldbsplit (PHP) is the fastest competitor at ~1.3 GB/s—surprisingly beating all other compiled tools on mysqldump format.
• sql-splitter streams with bounded memory (~20–150 MB depending on command), independent of file size.
• csplit is surprisingly fast for a shell tool, but relies on GNU coreutils (not available on stock macOS).
• Node.js is ~7x slower but still reasonable for JS-based workflows.
• Ruby/Bash/awk are 15-24x slower—fine for one-off use but not for automation.
• Python (extract-mysql-dump) is the slowest at 26-33x slower, designed specifically for multi-database extraction scenarios.

Format Compatibility

All competitors only work with standard mysqldump format that includes comment markers like:

-- Table structure for table `users`

sql-splitter parses actual SQL statements (CREATE TABLE, INSERT INTO, COPY), so it works with:

• TablePlus exports
• DBeaver exports
• pg_dump (PostgreSQL)
• sqlite3 .dump
• Any valid SQL file

Competitors produce 0 tables on non-mysqldump files.

Known Limitations

Tool	Issue
mysqldbsplit (PHP)	Requires PHP CLI; mysqldump format only
mysqldump-splitter (Scoopit)	Rust; mysqldump format only; hierarchical output structure
mysql-dump-splitter (Bekkema)	Go; mysqldump format only
Go (afrase)	Deadlocks on files where all INSERTs for one table come before the next table
Node.js (vekexasia)	Requires Node 10 (gulp 3.x incompatible with Node 12+)
Ruby	Project archived, unmaintained
Bash/awk	Slow, Unix-only
csplit	Requires GNU coreutils
extract-mysql-dump	Designed for multi-database dumps; no absolute paths; slowest of all tools
sql-splitter	Parses actual SQL rather than mysqldump comment markers, so it does strictly more work per byte

When to Use sql-splitter

1. Non-mysqldump formats (TablePlus, DBeaver, pg_dump, sqlite)
2. Large files (>1GB) where memory matters
3. CI/CD pipelines needing consistent behavior
4. Multi-dialect projects (MySQL + PostgreSQL + SQLite + MSSQL)

Docker Files

The benchmark infrastructure lives in the docker/ directory:

File	Purpose
Dockerfile.benchmark	Container with all tools installed
docker-compose.benchmark.yml	Compose configuration
run-benchmark.sh	Entry point script
benchmark-runner.sh	Core benchmarking logic
test-competitors.sh	Test tool compatibility

See Also

• Performance - Optimization tips
• Development - Development setup