Files
archer/docs/output-formats.md
voidandClaude Sonnet 4.6 366d61ca25 Add archer import and allow date-mode bulk delete
archer import replays a sealed archive into an existing table: shard
selection (`--select`), SHA-256 pre-verification, idempotent modes
(`ignore`/`insert`/`replace`), and both LOAD DATA and batched-INSERT
engines. It reads manifest.jsonl strictly so corruption is reported
instead of skipped, and supports --dry-run, --allow-partial, and
per-shard workers.

--bulk-delete no longer rejects --shard-by date. Range deletion now
derives its bounds from the chunk's actual keys (date scanners rewrite
LastKey into a resumption cursor) and keeps safe-auto-increment
semantics by excluding the current max key. A startup warning documents
the monotonic-shard-column assumption.

Also splits CLI/format/sharding/progress/safety reference docs out of
the README, and adds the corresponding OpenSpec change artifacts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-11-08 21:14:23 +08:00

2.1 KiB
Raw Permalink Blame History

输出格式与导入

输出格式

--format 说明
dump(默认) MySQL LOAD DATA 兼容的 TSV:TAB 分隔、\N 表示 NULL、MySQL 转义表、无表头
csv RFC4180:逗号分隔、表头、双引号转义、CRLF;--csv-null 覆盖 NULL 标记
jsonl 每行一个 JSON 对象;DECIMAL 为字符串、BLOB 为 base64、时间列为 RFC3339、NULL 为 null

压缩

--compress none|gzip|zstd|xz,默认 none。每个分片独立压缩,扩展名分别为 .gz / .zst / .xz。压缩文件可单独解压后导入。

--compress-level 1..9,默认 3。1–9 是跨压缩器的统一档位:数值越大压缩率越高、吞吐越低,9 表示该压缩器的最强档位。各格式的映射:

档位 gzip zstd xz 字典
1 1 fastest 1 MiB
2 2 fastest 2 MiB
3(默认) 3 default 4 MiB
4 4 default 4 MiB
5 5 default 8 MiB
6 6 better 8 MiB
7 7 better 16 MiB
8 8 better 32 MiB
9 9 best 64 MiB

默认档位与引入该参数前的版本保持一致,存量任务可直接续跑;非默认档位会计入配置指纹,续跑时不一致会被拒绝。--compress none 不能搭配非默认档位。

xz 由纯 Go 库实现,产物是标准 xz 流,xz -dc 可直接解码。其压缩吞吐明显低于 gzip/zstd,适合长期归档、写入速度不敏感的场景;zstd 则在吞吐与压缩率之间更均衡。

导入

# dump -> MySQL(列顺序一致时)
zcat 000001.1-1000000.dump.gz | mysql -e "LOAD DATA LOCAL INFILE '/dev/stdin' INTO TABLE orders"

# xz -> MySQL
xz -dc 000001.1-1000000.dump.xz | mysql -e "LOAD DATA LOCAL INFILE '/dev/stdin' INTO TABLE orders"

# csv -> PostgreSQL
gunzip -c 000001.1-1000.csv.gz | psql -c "\copy orders FROM STDIN WITH (FORMAT csv, HEADER true)"

# csv -> ClickHouse
clickhouse-client --query "INSERT INTO orders FORMAT CSVWithNames" < 000001.csv

# jsonl -> ClickHouse
clickhouse-client --query "INSERT INTO orders FORMAT JSONEachRow" < 000001.jsonl