archer import replays a sealed archive into an existing table: shard selection (`--select`), SHA-256 pre-verification, idempotent modes (`ignore`/`insert`/`replace`), and both LOAD DATA and batched-INSERT engines. It reads manifest.jsonl strictly so corruption is reported instead of skipped, and supports --dry-run, --allow-partial, and per-shard workers. --bulk-delete no longer rejects --shard-by date. Range deletion now derives its bounds from the chunk's actual keys (date scanners rewrite LastKey into a resumption cursor) and keeps safe-auto-increment semantics by excluding the current max key. A startup warning documents the monotonic-shard-column assumption. Also splits CLI/format/sharding/progress/safety reference docs out of the README, and adds the corresponding OpenSpec change artifacts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2.1 KiB
2.1 KiB
输出格式与导入
输出格式
--format |
说明 |
|---|---|
dump(默认) |
MySQL LOAD DATA 兼容的 TSV:TAB 分隔、\N 表示 NULL、MySQL 转义表、无表头 |
csv |
RFC4180:逗号分隔、表头、双引号转义、CRLF;--csv-null 覆盖 NULL 标记 |
jsonl |
每行一个 JSON 对象;DECIMAL 为字符串、BLOB 为 base64、时间列为 RFC3339、NULL 为 null |
压缩
--compress none|gzip|zstd|xz,默认 none。每个分片独立压缩,扩展名分别为 .gz / .zst / .xz。压缩文件可单独解压后导入。
--compress-level 1..9,默认 3。1–9 是跨压缩器的统一档位:数值越大压缩率越高、吞吐越低,9 表示该压缩器的最强档位。各格式的映射:
| 档位 | gzip | zstd | xz 字典 |
|---|---|---|---|
| 1 | 1 | fastest | 1 MiB |
| 2 | 2 | fastest | 2 MiB |
| 3(默认) | 3 | default | 4 MiB |
| 4 | 4 | default | 4 MiB |
| 5 | 5 | default | 8 MiB |
| 6 | 6 | better | 8 MiB |
| 7 | 7 | better | 16 MiB |
| 8 | 8 | better | 32 MiB |
| 9 | 9 | best | 64 MiB |
默认档位与引入该参数前的版本保持一致,存量任务可直接续跑;非默认档位会计入配置指纹,续跑时不一致会被拒绝。--compress none 不能搭配非默认档位。
xz 由纯 Go 库实现,产物是标准 xz 流,xz -dc 可直接解码。其压缩吞吐明显低于 gzip/zstd,适合长期归档、写入速度不敏感的场景;zstd 则在吞吐与压缩率之间更均衡。
导入
# dump -> MySQL(列顺序一致时)
zcat 000001.1-1000000.dump.gz | mysql -e "LOAD DATA LOCAL INFILE '/dev/stdin' INTO TABLE orders"
# xz -> MySQL
xz -dc 000001.1-1000000.dump.xz | mysql -e "LOAD DATA LOCAL INFILE '/dev/stdin' INTO TABLE orders"
# csv -> PostgreSQL
gunzip -c 000001.1-1000.csv.gz | psql -c "\copy orders FROM STDIN WITH (FORMAT csv, HEADER true)"
# csv -> ClickHouse
clickhouse-client --query "INSERT INTO orders FORMAT CSVWithNames" < 000001.csv
# jsonl -> ClickHouse
clickhouse-client --query "INSERT INTO orders FORMAT JSONEachRow" < 000001.jsonl