Files
archer/docs/output-formats.md
voidandClaude Sonnet 4.6 366d61ca25 Add archer import and allow date-mode bulk delete
archer import replays a sealed archive into an existing table: shard
selection (`--select`), SHA-256 pre-verification, idempotent modes
(`ignore`/`insert`/`replace`), and both LOAD DATA and batched-INSERT
engines. It reads manifest.jsonl strictly so corruption is reported
instead of skipped, and supports --dry-run, --allow-partial, and
per-shard workers.

--bulk-delete no longer rejects --shard-by date. Range deletion now
derives its bounds from the chunk's actual keys (date scanners rewrite
LastKey into a resumption cursor) and keeps safe-auto-increment
semantics by excluding the current max key. A startup warning documents
the monotonic-shard-column assumption.

Also splits CLI/format/sharding/progress/safety reference docs out of
the README, and adds the corresponding OpenSpec change artifacts.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-11-08 21:14:23 +08:00

51 lines
2.1 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# 输出格式与导入
## 输出格式
| `--format` | 说明 |
|---|---|
| `dump`(默认) | MySQL `LOAD DATA` 兼容的 TSV:TAB 分隔、`\N` 表示 NULL、MySQL 转义表、无表头 |
| `csv` | RFC4180:逗号分隔、表头、双引号转义、CRLF;`--csv-null` 覆盖 NULL 标记 |
| `jsonl` | 每行一个 JSON 对象;DECIMAL 为字符串、BLOB 为 base64、时间列为 RFC3339、NULL 为 `null` |
## 压缩
`--compress none|gzip|zstd|xz`,默认 `none`。每个分片独立压缩,扩展名分别为 `.gz` / `.zst` / `.xz`。压缩文件可单独解压后导入。
`--compress-level 1..9`,默认 `3`。1–9 是跨压缩器的统一档位:数值越大压缩率越高、吞吐越低,`9` 表示该压缩器的最强档位。各格式的映射:
| 档位 | gzip | zstd | xz 字典 |
|---|---|---|---|
| 1 | 1 | fastest | 1 MiB |
| 2 | 2 | fastest | 2 MiB |
| 3(默认) | 3 | default | 4 MiB |
| 4 | 4 | default | 4 MiB |
| 5 | 5 | default | 8 MiB |
| 6 | 6 | better | 8 MiB |
| 7 | 7 | better | 16 MiB |
| 8 | 8 | better | 32 MiB |
| 9 | 9 | best | 64 MiB |
默认档位与引入该参数前的版本保持一致,存量任务可直接续跑;非默认档位会计入配置指纹,续跑时不一致会被拒绝。`--compress none` 不能搭配非默认档位。
xz 由纯 Go 库实现,产物是标准 xz 流,`xz -dc` 可直接解码。其压缩吞吐明显低于 gzip/zstd,适合长期归档、写入速度不敏感的场景;zstd 则在吞吐与压缩率之间更均衡。
## 导入
```bash
# dump -> MySQL(列顺序一致时)
zcat 000001.1-1000000.dump.gz | mysql -e "LOAD DATA LOCAL INFILE '/dev/stdin' INTO TABLE orders"
# xz -> MySQL
xz -dc 000001.1-1000000.dump.xz | mysql -e "LOAD DATA LOCAL INFILE '/dev/stdin' INTO TABLE orders"
# csv -> PostgreSQL
gunzip -c 000001.1-1000.csv.gz | psql -c "\copy orders FROM STDIN WITH (FORMAT csv, HEADER true)"
# csv -> ClickHouse
clickhouse-client --query "INSERT INTO orders FORMAT CSVWithNames" < 000001.csv
# jsonl -> ClickHouse
clickhouse-client --query "INSERT INTO orders FORMAT JSONEachRow" < 000001.jsonl
```