archer import replays a sealed archive into an existing table: shard selection (`--select`), SHA-256 pre-verification, idempotent modes (`ignore`/`insert`/`replace`), and both LOAD DATA and batched-INSERT engines. It reads manifest.jsonl strictly so corruption is reported instead of skipped, and supports --dry-run, --allow-partial, and per-shard workers. --bulk-delete no longer rejects --shard-by date. Range deletion now derives its bounds from the chunk's actual keys (date scanners rewrite LastKey into a resumption cursor) and keeps safe-auto-increment semantics by excluding the current max key. A startup warning documents the monotonic-shard-column assumption. Also splits CLI/format/sharding/progress/safety reference docs out of the README, and adds the corresponding OpenSpec change artifacts. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1.4 KiB
1.4 KiB
清单、断点与一致性
清单与断点
每个分片关闭后向 manifest.jsonl 追加一行并 fsync:
{"seq":1,"path":"000001.1-1000000.dump","pk_lo":"1","pk_hi":"1000000","rows":1000000,"bytes":12345678,"sha256":"...","closed_at":"2026-11-06T12:00:00Z"}
state.json 在每个 chunk 提交后原子更新(tmp + fsync + rename),记录:
config_hash与完整的配置指纹;- 扫描游标(主键、日期键或当前日期);
- 当前日期桶与分片序号;
- 已处理行数。
续跑时使用相同输出目录与配置即可。指纹不一致会拒绝,除非显式
--force-resume;不一致的字段会列在错误信息里。密码不参与指纹,允许凭据轮换。
已封分片的文件如果缺少 manifest 记录(例如在 rename 与 manifest 追加之间崩溃),
续跑会自动识别并补录,不会被覆盖。带 --purge 时还会保留活动分片的
.partial 与 sidecar 元数据,续跑时截断到一致偏移并追加,避免已删除数据丢失。
一致性
at-least-once。文件写入与数据库提交不在同一个原子域内,崩溃后重跑可能重复写入
已处理的行,但不会丢失已提交的数据。导入端可用 INSERT IGNORE / 去重逻辑处理。
扫描期间不提供快照一致性:只保证读取到的行,并发写入可能改变行集合。