mirror of
https://github.com/yamadashy/repomix.git
synced 2026-05-30 11:18:53 +02:00
Kazuki Yamada
c8560d1bc6
Lower `EAGER_WARMUP_THREADS` from 2 to 1 when `tokenCountCacheFileExists()` returns true. With the persistent token-count disk cache populated by a prior run, `calculateFileMetrics` serves every per-file token count from the in-memory map and dispatches zero worker tasks. The only worker work that survives caching on a warm rerun is a small fixed set of dispatches: - the wrapper-token tokenization (cache hit after run #2) - git diff staged/worktree token counts (only when `output.git.includeDiffs` is enabled) - git log token count (only when `output.git.includeLogs` is enabled) That worst case is 2-3 short tasks (a few KB each) that fit a single warm worker serially in well under 30 ms. Spawning a second warm worker means a redundant ~340 ms BPE table parse that contends with the file-collection main thread for CPU AND extends the final `pool.destroy()` blocking wait (BPE-loaded workers take ~21 ms to terminate vs ~3 ms when idle). Cold-cache (no cache file) behavior is preserved: the unscoped path keeps 3 warm workers and the explicit-scope path keeps 2, so the actual file tokenizations still parallelise across the original worker counts. The probe is a coarse heuristic — a cache file written by a previous run that used a different `tokenCount.encoding` (e.g. cl100k_base instead of the default o200k_base) yields no hits for the current run, so the metrics phase pays one BPE parse sequentially on the critical path before tokenizing files. This is a one-time cost on encoding switches; subsequent runs rebuild the cache for the new encoding and hit again. Benchmark (paired, n=25, repomix self-pack on 1068 files): WARM CACHE (cache file present) BASELINE mean=968.9ms median=976.0ms sd=40.3ms AFTER mean=883.2ms median=875.0ms sd=33.1ms DELTA mean=85.6 ms (8.84%) median=87.0 ms sd=42.7 t=10.02 (df=24) faster=24/25 COLD CACHE (cache file deleted before each run, n=12) BASELINE mean=1606.3ms median=1588.0ms sd=58.6ms AFTER mean=1593.2ms median=1598.5ms sd=58.6ms DELTA mean=13.2 ms (0.82%) t=0.62 faster=9/12 — within noise Stacks on top of the existing warm-cache wins on this branch (token-count disk cache, output-wrapper cache, prefetched template, native ignore-file prescan, etc.); this single change pushes warm-cache wall-clock another ~86 ms below the previous floor.