mirror of
https://github.com/yamadashy/repomix.git
synced 2026-05-01 12:26:07 +02:00
Claude
98866f001f
Cuts per-thread IPC overhead in calculateFileMetrics by ~80% by sending
fewer, larger batches to the metrics worker pool. At ~1000 files this
reduces the dispatched task count from ~100 to ~20 (~5 batches per
worker on a 4-thread pool instead of ~25), saving ~40 estimated
worker-ms of pure serialization/dispatch overhead per worker on the
critical path.
The previous size of 10 was tuned for fast worker availability to
overlap with output generation, but tinypool round-trips were measured
at ~2ms minimum (batches of trivially small files bottom out there),
so the IPC dominated for short batches. Increasing to 50 keeps load
balance comparable (~5:1 per thread vs. the prior ~25:1) and avoids
the load-imbalance pitfall of even larger sizes (e.g. 100), where one
oversized batch can monopolize a worker and stretch the tail.
Benchmark on this repo (1016 files, 4 cores; 100 paired interleaved
runs alternating BATCH=10 and BATCH=50 with the same prebuilt JS,
swapping only the constant, full pipeline timed end-to-end):
BATCH=10 mean 1.5589s median 1.5480s stdev 0.0966s
BATCH=50 mean 1.5111s median 1.5030s stdev 0.0625s
Saved 47.8ms (3.07%) mean / 45.0ms (2.91%) median
(trimmed mean 41.0ms, dropping top/bottom 10%)
Wins 69/100 paired runs faster with BATCH=50
t-test paired t = 4.545 (df=99, p < 0.001)
95% CI [27.2 ms, 68.4 ms] on the mean improvement
Sign test 2-sided p = 0.0002
Also adds a regression test for the batching path: with 120 input files
the test asserts result order/completeness and that the progress
callback fires once per dispatched batch (not per file, not just once).
All 1250 tests pass; output is byte-identical for the same source tree.