Earlier push commit (03b8e70b) accidentally dropped the
`expect(actualOutput).toContain('# Directory Structure')` assertion
inside the markdown branch of the integration test's case-style switch.
That line still exists on disk and was present pre-change; restoring it
to keep the markdown-style coverage symmetric with the plain-style
case below it.
Final commit of the perf(core) Pre-warm security worker pool change —
extends the unit packager test and the integration packager test:
- tests/core/packager.test.ts: adds `createSecurityTaskRunner` mock to
the orchestration test's `mockDeps` and to the `parallel error
handling` `baseDeps()` shared fixture, updates the
`validateFileSafety.toHaveBeenCalledWith` assertion to expect the new
6th-argument deps object (`{ taskRunner: <Object> }`), and adds
positive/negative gate assertions —
`expect(deps.createSecurityTaskRunner).toHaveBeenCalled()` for the
default unscoped path, `.not.toHaveBeenCalled()` for the
`--include 'src'` and `explicitFiles` (--stdin) paths.
- tests/integration-tests/packager.test.ts: adds the
`createSecurityTaskRunner` stub so the default-scope path no longer
attempts to spawn a real worker pool (the previous unhandled-rejection
noise from a missing worker file URL is gone with this change).
(See PR description / first source commit for the full perf change
rationale, benchmark numbers, and correctness notes.)
Move worker thread warmup from packager into createMetricsTaskRunner,
which now returns both a taskRunner and warmupPromise. This keeps the
packager clean — it no longer needs to know warmup implementation details.
Also:
- Skip metrics worker pool creation on skill-generation path where
it is unused
- Await warmupPromise in finally block before cleanup to prevent
tearing down workers during initialization
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pipeline-level optimizations that produce measurable end-to-end improvement:
- Pre-initialize metrics worker pool during file collection phase so tiktoken
WASM loading overlaps with security checks and file processing. First token
count task dropped from 381ms to 22ms (worker already warmed).
- Lazy-load Jiti via dynamic import — only loaded when TS/JS config files are
detected, saving startup time for the common JSON/default config path.
- Fix O(n²) file path re-grouping in packager by using Map + Set for O(1)
membership checks instead of .find() + .includes().
- Move binary extension check before fs.stat in fileRead to skip unnecessary
stat syscalls for binary files.
- Parallelize split output file writes with Promise.all instead of sequential
for-loop.
Benchmark (15 runs each, median ± IQR, packing repomix repo ~1000 files):
main branch: 3515ms (P25: 3443, P75: 3581)
perf branch: 3318ms (P25: 3215, P75: 3383)
Improvement: -197ms (-5.6%)
Pipeline stage breakdown (instrumented):
- Metrics first-file init: 381ms → 22ms (worker pre-warmed)
- Total metrics stage: 793ms → ~450ms
All 1096 tests pass. Lint clean.
https://claude.ai/code/session_01JoNjFe7S2roMfHfNcw6bso
After the UTF-8 fast path optimization eliminated the CPU-heavy jschardet
bottleneck, file collection became I/O-bound. Worker threads now add pure
overhead (Tinypool init, structured clone, IPC) without benefit.
Benchmark (954 files, M2 Pro 10-core):
- Worker Threads: ~108ms → Promise Pool (c=50): ~37ms (2.9x faster)
Changes:
- Replace Tinypool worker dispatch with a simple promise pool (c=50)
- Inject readRawFile via deps for testability
- Remove unused concurrentTasksPerWorker from WorkerOptions
- Simplify tests to use readRawFile mock instead of 5+ module mocks
- Move makeChunkConfig and renderGroups to module level for better readability
- Add GenerateOutputFn type alias using typeof generateOutput
- Add comment explaining O(N²) complexity and why it's acceptable
- Fix test mock property names to match actual GitDiffResult/GitLogResult types
- Update integration tests to use produceOutput instead of individual functions
Added override configuration to disable Biome's organizeImports feature
specifically for src/index.ts to allow manual import order management
while keeping automatic import organization enabled for other files.
Updated biome from v1.9.4 to v2.2.4 to take advantage of latest linting improvements.
- Upgraded @biomejs/biome from ^1.9.4 to ^2.2.4
- Updated biome.json configuration for v2 compatibility:
- Changed schema to 2.2.4
- Updated file includes/ignores syntax
- Added Vue file overrides to disable noUnusedVariables/noUnusedImports
- Fixed all lint errors:
- Added radix parameter to parseInt calls
- Prefixed unused parameters with underscore
- Removed unused imports
- Fixed biome suppression comments
- Removed !important from CSS
- Added type ignores for Vue component definitions
All 325 files now pass lint with 0 warnings and 0 errors.
Replace executeGlobbyInWorker with direct globby calls since worker isolation
is no longer necessary for globby execution.
- Remove src/core/file/globbyExecute.ts wrapper
- Remove src/core/file/workers/globbyWorker.ts
- Update fileSearch.ts to import and use globby directly
- Update tests to mock globby instead of executeGlobbyInWorker
- Simplify integration tests by removing worker mocks
- Reorder output formats from xml, markdown, plain, json to xml, markdown, json, plain throughout README.md, CLI help, and schema
- Move JSON Format section before Plain Text Format in documentation
- Enhance jq command examples with comprehensive JSON manipulation patterns
- Change jq examples to use `cat | jq` pipeline format for consistency
- Update all references to maintain xml, markdown, json, plain order in:
- CLI options help text
- Configuration schema enum
- GitHub Actions examples
- Documentation tables
- Fix linting issue with switch case block scoping in integration tests
This improves the logical flow by placing JSON (programmatic format) before Plain (simple format) and maintains consistent ordering across all documentation and code.
Add comprehensive JSON output style with camelCase naming convention:
- JSON output generates machine-readable format using JSON.stringify()
- Always uses parsable style for valid JSON structure
- Parameter names follow camelCase convention (fileSummary, userProvidedHeader, etc.)
- Includes CLI option, config schema, and interactive prompt support
- Comprehensive test coverage for unit and integration tests
- Error handling with clear unsupported style messages
This enables easier programmatic consumption of Repomix output for AI systems and automated tools.
Add WorkerRuntime type and configurable runtime parameter to createWorkerPool and initTaskRunner functions. This allows choosing between 'worker_threads' and 'child_process' runtimes based on performance requirements.
- Add WorkerRuntime type definition for type safety
- Add optional runtime parameter to createWorkerPool with child_process default
- Add optional runtime parameter to initTaskRunner with child_process default
- Configure fileCollectWorker to use worker_threads for better performance
- Update all test files to use WorkerRuntime type
- Add comprehensive tests for runtime parameter functionality
- Maintain backward compatibility with existing code
The fileCollectWorker now benefits from worker_threads faster startup and shared memory, while other workers continue using child_process for stability.
This feature allows users to include git log information in the output to help AI understand development patterns and file change relationships.
Key changes:
- Added --include-logs and --include-logs-count CLI options
- Default to 50 commits, configurable via CLI and config file
- Includes commit date, message, and changed file paths (excludes commit hashes)
- Added security checks and metrics calculation for git logs
- Updated output templates to include git logs section
- Comprehensive test coverage and TypeScript fixes
Resolves user request for including git commit history to provide development context for AI analysis.
- Create globbyWorker.ts for isolated globby execution
- Add globbyExecute.ts to manage worker-based globby calls
- Update fileSearch.ts to use worker isolation for all globby operations
- Remove redundant cleanup calls in worker-using functions
- Fix processConcurrency.ts Bun detection logic
- Update tests to mock worker functionality for test environment
This change isolates globby memory usage in child processes, preventing
memory accumulation in the main process when processing large codebases.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add generic initTaskRunner function to processConcurrency.ts to eliminate
duplicate initialization logic across multiple modules. This reduces code
duplication and provides consistent worker pool management with proper
type safety through generic parameters.
- Add TaskRunner<T, R> interface and initTaskRunner function
- Remove duplicate createTaskRunner wrappers from 5 modules
- Update all deps parameters to use shared initTaskRunner directly
- Maintain type safety with explicit generic type parameters
- Update corresponding test mocks to match new signature
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
TokenCounter instances were not being properly freed when worker threads
were terminated by Tinypool's idle timeout. This caused memory leaks
when using runCli as a library.
Changes:
- Add SIGTERM/SIGINT handlers to fileMetricsWorker and outputMetricsWorker
- Add freeTokenCounters function with proper cleanup and debug logging
- Convert all worker usage to consistent taskRunner pattern with cleanup
- Add cleanupWorkerPool function for explicit worker pool termination
- Update all related tests to match new taskRunner interface
The fix ensures TokenCounter resources are properly freed when workers
terminate, preventing memory accumulation during library usage.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Update dependency injection parameter names to be more descriptive of the actual functionality.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
Kazuki Yamada
Claude
Kazuki Yamada
Devin AI
Riqwan Thamir
paperboardofficial
Rakesh Chatrath
gitkenan