Commit Graph

12 Commits

Author SHA1 Message Date
Kazuki Yamada 5c55597512 docs(core): Replace tiktoken references with gpt-tokenizer
Update all documentation, build configuration, and source comments
to reflect the completed migration from tiktoken to gpt-tokenizer.

- Update tokenCount.encoding description to say "OpenAI-compatible
  tokenization" with gpt-tokenizer link (README + 15 language docs)
- Remove tiktoken from bundling external deps list since
  gpt-tokenizer is pure JS (README + 15 language docs)
- Remove tiktoken COPY from Dockerfile and external from bundle.mjs
- Simplify source code comments removing migration artifacts
- Regenerate scripts/memory/package-lock.json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-06 15:37:26 +09:00
autofix-ci[bot] c839071105 [autofix.ci] apply automated fixes 2026-01-04 11:40:40 +00:00
Kazuki Yamada 1ee63f405b fix(website-server): Copy web-tree-sitter.wasm to dist-bundled
web-tree-sitter looks for the WASM file in the same directory as the
JavaScript file. Without this, the Compress feature fails with ENOENT.
2026-01-04 20:32:37 +09:00
Kazuki Yamada 5b59cfc5a2 fix(website-server): Remove @google-cloud/logging-winston to fix gRPC bundling issues
Replace @google-cloud/logging-winston with Console transport and Cloud Logging
severity mapping. Cloud Run automatically sends stdout to Cloud Logging, so
a dedicated transport is unnecessary.

Changes:
- Remove @google-cloud/logging-winston dependency
- Add severity mapping for Cloud Logging compatibility
- Add trace context extraction for distributed tracing
- Implement code splitting for server and worker bundles
- Add worker-entry.ts for minimal worker bundle
- Simplify Dockerfile to only copy tinypool and tiktoken
2026-01-04 20:32:37 +09:00
autofix-ci[bot] bfcf0555ab [autofix.ci] apply automated fixes 2026-01-04 03:20:51 +00:00
Alexander Lichter d6db26ed11 fmt 2026-01-03 20:25:54 +01:00
Alexander Lichter 217c19dcfa refactor: remove aggressive treeshaking options 2026-01-03 20:23:48 +01:00
Alexander Lichter 86c99d6955 build(website): build with rolldown and swc 2026-01-03 19:11:11 +01:00
Kazuki Yamada bc09b1f1cb refactor(website-server): Address review feedback
- Remove redundant treeShaking option (enabled by default with ESM)
- Change legalComments from 'none' to 'eof' for license compliance
2026-01-03 22:53:30 +09:00
Kazuki Yamada 1245e51b28 perf(website-server): Enable esbuild minification for smaller bundle
Add minification options to esbuild bundle configuration:
- minify: true - Enable code minification
- treeShaking: true - Remove unused code
- legalComments: 'none' - Strip license comments
- drop: ['debugger'] - Remove debugger statements

Bundle size reduced from 10.32 MB to 5.2 MB (50% reduction).
This improves Cloud Run cold start time by reducing file I/O.
2026-01-03 22:47:50 +09:00
Kazuki Yamada ac0785fe65 fix(website-server): Update esbuild target to node24 2026-01-03 21:32:59 +09:00
Kazuki Yamada 8293b68056 refactor(website-server): Create JS bundle script
Replace inline npm scripts with a dedicated bundle.mjs script for better
readability and maintainability.

The script handles:
- TypeScript compilation
- esbuild bundling with ESM banner
- Tree-sitter WASM file collection
2026-01-03 21:32:59 +09:00