Kazuki Yamada 5c55597512 docs(core): Replace tiktoken references with gpt-tokenizer ...

Update all documentation, build configuration, and source comments
to reflect the completed migration from tiktoken to gpt-tokenizer.

- Update tokenCount.encoding description to say "OpenAI-compatible
  tokenization" with gpt-tokenizer link (README + 15 language docs)
- Remove tiktoken from bundling external deps list since
  gpt-tokenizer is pure JS (README + 15 language docs)
- Remove tiktoken COPY from Dockerfile and external from bundle.mjs
- Simplify source code comments removing migration artifacts
- Regenerate scripts/memory/package-lock.json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

2026-04-06 15:37:26 +09:00

..

bundle.mjs

docs(core): Replace tiktoken references with gpt-tokenizer

2026-04-06 15:37:26 +09:00