Commit Graph

1 Commits

Author SHA1 Message Date
Michael Ilseman
a37a823e6e [stdlib] Update non-contiguous NSStrings to Unicode 9
This adds Unicode 9 grapheme breaking support for non-contiguous
NSStrings. Non-contiguous NSStrings that don't hit our fast paths are
very rare, but should still behave identically to contiguous
strings.

We first copy a fixed number of code units into a fixed size buffer
(currently 16 in size) and try to grapheme break inside of that
buffer. This is sufficient storage for all known non-pathological
graphemes. Any graphemes larger than the buffer are handled by copying
larger portions of the string into an Array.

Test cases added, including pathological "zalgo" text that stresses
extremely long graphemes.
2017-06-28 15:35:25 -07:00