* Give Sequence a top-level Element, constrain Iterator to match
* Remove many instances of Iterator.
* Fixed various hard-coded tests
* XFAIL a few tests that need further investigation
* Change assoc type for arrayLiteralConvertible
* Mop up remaining "better expressed as a where clause" warnings
* Fix UnicodeDecoders prototype test
* Fix UIntBuffer
* Fix hard-coded Element identifier in CSDiag
* Fix up more tests
* Account for flatMap changes
Use UBreakIterators to perform grapheme breaking. This gives Unicode 9
grapheme breaking (e.g. family emoji) and provides a means to upgrade
to future versions. It also serves as a model for how to serve up
other advanced functionality in ICU to users.
This has tricky performance implications. Some things are faster and a
number of cases are slower. But, careful use of ICU can help mitigate
and amortize these costs. In conjunction with more early detection of
fast paths, overall grapheme breaking for the average user should be
much faster than in Swift 3.
NOTE: This is incomplete. It currently falls back on the legacy tries
for some bridged strings. There are many potential directions for a
general solution, but for now we'll be interatively adding support for
more and more special cases.
UnsafeBufferPoiunter subscript used in the fast path only checks bounds
in Debug mode, therefore extra checks are needed.
Addresses: <rdar://problem/31992473>
This adds a fast path for single-code-unit Character
construction. Rather than use the general purpose String based
initializer (which then repeats grapheme breaking to ensure a trap,
amongst other inefficiencies), just make the Character from the single
unicode scalar value directly.
This also speeds up simple iteration of BMP strings when the optimizer
is unable to eliminate the subscript. Around 2x for ASCII, and around
20% for BMP UTF16.
Many of StringCore private APIs, when the StringCore is itself a
substring, expect relative offsets rather than absolute offsets. This
fixes a bug in the sub-0x300 fast path where we were using absolute
offsets. Test cases added.
Adds in a special case grapheme break detection between two values
within scalar ranges where we have special knowledge. Any sub-0x300
scalars, except CR-LF, are guaranteed to have grapheme breaks between
them. We're reasonably confident this will not change in future
versions of Unicode. We might add more ranges in the future, but
should do so conservatively, anticipating future Unicode changes.
In these cases we can very quickly break, even for strings that have
mixed latin and emoji characters. In a ASCII string with a single
emoji in it, we traverse the string 2x faster forwards and 3x faster
in reverse. (Reverse is 3x faster as it involves some forwards
traversal inside of the index). For a string that's half Latin half
non-Latin, we're about 1.5x faster forwards and backwards.
By removing the _fastPath inside of grapheme breaking, we avoid
cooling off all the other blocks in the function, restoring
performance parity with before the ASCII special case logic. This does
not meaningfully impact the performance of the special case, so we are
still about 2-3x faster iterating forwards and backwards over ASCII
characters than before.
By doing a fast check for CR-LF, we can speed up forwards and
backwards character iteration on ASCII strings by ~2-3x. This speedup
can be seen in the stringTests suite (previous PR).
It is done as a fast path inside of
_measureExtendedGraphemeClusterForward/Backward, which is never
inlined. Doing the fast path inline would yield even more dramatic
improvements, and might be an area for future exploration, but would
slightly pessimize non-ASCII strings due to code bloat. Additionally,
the current implementation has not been micro-optimized yet. It's
possible that more optimal code would be smaller and have less of an
impact on the general case.
* [stdlib] Fix String.UTF16View index sharing
* [stdlib] Fix String.UnicodeScalarView index sharing
* [stdlib] Fix String.CharacterView index sharing
* [stdlib] Test advancing string indices past their ends
* [stdlib] Simplify CharacterView ranged subscript
We should not make grapheme segmentation algorithm inlineable now (and
possibly ever). The immediate reason is that Unicode 8.0 grapheme
segmentation algorithm has been changed significantly in Unicode 9.0.
Unicode 9.0 requires a stateful algorithm. Current indices and
iterators just don't have the storage for that state, so we can't mark
them as fragile.
This removes the `_core` property from UnicodeScalarView.Index
and moves any remaining index-moving logic from the index to
the view in UnicodeScalarView and CharacterView.
This documentation revision covers a large number of types & protocols:
String, its views and their indices, the Unicode codec types and protocol,
as well as Character, UnicodeScalar, and StaticString, among others.
This also includes a few small changes across the standard library for
consistency.
1_stdlib/Collection.swift test runs without error.
Note: advance and distance implementations for which the default works
were removed from StringCharacterView.swift
-removed fatal stub Collection.next(Index)
-added default Collection.next(Index) where Index is Strideable
-added custom next(Index) on some collections
-added fatal stub next(Index) on some collections
