Move the shifts to index creation time rather than index comparison
time. This seems to benefit micro benchmarks and cover up
inefficiencies in our generic index distance calculations.
Simplify String.Index by sinking transcoded offsets into the .utf8
variant. This is in preparation for a more resilient index type
capable of supporting existential string indices.
Drop append-related @inlinable annotations for String, StringGuts,
StringStorage, and the Views. Drop several for larger operations, such
as case conversion. Drop as many as we can from StringGuts for now.
- Rename `Substring.CharacterView` to `Substring._CharacterView`, adding a deprecated typealias for the original name, like we do for `String.CharacterView`.
- Add a non-deprecated `Substring._characters` property, emulating `String.characters`.
- Explicitly deprecate the following members:
* String.withMutableCharacters<R>(_: (inout CharacterView) -> R) -> R
* String.subscript(Range<Index>) -> String.CharacterView
* Substring._CharacterView.subscript(Range<Index>) -> Substring.CharacterView
* Substring.init(_: CharacterView)
* String.init(_: Substring.CharacterView)
Include the initial implementation of _StringGuts, a 2-word
replacement for _LegacyStringCore. 64-bit Darwin supported, 32-bit and
Linux support in subsequent commits.
In grand LLVM tradition, the first step to redesigning _StringCore is
to first rename it to _LegacyStringCore. Subsequent commits will
introduce the replacement, and eventually all uses of the old one will
be moved to the new one.
NFC.
* Eradicate IndexDistance associated type, replacing with Int everywhere
* Consistently use Int for ExistentialCollection’s IndexDistance type.
* Fix test for IndexDistance removal
* Remove a handful of no-longer-needed explicit types
* Add compatibility shims for non-Int index distances
* Test compatibility shim
* Move IndexDistance typealias into the Collection protocol
- Update NSRange -> Range guidance
- Fix example in Optional
- Improve RangeExpression docs
- Fix issue in UnsafeRawBufferPointer.initializeMemory
- Code point -> scalar value most places
- Reposition the dot above the scripty `i'
- Fix ExpressibleByArrayLiteral code sample
CharacterView is now entirely redundant in Swift 4. Deprecate its
use. This also allows us to schedule the unbreaking of
String.CharacterView leakiness without a hard source break.
I failed to merge the upstream changes to swift-corelibs-foundation at the same
time as I merged that #9806, and it broke on linux. Going to get it right this
time.
Removes the legacy grapheme breaking code paths. Simplifies and
clarifies the non-contiguous grapheme breaking code through consistent
naming and handling of absolute positions vs relative offsets.
This adds Unicode 9 grapheme breaking support for non-contiguous
NSStrings. Non-contiguous NSStrings that don't hit our fast paths are
very rare, but should still behave identically to contiguous
strings.
We first copy a fixed number of code units into a fixed size buffer
(currently 16 in size) and try to grapheme break inside of that
buffer. This is sufficient storage for all known non-pathological
graphemes. Any graphemes larger than the buffer are handled by copying
larger portions of the string into an Array.
Test cases added, including pathological "zalgo" text that stresses
extremely long graphemes.
Many strings use non-sub-300 punctuation characters (e.g. unicode
hyphen, CJK quotes, etc). This can cause switching between fast and
slow paths for grapheme breaking. Add in fast-paths for general
punctuation characters and CJK punctuation and symbol characters.
This results in about a 5-8x speedup for heavily (unicode) punctuated
Latiny and CJKy workloads.
This takes care of the standard library portion, but we need a new
BuiltinUTF16ExtendedGraphemeClusterLiteralConvertible protocol in order to
fully recover the performance of character literals.
Note that part of the character_literals.swift test is currently disabled. That
will need to be fixed before we can merge this work.
Add in more grapheme break fast paths for scripts based on Cyrillic,
Arabic, or Hangul. Generates significant performance wins, similar to
those for the unihan fast paths.
While every extra check does slow down the runtime of
_internalExtraCheckGraphemeBreakBetween as currently implemented, I've
not found the performance cost to be relevant for workloads with
occasional mixed emoji contents, nor for workloads that his the
earlier checks. A pure Korean workload (currently the last check) does
pays a rather noticable price for the previous checks, but this is
only because the workload is now so greatly improved. Optimizing this
implementation is interesting future work, but not urgent.
Use UBreakIterators to perform grapheme breaking. This gives Unicode 9
grapheme breaking (e.g. family emoji) and provides a means to upgrade
to future versions. It also serves as a model for how to serve up
other advanced functionality in ICU to users.
This has tricky performance implications. Some things are faster and a
number of cases are slower. But, careful use of ICU can help mitigate
and amortize these costs. In conjunction with more early detection of
fast paths, overall grapheme breaking for the average user should be
much faster than in Swift 3.
NOTE: This is incomplete. It currently falls back on the legacy tries
for some bridged strings. There are many potential directions for a
general solution, but for now we'll be interatively adding support for
more and more special cases.
* Give Sequence a top-level Element, constrain Iterator to match
* Remove many instances of Iterator.
* Fixed various hard-coded tests
* XFAIL a few tests that need further investigation
* Change assoc type for arrayLiteralConvertible
* Mop up remaining "better expressed as a where clause" warnings
* Fix UnicodeDecoders prototype test
* Fix UIntBuffer
* Fix hard-coded Element identifier in CSDiag
* Fix up more tests
* Account for flatMap changes
This adds more fast path checks for grapheme breaks between BMP
scalars. Notably the rather vast range of 0x3400–0xA4CF which includes
unified common Han ideographs as well as the first extension to
unified Han ideographs. It also happens to pick up various Yijin and
Yi symbols/radicals. Additionally, the narrow hiragana/katakana ranges
0x3041-0x3096 and 0x30A1-0x30FA (including pre-composed semi-voiced
characters but excluding the combining semi-voice marks) have fast
paths.
The net effect is that the vast majority of modern Chinese and
Japanese text should be fast-pathed. This is especially important, as
adopting Unicode 9 might otherwise pessimize performance here relative
to the tries.
Use UBreakIterators to perform grapheme breaking. This gives Unicode 9
grapheme breaking (e.g. family emoji) and provides a means to upgrade
to future versions. It also serves as a model for how to serve up
other advanced functionality in ICU to users.
This has tricky performance implications. Some things are faster and a
number of cases are slower. But, careful use of ICU can help mitigate
and amortize these costs. In conjunction with more early detection of
fast paths, overall grapheme breaking for the average user should be
much faster than in Swift 3.
NOTE: This is incomplete. It currently falls back on the legacy tries
for some bridged strings. There are many potential directions for a
general solution, but for now we'll be interatively adding support for
more and more special cases.
UnsafeBufferPoiunter subscript used in the fast path only checks bounds
in Debug mode, therefore extra checks are needed.
Addresses: <rdar://problem/31992473>
This adds a fast path for single-code-unit Character
construction. Rather than use the general purpose String based
initializer (which then repeats grapheme breaking to ensure a trap,
amongst other inefficiencies), just make the Character from the single
unicode scalar value directly.
This also speeds up simple iteration of BMP strings when the optimizer
is unable to eliminate the subscript. Around 2x for ASCII, and around
20% for BMP UTF16.
