String.Index has an encodedOffset-based initializer and computed
property that exists for serialization purposes. It was documented as
UTF-16 in the SE proposal introducing it, which was String's
underlying encoding at the time, but the dream of String even then was
to abstract away whatever encoding happend to be used.
Serialization needs an explicit encoding for serialized indices to
make sense: the offsets need to align with the view. With String
utilizing UTF-8 encoding for native contents in Swift 5, serialization
isn't necessarily the most efficient in UTF-16.
Furthermore, the majority of usage of encodedOffset in the wild is
buggy and operates under the assumption that a UTF-16 code unit was a
Swift Character, which isn't even valid if the String is known to be
all-ASCII (because CR-LF).
This change introduces a pair of semantics-preserving alternatives to
encodedOffset that explicitly call out the UTF-16 assumption. These
serve as a gentle off-ramp for current mis-uses of encodedOffset.
Defining a custom iterator for the UTF16View avoid some redundant
computation over the indexing model. This speeds up iteration by
around 40% on non-ASCII strings.
Provide a custom iterator rather than relying a the IndexingIterator,
as an indexing model is less efficient for stateful processing of
strings. Provides around a 30% speedup.
After rebasing on master and incorporating more 32-bit support,
perform a bunch of cleanup, documentation updates, comments, move code
back to String declaration, etc.
Refactor and rename _StringGutsSlice, apply NFC-aware fast paths to a
new buffered iterator.
Also, fix bug in _typeName which used to assume ASCIIness and better
SIL optimizations on StringObject.
Add inlinability annotations to restore performance parity with 4.2 String.
Take advantage of known NFC as a fast-path for comparison, and
overhaul comparison dispatch.
RRC improvements and optmizations.
* Refactor out RRC implementation into dedicated file.
* Change our `_invariantCheck` pattern to generate efficient code in
asserts builds and make the optimizer job's easier.
* Drop a few Bidi shims we no longer need.
* Restore View decls to String, workaround no longer needed
* Cleaner unicode helper facilities
This is a giant squashing of a lot of individual changes prototyping a
switch of String in Swift 5 to be natively encoded as UTF-8. It
includes what's necessary for a functional prototype, dropping some
history, but still leaves plenty of history available for future
commits.
My apologies to anyone trying to do code archeology between this
commit and the one prior. This was the lesser of evils.
_StringGuts shouldn't expose a subscript, implying efficient
access. Switch to the explicit code unit fetch method. Update tests
accordingly, and switch off of deprecated typealiases.
Move the shifts to index creation time rather than index comparison
time. This seems to benefit micro benchmarks and cover up
inefficiencies in our generic index distance calculations.
Drop append-related @inlinable annotations for String, StringGuts,
StringStorage, and the Views. Drop several for larger operations, such
as case conversion. Drop as many as we can from StringGuts for now.
Switch StringObject and StringGuts from opaquely storing tagged cocoa
strings into storing small strings. Plumb small string support
throughout the standard library's routines.
Use the visitor pattern in most of the opaque-by-hand call
sites. Inspecting the compiler output does not show excessive and
unanticipated ARC, but there may need to be further tweaks.
One downside of the visitor pattern as written is that there's extra
shuffling around of registers for the closure CC. Hopefully this will
also be fixed soon.
Stop inlining _asOpaque into user code. Inlining it bloats user code
as there's a bit-test-and-branch to a block containing the _asOpaque
call, followed up some operations to e.g. manipulate the range or
re-align the calling convention, etc., followed by a final branch to
opaque stdlib code.
Instead, branch directly into opaque stdlib code. In theory, this
means that supporting all opaque patterns can be done with minimal
bloat. On ARM, this is a single tbnz instruction.
Deprecated the `PlaygroundQuickLook` enum and `CustomPlaygroundQuickLookable`
protocol. These are being targeted for removal in Swift 5, so we want to
unconditionally deprecate them now to encourage use of
`CustomPlaygroundDisplayConvertible` instead.
This commit includes deprecated the various `CustomPlaygroundQuickLookable`
conformances across the standard library and overlay libraries.
Include the initial implementation of _StringGuts, a 2-word
replacement for _LegacyStringCore. 64-bit Darwin supported, 32-bit and
Linux support in subsequent commits.
In grand LLVM tradition, the first step to redesigning _StringCore is
to first rename it to _LegacyStringCore. Subsequent commits will
introduce the replacement, and eventually all uses of the old one will
be moved to the new one.
NFC.
* Nest various top-level Iterator and Index types, and flatten extensions.
* Fix tests from nesting iterator
* Nest Unsafe*BufferPointer.Iterator, extensionify UnsafeBufferPointer
* Degyb LazyCollection
* Nest Flatten iterator and index
* Eradicate IndexDistance associated type, replacing with Int everywhere
* Consistently use Int for ExistentialCollection’s IndexDistance type.
* Fix test for IndexDistance removal
* Remove a handful of no-longer-needed explicit types
* Add compatibility shims for non-Int index distances
* Test compatibility shim
* Move IndexDistance typealias into the Collection protocol
This necessary for ensuring the property that String doesn't keep
inaccessible memory alive. For example, before this change,
String(s.dropFirst().unicodeScalars)
would compile and produce a String that owned inaccessible memory.
Now it no longer compiles.
String's view's SubSequences are the same as the Substring's
view. E.g. String.UnicodeScalarView.SubSequence is
Substring.UnicodeScalarView.
New compatibility inits added, to work around the fact that many
previously failable initializers are now non-failable.
The simplest way to keep the string buffer alive is simply to always grab its
iterator, even if we're not going to use it. Thanks @milseman for that idea and
@jckarter for the diagnosis help!
I failed to merge the upstream changes to swift-corelibs-foundation at the same
time as I merged that #9806, and it broke on linux. Going to get it right this
time.
* Give Sequence a top-level Element, constrain Iterator to match
* Remove many instances of Iterator.
* Fixed various hard-coded tests
* XFAIL a few tests that need further investigation
* Change assoc type for arrayLiteralConvertible
* Mop up remaining "better expressed as a where clause" warnings
* Fix UnicodeDecoders prototype test
* Fix UIntBuffer
* Fix hard-coded Element identifier in CSDiag
* Fix up more tests
* Account for flatMap changes
