Add ICU shims for creating UTexts and performing grapheme breaking
using UTexts. This will allow us to explore multi-encoding grapheme
support throughout string.
- numericValue returns nil instead of .nan for non-numerics
- Remove small-string optimizations from _scalarName that failed on 32-bit archs
- Put case mappings back into U.S.Properties
- Added more sanity tests
* Add partial range subscripts to _UnmanagedOpaqueString
* Use SipHash13+_NormalizedCodeUnitIterator for String hashes on all platforms
* Remove unecessary collation algorithm shims
* Pass the buffer to the SipHasher for ASCII
* Hash the ascii parts of UTF16 strings the same way we hash pure ascii strings
* De-dupe some code that can be shared between _UnmanagedOpaqueString and _UnmanagedString<UInt16>
* ASCII strings now hash consistently for in hashASCII() and hashUTF16()
* Fix zalgo comparison regression
* Use hasher
* Fix crash when appending to an empty _FixedArray
* Compact ASCII characters into a single UInt64 for hashing
* String: Switch to _hash(into:)-based hashing
This should speed up String hashing quite a bit, as doing it through hashValue involves two rounds of SipHash nested in each other.
* Remove obsolete workaround for ARC traffic
* Ditch _FixedArray<UInt8> in favor of _UIntBuffer<UInt64, UInt8>
* Bad rebase remnants
* Fix failing benchmarks
* michael's feedback
* clarify the comment about nul-terminated string hashes
Adjust the signature to match the ICU declaration for
`unorm2_normalize`. This was adjusted to allow building against ICU
59.1. The shim type definition for the UChar ensures that the signature
is correct on all the targets. NFC.
Restore (un-revert) sting comparison, with fixes
More exhaustive testing of opaque strings, which consistently reproduces prior sporadic failure. Shims fixups. Some test tweaking.
The default ICU build will change the underlying type of the UChar type,
with C++ using the builtin `char16_t` rather than `unsigned short`.
This adjusts the interface to account for that. I've verified across
Apple's implementation that they always use the `unsigned short` as the
type for `UChar`. Since we cannot guarantee that the ICU interfaces are
built the same way on all targets, especially when using the underlying
system's ICU.
Adjust the stubs implementation declaration to match the ICU header's
declaration.
Introduce shims for using UBreakIterators from ICU. Also introduce
shims for using thread local storage via pthreads.
We will be relying on ICU and UBreakIterators for grapheme
breaking. But, UBreakIterators are very expensive to create,
especially for the way we do grapheme breaking, which is relatively
stateless. Thus, we will stash one or more into thread local storage
and reset it as needed.
Note: Currently, pthread_key_t is hard coded for a single platform
(Darwin), but I have a static_assert alongside directions on how to
adapt it to any future platforms who differ in key type.
Introduce shims for using UBreakIterators from ICU. Also introduce
shims for using thread local storage via pthreads.
We will be relying on ICU and UBreakIterators for grapheme
breaking. But, UBreakIterators are very expensive to create,
especially for the way we do grapheme breaking, which is relatively
stateless. Thus, we will stash one or more into thread local storage
and reset it as needed.
Note: Currently, pthread_key_t is hard coded for a single platform
(Darwin), but I have a static_assert alongside directions on how to
adapt it to any future platforms who differ in key type.
Update for SE-0107: UnsafeRawPointer
This adds a "mutating" initialize to UnsafePointer to make
Immutable -> Mutable conversions explicit.
These are quick fixes to stdlib, overlays, and test cases that are necessary
in order to remove arbitrary UnsafePointer conversions.
Many cases can be expressed better up by reworking the surrounding
code, but we first need a working starting point.
This is a purely mechanical change replacing the attributes with the reserved
spelling. Compilers are to not error when they encounter a reserved spelling
for an attribute which they do not support.
The general rule here is that something needs to be SWIFT_CC(swift)
if it's just declared in Swift code using _silgen_name, as opposed to
importing something via a header.
Of course, SWIFT_CC(swift) expands to nothing by default for now, and
I haven't made an effort yet to add the indirect-result / context
parameter ABI attributes. This is just a best-effort first pass.
I also took the opportunity to shift a few files to just implement
their shims header and to demote a few things to be private stdlib
interfaces.
...and explicitly mark symbols we export, either for use by executables or for runtime-stdlib interaction. Until the stdlib supports resilience we have to allow programs to link to these SPI symbols.
These come in two categories of functions:
1. Comparison.
- swift_stdlib_unicode_compare_utf16_utf16
- swift_stdlib_unicode_compare_utf8_utf16
- swift_stdlib_unicode_compare_utf8_utf8
2. Hashing.
- swift_stdlib_unicode_hash
- swift_stdlib_unicode_hash_ascii
Revert "Add test cases to exercise the native String vs cocoa buffer String path."
Revert "stdlib: Add back a test I removed"
Revert "stdlib: Fix hasPrefix,hasSuffix tests"
Revert "stdlib: Add documentation for the cached ascii collation tables"
This reverts commit 31493, 31492, 31491, 31490, 31489.
There are linking errors in SwiftExternalProjects (we probably have to link
against libicucore somewhere).
Swift SVN r31543
Reapply of 31474 with a fix in _compareCocoaBuffer to use the bufferSizeRhs
variable instead of bufferSizeLhs for the right hand side buffer.
We no longer create intermediate NSString copies to compare and hash swift
Strings. Instead we call directly into the ICU library.
I measured a 1.2 to 2x improvement on dictionary benchmarks as a result of this.
The SuperChars benchmark is also about 1.2x faster because of this.
Pure ASCII comparison has gotten a little bit slower (20% on a pure comparison
micro-benchmark) because we no longer do a memcmp. Doing a memcmp on ASCII is
not the same as the default unicode collation. Instead we have to a string scan.
The default unicode collation does not order like ASCII does and ignores
characters (for example the \0 character).
rdar://18992510
Swift SVN r31489
Revert "stdlib: Add back a test I removed"
Revert "Add test cases to exercise the native String vs cocoa buffer String path."
Revert "stdlib: Move the darwin String implementation over to use the ICU library."
This reverts commit r31477, r31476, r31475, r31474.
Commit r31474 broke the ASAN build.
Swift SVN r31488
We no longer create intermediate NSString copies to compare and hash swift
Strings. Instead we call directly into the ICU library.
I measured a 1.2 to 2x improvement on dictionary benchmarks as a result of this.
The SuperChars benchmark is also about 1.2x faster because of this.
Pure ASCII comparison has gotten a little bit slower (20% on a pure comparison
micro-benchmark) because we no longer do a memcmp. Doing a memcmp on ASCII is
not the same as the default unicode collation. Instead we have to a string scan.
The default unicode collation does not order like ASCII does and ignores
characters (for example the \0 character).
rdar://18992510
Swift SVN r31474
The standard library has grown significantly, and we need a new
directory structure that clearly reflects the role of the APIs, and
allows future growth.
See stdlib/{public,internal,private}/README.txt for more information.
Swift SVN r25876
