There's a longstanding problem in implementing `-isEqualToString:`,
where if you don't know how to get fast access to the other NSString's
contents, you have to pick between doing it character by character (very
slow), or calling [other isEqualToString: self], which risks infinite
recursion if the other string does the same.
This cuts the gordian knot by adding a new method
`isEqualToBytes:encoding:count:`, so you can get the contents out of
`self`, and hand it to the other string, confident that it will not need
to (nor, in fact be able to) ask you anything that might recurse.
Functions like withUnsafeBufferPointer are, by themselves, safe to
call. It's only the operations on the unsafe pointers passed into the
closure that are the safety issue.
This was the intent spelled out in SE-0458 but was not fully realized
in the library.
Fixes rdar://174519372.
Previously, we skipped checking the return type of a function for safety
as we expected to warn at the use of the returned value:
let x = returnsUnsafe()
usesUnsafe(x) // warn here
Unfortunately, this resulted in missing some unsafe constructs that can
introduce memory safety issues when the use of the return value had a
different shape resulting in false negatives for cases like:
return returnsUnsafe()
or
usesUnsafe(returnsUnsafe())
This PR changes the analysis to always take return types of function
calls into account.
rdar://157237301
Speed up conversion between UTF-16 offset ranges
and string index ranges, by carefully switching
between absolute and relative index calculations,
depending on the distance we need to go.
It is a surprisingly tricky puzzle to do this
correctly while avoiding redundant calculations.
Offset ranges within substrings add the additional
complication of having to bias offset values with
the absolute offset of the substring’s start index.
Commit the platform definition and build script work necessary to
cross-compile for arm64_32.
arm64_32 is a variant of AARCH64 that supports an ILP32 architecture.
withUTF8 currently vends a typed UInt8 pointer to the underlying
SmallString. That pointer type differs from SmallString's
representation. It should simply vend a raw pointer, which would be
both type safe and convenient for UTF8 data. However, since this
method is already @inlinable, I added calls to bindMemory to prevent
the optimizer from reasoning about access to the typed pointer that we
vend.
rdar://67983613 (Undefinied behavior in SmallString.withUTF8 is miscompiled)
Additional commentary:
SmallString creates a situation where there are two types, the
in-memory type, (UInt64, UInt64), vs. the element type,
UInt8. `UnsafePointer<T>` specifies the in-memory type of the pointee,
because that's how C works. If you want to specify an element type,
not the in-memory type, then you need to use something other than
UnsafePointer to view the memory. A trivial `BufferView<UInt8>` would
be fine, although, frankly, I think UnsafeRawPointer is a perfectly
good type on its own for UTF8 bytes.
Unfortunately, a lot of the UTF8 helper code is ABI-exposed, so to
work around this, we need to insert calls to bindMemory at strategic
points to avoid undefined behavior. This is high-risk and can
negatively affect performance. So far, I was able to resolve the
regressions in our microbenchmarks just by tweaking the inliner.
* Don't allocate breadrumbs pointer if under threshold
* Increase breadrumbs threshold
* Linear 16-byte bucketing until 128 bytes, malloc_size after
* Allow cap less than _SmallString.capacity (bridging non-ASCII)
This change decreases the amount of heap usage for moderate-length
strings (< 64 UTF-8 code units in length) and increases the amount of
spare code unit capacity available (less growth needed).
Average improvements for moderate-length strings:
* 64-bit: on average, 8 bytes saved and 4 bytes of extra capacity
* 32-bit: on average, 4 bytes saved and 6 bytes of extra capacity
Additionally, on 32-bit, large-length strings also gain an average of
6 bytes of extra spare capacity.
Details:
On 64-bit, half of moderate-length allocations will save 16 bytes
while the other half get an extra 8 bytes of spare capacity.
On 32-bit, a quarter of moderate-length allocations will save 16
bytes, and the rest get an extra 4 bytes of spare
capacity. Additionally, 32-bit string's storage class now claims its
full allocation, which is its birthright. Prior to this change, we'd
have on average 1.5 bytes of spare capacity, and now we have 7.5 bytes
of spare capacity.
Breadcrumbs threshold is increased from the super-conservative 32 to
the pretty-conservative 64. Some speed improvements are incorporated
in this change, but more are in flight. Even without those eventual
improvements, this is a worthwhile change (ASCII is still fast-pathed
and irrelevant to breadcrumbing).
For a complex real-world workload, this amounts to around a 5%
improvement to transient heap usage due to all strings and a 4%
improvement to peak heap usage due to all strings. For moderate-length
strings specifically, this gives around 11% improvement to both.
It is causing bots to fail.
* Revert "The __has_include(<os/system_version.h>) branch here wasn't quite right, we'll just use the dlsym one for now"
This reverts commit f824922456.
* Revert "Remove stdlib and runtime dependencies on Foundation and CF"
This reverts commit 3fe46e3f16.
rdar://54709269
One additional tweak (setting the scalar-aligned bit on foreign indices) had to be made to avoid a performance regression for long non-ASCII foreign strings.
David Smith
Allan Shortlidge
Doug Gregor
Gabor Horvath
Kuba Mracek
Alejandro Alonso
Karoy Lorentey
Hassan
Robert Widmann
Andrew Trick
Károly Lőrentey
Mishal Shah
Michael Ilseman
Alexis Laferrière
Max Desiatov
Luciano Almeida
Arnold Schwaighofer