Commit Graph

7182 Commits

Author SHA1 Message Date
Karoy Lorentey
b29d8f4805 [stdlib] Substring: restrict grapheme breaking to the bounds of the substring
(Oops)
2022-03-29 20:00:08 -07:00
Karoy Lorentey
9714f97ad8 [stdlib] Substring: round indices down to nearest character in indexing operations
Distances between indices aren’t well-defined without this.
2022-03-29 20:00:08 -07:00
Karoy Lorentey
755712a25d [stdlib] StringGuts.replaceSubrange: Fast path for replacing with a fast substring
If the replacement collection is a fast UTF-8 substring, we can simply
access its backing store directly — we don’t need to use a circuituous
lazy algorithm.
2022-03-29 20:00:08 -07:00
Karoy Lorentey
dc6990370e [stdlib] StringGuts.scalarAlign: Preserve encoding flags in returned index 2022-03-29 20:00:08 -07:00
Karoy Lorentey
4eab8355ca [stdlib] String: prefer passing ranges to start+end argument pairs 2022-03-29 20:00:08 -07:00
Karoy Lorentey
4aae824124 [stdlib] String: Deprecate old bounds checking methods
These weren’t doing the right thing, and all callers have now
migrated to the new `_StringGuts.validate*` methods, which combine
bounds checks with encoding validation and scalar alignment.
2022-03-29 20:00:08 -07:00
Karoy Lorentey
e8212690d1 [stdlib] String: Apply transcoded offset when converting indices from UTF-16 2022-03-29 20:00:08 -07:00
Karoy Lorentey
4ad8b26ab3 [stdlib] String.UTF16View: Review/fix index validation
Also, in UTF-16 slices, forward collection methods to the base view
instead of `Slice`, to make behavior a bit easier to understand.

(There is no need to force readers to page in `Slice`
implementations _in addition to_ whatever the base view is doing.)
2022-03-29 20:00:08 -07:00
Karoy Lorentey
5f6c300adb [stdlib] String.UTF8View: Review/fix index validation
Also, in UTF-8 slices, forward collection methods to the base view
instead of `Slice`, to make behavior a bit easier to understand.

(There is no need to force readers to page in `Slice`
implementations _in addition to_ whatever the base view is doing.)
2022-03-29 18:40:25 -07:00
Karoy Lorentey
8610bdf515 [stdlib] String.unicodeScalars: Add a _modify accessor
This will eliminate unnecessary CoW copies when calling mutating
Unicode scalar view methods directly through this property.
2022-03-29 18:40:25 -07:00
Karoy Lorentey
d58811262d [stdlib] String.UnicodeScalarView: Review index validation 2022-03-29 18:40:25 -07:00
swift_jenkins
b760e46c74 Merge remote-tracking branch 'origin/main' into next 2022-03-29 11:00:34 -07:00
Guillaume Lessard
3e739f62fc Merge pull request #42076 from glessard/se-withMemoryRebound
[stdlib] remove preconditions from compatibility entry point
2022-03-29 11:41:45 -06:00
Guillaume Lessard
ce7d8134a6 [stdlib] remove preconditions from compatibility entry point
- The old implementation of `UnsafePointer.withMemoryRebound` had no preconditions.
- When implementing SE-0333, we forwarded the old entry point to the new implementation, which has preconditions.
- This change removes the precondition from the compatibility entry point, reverting it to its previous behaviour.
- This resolves rdar://90462471
2022-03-28 18:30:11 -06:00
swift_jenkins
be50947706 Merge remote-tracking branch 'origin/main' into next 2022-03-25 03:20:42 -07:00
Alastair Houghton
eb6f9e0bba Merge pull request #41452 from al45tair/eng/PR-89139049
[Demangler] Make Node::addChild(NULL, ...) always assert.
2022-03-25 10:03:59 +00:00
Karoy Lorentey
67f01a1159 [stdlib] Stop inlining String.subscript
`index(after:)`/`index(before:)` aren’t inlinable, so I don’t expect
force-inlining the subscript has much benefit.
2022-03-24 21:00:00 -07:00
Karoy Lorentey
298899264d [stdlib] String: Add some extra invariant checks 2022-03-24 21:00:00 -07:00
Karoy Lorentey
90fee621b6 [stdlib] String.UTF16View: Mark foreign indices as UTF-16 encoded 2022-03-24 21:00:00 -07:00
Karoy Lorentey
98d5959478 [stdlib] String.Index: Adjust printing 2022-03-24 21:00:00 -07:00
Karoy Lorentey
1326c43b0f [stdlib][NFC] Update some outdated comments 2022-03-24 21:00:00 -07:00
Karoy Lorentey
0523b67e1f [stdlib] String.index(_:offsetBy:limitedBy:): compare limit against original index
Whether the limit actually applies depends on how it’s ordered
relative to the original index `i`, not the one we round down to the
nearest Character.
2022-03-24 21:00:00 -07:00
Karoy Lorentey
c436654b61 [stdlib] Substring._characterStride(startingAt:): Limit stride to the correct bounds 2022-03-24 21:00:00 -07:00
Karoy Lorentey
6d400c81a2 [stdlib] Substring: remove _encodedOffsetRange in favor of existing _offsetRange 2022-03-24 21:00:00 -07:00
Karoy Lorentey
2464aa681e [stdlib] String: Ensure indices are marked scalar aligned before rounding down to Character 2022-03-24 21:00:00 -07:00
Karoy Lorentey
5a22ceb72b [stdlib] _StringGutsSlice: Small adjustments 2022-03-24 21:00:00 -07:00
Karoy Lorentey
a3435704f0 [stdlib][NFC] String normalization: fix terminology (index ⟹ offset) 2022-03-24 21:00:00 -07:00
Karoy Lorentey
321284e9a9 [stdlib] Review & fix index validation during String index conversions
- Validate that the index has the same encoding as the string
- Validate that the index is within bounds
2022-03-24 21:00:00 -07:00
Karoy Lorentey
0c0cbe290d [stdlib] _StringGutsSlice: Don’t mark methods on non-@usableFromInline internal types @inlinable
(This really ought to be diagnosed by the compiler.)
2022-03-24 21:00:00 -07:00
Karoy Lorentey
6245da2457 [stdlib] Substring: Be consistent about how we refer to the underlying string
Prefer direct stored properties to computed ones — there is no reason
to risk inlining issues, esp. since things like `Slice.base` aren’t
even force-inlined.

Prefer using `_wholeGuts` to spelling out the full incantation.
2022-03-24 21:00:00 -07:00
Karoy Lorentey
836bf9ad73 [stdlib] Mark index encodings in String.UTF8View & UTF16View 2022-03-24 21:00:00 -07:00
Karoy Lorentey
8ab2379946 [stdlib] Round indices down to nearest Character in String’s index algorithms
To prevent unaligned indices from breaking well-defined index distance
and index offset calculations, round every index down to the nearest
whole Character.

For the horrific details, see the forum discussion below.

https://forums.swift.org/t/string-index-unification-vs-bidirectionalcollection-requirements/55946

To avoid rounding from regressing String performance in the regular
case (when indices aren’t being passed across string views), introduce
a new String.Index flag bit that indicates that the index is already
Character aligned.
2022-03-24 21:00:00 -07:00
Karoy Lorentey
87073f2af8 [stdlib] Substring.replaceSubrange: fix startIndex/endIndex adjustment
This used to forward to `Slice.replaceSubrange`, but that’s a generic algorithm that isn’t aware of the pecularities of Unicode extended grapheme clusters, and it can be mislead by unusual cases, like a substring or subrange whose bounds aren’t `Character`-aligned, or a replacement string that starts with a continuation scalar.
2022-03-24 21:00:00 -07:00
Karoy Lorentey
a44997eeea [stdlib] Factor scalar-aligned String index validation out into a set of common routines
There are three flavors, corresponding to i < endIndex, i <= endIndex, and range containment checks.
Additionally, we have separate variants for index validation in substrings.
2022-03-24 21:00:00 -07:00
Karoy Lorentey
15c7721caf [stdlib] Use the new index encoding flags when marking the encoding of indices
This removes an unnecessary opaque call from the inlinable path, but it preserves a runtime version check.
2022-03-24 20:59:59 -07:00
Karoy Lorentey
6e18955f90 [stdlib] Add bookkeeping to keep track of the encoding of strings and indices
Assign some previously reserved bits in String.Index and _StringObject to keep track of their associated storage encoding (either UTF-8 or UTF-16).

None of these bits will be reliably set in processes that load binaries compiled with older stdlib releases, but when they do end up getting set, we can use them opportunistically to more reliably detect cases where an index is applied on a string with a mismatching encoding.

As more and more code gets recompiled with 5.7+, the stdlib will gradually become able to detect such issues with complete accuracy.

Code that misuses indices this way was always considered broken; however, String wasn’t able to reliably detect these runtime errors before. Therefore, I expect there is a large amount of broken code out there that keeps using bridged Cocoa String indices (UTF-16) after a mutation turns them into native UTF-8 strings. Therefore, instead of trapping, this commit silently corrects the issue, transcoding the offsets into the correct encoding.

It would probably be a good idea to also emit a runtime warning in addition to recovering from the error. This would generate some noise that would gently nudge folks to fix their code.

rdar://89369680
2022-03-24 20:59:59 -07:00
Karoy Lorentey
683b9fa021 [stdlib] Adjust/fix String’s indexing operations to deal with the consequences of SE-0180 2022-03-24 20:59:59 -07:00
Alastair Houghton
71efd95052 [Demangler][Runtime] Give the demangler its own error handling.
The demangling library can't use the error handling from the main runtime
because it isn't always linked with it.  However, it's useful to have
some error handling, and in particular to be able to get data into the
crash logs.

This is complicated because of the way the demangling library gets used,
the upshot of which is that I've had to add a second object library just
for libswiftCore's use, so that the demangler will use the runtime's
error handling functions when present, and fall back on its own when
they aren't.

rdar://89139049
2022-03-24 13:05:13 +00:00
swift_jenkins
cfa14e3634 Merge remote-tracking branch 'origin/main' into next 2022-03-23 14:21:51 -07:00
David Smith
cffe105d98 Merge pull request #41964 from Catfish-Man/single-instruction-multiple-ifdef
Only use SIMD when stdlib vector types are available
2022-03-23 14:05:13 -07:00
swift_jenkins
1dd02c09e8 Merge remote-tracking branch 'origin/main' into next 2022-03-22 19:00:53 -07:00
David Smith
398de941b6 Merge pull request #41897 from Catfish-Man/we-control-the-horizontal
Stay in vectors longer before doing a horizontal sum
2022-03-22 18:54:40 -07:00
David Smith
c05e47dd60 Only use SIMD when stdlib vector types are available 2022-03-22 15:48:24 -07:00
swift_jenkins
76f91dbe18 Merge remote-tracking branch 'origin/main' into next 2022-03-18 16:41:36 -07:00
David Smith
36b41d4940 Merge pull request #41869 from Catfish-Man/stack-promotion-and-ar-raise
Use withUnsafeTemporaryAllocation instead of a temporary Array
2022-03-18 16:27:31 -07:00
David Smith
dbaada435c Stay in vectors longer before doing a horizontal sum 2022-03-18 15:27:40 -07:00
swift_jenkins
1e556247b1 Merge remote-tracking branch 'origin/main' into next 2022-03-18 10:01:20 -07:00
David Smith
cb082e185c Merge pull request #41866 from Catfish-Man/what-a-crumb-y-optimization
Vectorize UTF16 offset calculations
2022-03-18 09:53:55 -07:00
David Smith
9ad3c9a5db Use withUnsafeTemporaryAllocation instead of a temporary Array 2022-03-17 17:08:05 -07:00
David Smith
eaf3f316ec Vectorize UTF16 offset calculations 2022-03-17 14:18:21 -07:00