* [Parser] Update 'Confusables.def' file to include confusable and base character names
* [Parser] Add a new utility method to return the names of the confusable and base characters for a given confusable codepoint
* [Parser] Update diagnostic for confusable character during lexing to mention confusable and base character names
* [Sema] If there is just a single confusable character, emit a tailored diagnostic that also mentions the character names
* [Diagnostics] Add new diagnostic messages to the localization file
* [Test] Update confusables test
* [Utils] Update unicode confusables txt file and update script to regenerate confusables def file
* [Parse] Regenerate 'Confusables.def' using updated script
* [Utils] Adjust generate_confusables script based on review feedback
Fix a mistake with name mapping. Updated header comment. Fix a couple of linting issues.
* [Parse] Regenerate 'Confusables.def' file once again after script changes
* [Parse] Add the newline after end of 'getConfusableAndBaseCodepointNames' method
* [Test] Update diagnostic message in 'Syntax/Parser/diags.swift'
The user experience with extended grapheme literals is currently:
1. Strict: we hard error on "invalid" grapheme literals.
2. Complete: we validate all literals to either be
known-single-grapheme or not.
3. Incorrect: we have Unicode 8 semantics implemented but applications
will have some other version of Unicode as dictated by the OS they are
running on.
In Swift 4.0, this incorrectness mostly crops up in obscure corner
case areas, where we are overly restrictive in some ways and overly
relaxed in others. But, there is one particularly embarrassing area
where it does come up: we reject emoji introduced after Unicode 8 as
grapheme literals, counter to common user expectations.
In a future (sub-)version of Swift we should completely re-evaluate
this user story, but doing so in time for Swift 4.0 is untenable. This
patch attempts to tweak the way in which we are incorrect in the most
minimally invasive way possible to preserve the same user experience
while permitting many post-Unicode-8 emoji as valid grapheme literals.
This change overrides processing of ZWJ and emoji modifiers to not
declare a grapheme break after/before, respectively.
Disables the portions of the regional break test file corresponding to
Unicode 8.0 vs 9.0 differences. Once the compiler and stdlib are in
sync with Unicode 9.0, and the legacy tries are dropped, we should
splat down newer versions of these test files.
Disables the portions of the regional break test file corresponding to
Unicode 8.0 vs 9.0 differences. Once the compiler and stdlib are in
sync with Unicode 9.0, and the legacy tries are dropped, we should
splat down newer versions of these test files.
This is only for the frontend, not for stdlib. The implementation is very
slow, optimizing it is the next step.
rdar://16755123 rdar://16013860
Swift SVN r18928
