averello/swift-mirror
1
0
Fork 0
mirror of https://github.com/apple/swift.git synced 2025-12-21 12:14:44 +01:00
Code Issues Packages Projects Releases Wiki Activity

Be a little more permissive in emoji grapheme literals.

Browse Source 
The user experience with extended grapheme literals is currently:

1. Strict: we hard error on "invalid" grapheme literals.

2. Complete: we validate all literals to either be
known-single-grapheme or not.

3. Incorrect: we have Unicode 8 semantics implemented but applications
will have some other version of Unicode as dictated by the OS they are
running on.

In Swift 4.0, this incorrectness mostly crops up in obscure corner
case areas, where we are overly restrictive in some ways and overly
relaxed in others. But, there is one particularly embarrassing area
where it does come up: we reject emoji introduced after Unicode 8 as
grapheme literals, counter to common user expectations.

In a future (sub-)version of Swift we should completely re-evaluate
this user story, but doing so in time for Swift 4.0 is untenable. This
patch attempts to tweak the way in which we are incorrect in the most
minimally invasive way possible to preserve the same user experience
while permitting many post-Unicode-8 emoji as valid grapheme literals.

This change overrides processing of ZWJ and emoji modifiers to not
declare a grapheme break after/before, respectively.
This commit is contained in:
Michael Ilseman
2017-07-21 13:33:03 -07:00
parent cabd44d6ca
commit f88fb9a97a
3 changed files with 26 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
lib/Basic/Unicode.cpp
View File

@@ -16,6 +16,20 @@
using namespace swift;
// HACK: Allow support for many newer emoji by overriding behavior of ZWJ and
// emoji modifiers. This does not make the breaks correct for any version of
// Unicode, but shifts the ways in which it is incorrect to be less harmful.
//
// TODO: Remove this hack and reevaluate whether we should have any static
// notion of what a grapheme is.
//
// Returns true if lhs and rhs shouldn't be considered as having a grapheme
// break between them. That is, whether we're overriding the behavior of the
// hard coded Unicode 8 rules surrounding ZWJ and emoji modifiers.
static inline bool graphemeBreakOverride(llvm::UTF32 lhs, llvm::UTF32 rhs) {
return lhs == 0x200D || (rhs >= 0x1F3FB && rhs <= 0x1F3FF);
}
StringRef swift::unicode::extractFirstExtendedGraphemeCluster(StringRef S) {
// Extended grapheme cluster segmentation algorithm as described in Unicode
// Standard Annex #29.
@@ -53,7 +67,8 @@ StringRef swift::unicode::extractFirstExtendedGraphemeCluster(StringRef S) {
GraphemeClusterBreakProperty GCBForC1 =
getGraphemeClusterBreakProperty(C[1]);
if (isExtendedGraphemeClusterBoundary(GCBForC0, GCBForC1))
if (isExtendedGraphemeClusterBoundary(GCBForC0, GCBForC1) &&
!graphemeBreakOverride(C[0], C[1]))
return S.slice(0, C1Offset);
C[0] = C[1];