Commit Graph

12 Commits

Author SHA1 Message Date
Dmitri Hrybenko
9bd0b1e8d3 stdlib: fix some bugs in UTF-16/UTF-8 transcoder that is hidden inside String
implementation:

- don't crash on NSStrings that contain isolated surrogates (partial fix, there
  is a different code path that dispatches CoreFoundation and it still
  crashes);

- insert U+FFFD according to Unicode recommendation;

- fix decoding of surrogate pairs when the internal buffer is almost full
  (we used to drop characters in that case, rdar://16833733 + dups).


Swift SVN r19147
2014-06-24 21:59:19 +00:00
Dmitri Hrybenko
3cdc901354 stdlib/Unicode: fix UTF-32 decoder not to crash on invalid code units
Also implemented U+FFFD insertion in UTF-32 decoder according to Unicode
reccomendation.


Swift SVN r19092
2014-06-23 15:40:22 +00:00
Dmitri Hrybenko
2103b1d995 stdlib/Unicode: fix UTF-16 decoder not to crash on invalid code unit sequences
Also implemented U+FFFD insertion in UTF-16 decoder according to Unicode
reccomendation.


Swift SVN r19091
2014-06-23 14:52:24 +00:00
Dmitri Hrybenko
f370ca0746 stdlib: fix a bunch of various Unicode issues, primarily in UTF-8 decoding
In UTF-8 decoder:
- implement U+FFFD insertion according to the recommendation given in the
  Unicode spec.  This required changing the decoder to become stateful, which
  significantly increased complexity due to the need to maintain an internal
  buffer.
- reject invalid code unit sequences properly instead of crashing rdar://16767868
- reject overlong sequences rdar://16767911

In stdlib:
- change APIs that assume that UTF decoding can never fail to account for
  possibility of errors
- fix a bug in UnicodeScalarView that could cause a crash during backward
  iteration if U+8000 is present in the string
- allow noncharacters in UnicodeScalar.  They are explicitly allowed in the
  definition of "Unicode scalar" in the specification.  Disallowing noncharacters
  in UnicodeScalar prevents actually using these scalar values as internal
  special values during string processing, which is exactly the reason why they
  are reserved in the first place.
- fix a crash in String.fromCString() that could happen if it was passed a null
  pointer

In Lexer:
- allow noncharacters in string literals.  These Unicode scalar values are not
  allowed to be exchanged externally, but it is totally reasonable to have them
  in literals as long as they don't escape the program.  For example, using
  U+FFFF as a delimiter and then calling str.split("\uffff") is completely
  reasonable.

This is a lot of changes in a single commit; the primary reason why they are
lumped together is the need to change stdlib APIs to account for the
possibility of UTF decoding failure, and this has long-reaching effects
throughout stdlib where these APIs are used.


Swift SVN r19045
2014-06-20 13:07:40 +00:00
Chris Lattner
70076cf958 switch the testsuite to use the ..< operator instead of ..
Swift SVN r19003
2014-06-19 17:18:23 +00:00
Dave Abrahams
68abebdffb [stdlib] withUnsafePointerToElements for Array
Also privatize ArrayType's elementStorage. Per Array API review.

Swift SVN r18330
2014-05-18 16:59:46 +00:00
Dave Abrahams
92307dee66 [stdlib] Array API Review: swap init parameters
Swift SVN r18325
2014-05-18 15:54:33 +00:00
Arnold Schwaighofer
da6d9152b6 Differentiate between user assertion and preconditions and the like
assert() and fatalError()
These functions are meant to be used in user code. They are enabled in debug
mode and disabled in release or fast mode.

_precondition() and _preconditionFailure()
These functions are meant to be used in library code to check preconditions at
the api boundry. They are enabled in debug mode (with a verbose message) and
release mode (trap). In fast mode they are disabled.

_debugPrecondition() and _debugPreconditionFailure()
These functions are meant to be used in library code to check preconditions that
are not neccesarily comprehensive for safety (UnsafePointer can be null or an
invalid pointer but we can't check both). They are enabled only in debug mode.

_sanityCheck() and _fatalError()
These are meant to be used for internal consistency checks. They are only
enabled when the library is build with -DSWIFT_STDLIB_INTERNAL_CHECKS=ON.

I modified the code in the standard library to the best of my judgement.

rdar://16477198

Swift SVN r18212
2014-05-16 20:49:54 +00:00
Ted Kremenek
9eea282719 Switch range operators ".." and "...".
- 1..3 now means 1,2
- 1...3 now means 1,2,3

Implements <rdar://problem/16839891>

Swift SVN r18066
2014-05-14 07:36:00 +00:00
Dmitri Hrybenko
2cc8fe40d4 stdlib/printing: replace four printing systems with one new one
The old ones were:

- print/println
- printAny
- printf
- Console

The new printing story is just print/println.  Every object can be printed.
You can customize the way it is printed by adopting Printable protocol.  Full
details in comments inside stdlib/core/OutputStream.swift.

Printing is not completely finished yet.  We still have ReplPrintable, which
should be removed, string interpolation still uses String constructors, and
printing objects that don't conform to Printable will result in printing
mangled names.


Swift SVN r18001
2014-05-13 13:07:59 +00:00
Ted Kremenek
fad874708e Adjust test cases.
Swift SVN r17964
2014-05-12 22:01:52 +00:00
Dave Abrahams
38b9e6808f [stdlib] CharacterEncoding.swift => Unicode.swift
Also enshrine the tests in the test suite

Swift SVN r10057
2013-11-08 22:17:50 +00:00