RFC 2279 states that, in UTF-8:
"The octet values FE and FF never appear."
RFC 3629 states that, in UTF-8:
"The octet values C0, C1, F5 to FF never appear."
Generalize the check to advance past invalid starting bytes for
a UTF-8 sequence to fix a crash in the lexer.
Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.
Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.
Trivia are now data attached to the ends of tokens, not tokens
themselves.
Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.
Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:
fuzzer => token stream => file1
=> Lexer => token stream => file 2 => diff(file1, file2)
Will arrive in a subsequent commit.
This patch does not change the grammar.
Like cursor-info, range info (""source.request.cursorinfo"") answers some
questions clients have for a code snippet under selection, for instance, the type of a selected
expression. This commit implements this new quest kind and provides two
simple information about the selected code: (1) the kind of the
snippet, currently limited to single-statement and expression; and (2)
the type of the selected expression. Gradually, we will enrich the
response to provide more insight into the selected code snippet.
When in Swift 3 Compatibility Mode we now acceptable a standalone
'$' as an identifier. In all other cases this is now disallowed
and must be surrounded by backticks.
Previously, builtin operators followed by comment block were tokenized as
normal operators (e.g. tok::oper_binary_spaced) instead of dedicated
token(e.g. tok::equal).
That used to cause strange parse errors:
test.swift:1:3: error: use of unresolved operator '='
_ =/* */2
^
silently, have the lexer return it as an unknown token. Enhance the expr parser to
detect these things and squash any expression in progress into an ErrorExpr. This
allows us to silence really bad downstream errors. For example, on:
struct S { func f() {} }
func f() {
_ = S.
}
we formerly produced:
x.swift:5:8: error: expected member name following '.'
x.swift:5:7: error: expected member name or constructor call after type name
x.swift:5:7: note: add arguments after the type to construct a value of the type
x.swift:5:7: note: use '.self' to reference the type object
we now emit just the first one. This fixes:
<rdar://problem/22290244> QoI: "UIColor." gives two issues, should only give one
Lexing for conflict markers is inspired by the way clang handles them with a few
updates of our own. Clang's lexer currently searches for the start of the
conflict marker, attempts to find the divider points, lexes that, then finds the
end. We, unfortunately, cannot be so forgiving because of operator overloads.
Instead, we search for the start and end markers and ignore all text in between.
Even if what is found is not conflict markers, it certainly is not valid Swift either.
* Implement the majority of parsing support for SE-0039.
* Parse old object literals names using new syntax and provide FixIt.
For example, parse "#Image(imageLiteral:...)" and provide a FixIt to
change it to "#imageLiteral(resourceName:...)". Now we see something like:
test.swift:4:9: error: '#Image' has been renamed to '#imageLiteral
var y = #Image(imageLiteral: "image.jpg")
^~~~~~ ~~~~~~~~~~~~
#imageLiteral resourceName
Handling the old syntax, and providing a FixIt for that, will be handled in a separate
commit.
Needs tests. Will be provided in later commit once full parsing support is done.
* Add back pieces of syntax map for object literals.
* Add parsing support for old object literal syntax.
... and provide fixits to new syntax.
Full tests to come in later commit.
* Improve parsing of invalid object literals with old syntax.
* Do not include bracket in code completion results.
* Remove defunct code in SyntaxModel.
* Add tests for migration fixits.
* Add literals to code completion overload tests.
@akyrtzi told me this should be fine.
* Clean up response tests not to include full paths.
* Further adjust offsets.
* Mark initializer for _ColorLiteralConvertible in UIKit as @nonobjc.
* Put attribute in the correct place.
Previously, comments were treated as non-whitespace. Operators
also checked for right-boundedness before detecting comments
which start in the middle of an operator.
This resolves both of these issues so that comments are
consistently treated as whitespace when determining whether
operators are left- or right-bound.
Fixes SR-186 and SR-960 (SE-0037)
The if statements check whether a number literal consists
of just the 0b/0o/0x prefix which would be invalid. However, in all three cases there's a preceding check
that already addresses this issue.
This will be used to help IRGen record protocol requirements
with resilient default implementations in protocol metadata.
To enable testing before all the Sema support is in place, this
patch adds SIL parser, printer and verifier support for default
witness tables.
For now, SILGen emits empty default witness tables for protocol
declarations in resilient modules, and IRGen ignores them when
emitting protocol metadata.
We were miscalculating 'isRightBound' when the RHS was a code-completion
token leading to missing completions in unspaced binary expressions
1+<here>
1...<here>
rdar://problem/24278699
A char is unsigned by default on PowerPC. CurPtr is cast
to a signed char whenever a signed comparison is required,
but we are missing one place.
This fixes a testcase failure in Parse/BOM.swift
This adds some heuristics so we can emit a fixit to remove extraneous
whitespace after a . and diagnose the case where a member just hasn't
been written yet better. This also improves handling of tok::unknown
throughout the parser a bit.
This is a re-commit of ff4ea54 with an update for a SourceKit test.
This adds some heuristics so we can emit a fixit to remove extraneous
whitespace after a . and diagnose the case where a member just hasn't
been written yet better. This also improves handling of tok::unknown
throughout the parser a bit.
This is a case that the operator splitting code didn't handle because of
the bizarre lexer code for handling operators with periods in them. We had
accreted some weird special case logic for what is valid in an operator
(but which even had a special case hack for ..<). The policy is now very
simple: if an operator name starts with a dot, it is allowed to include other
dots in its name. If it doesn't, it doesn't. This allows us to get lexer
level operator splitting in cases like x=.Foo, allowing our existing operator
set that have dots in them without hacks, and provides a superior QoI
experience due to operator splitting.
This is technically a language change, but the TSPL operator grammar was
incorrect for it anyway. I will file an internal radar to get TSPL updated
with the actual behavior (now that it is defensible).
This is a frequently reported and surprising issue where lack of whitespace leads to
rejecting common code like "X*-4". Fix this by diagnosing it specifically as a lack
of whitespace problem, including a fixit to insert the missing whitespace (to transform
it into "X * -4". This even handles the cases where there are multiple valid (single)
splits possible by emitting a series of notes.
mode (take 2)
Allow untyped placeholder to take arbitrary type, but default to Void.
Add _undefined<T>() function, which is like fatalError() but has
arbitrary return type. In playground mode, merely warn about outstanding
placeholders instead of erroring out, and transform placeholders into
calls to _undefined(). This way, code with outstanding placeholders will
only crash when it attempts to evaluate such placeholders.
When generating constraints for an iterated sequence of type T, emit
T convertible to $T1
$T1 conforms to SequenceType
instead of
T convertible to SequenceType
This ensures that an untyped placeholder in for-each sequence position
doesn't get inferred to have type SequenceType. (The conversion is still
necessary because the sequence may have IUO type.) The new constraint
system precipitates changes in CSSimplify and CSDiag, and ends up fixing
18741539 along the way.
(NOTE: There is a small regression in diagnosis of issues like the
following:
class C {}
class D: C {}
func f(a: [C]!) { for _: D in a {} }
It complains that [C]! doesn't conform to SequenceType when it should be
complaining that C is not convertible to D.)
<rdar://problem/21167372>
(Originally Swift SVN r31481)
In addition to just a whitespace char, also look ahead to a null
character to lex the #endif token. This allows the token to be formed
when at the end of a file.
Fix endif-on-end-of-file.swift test file to not end in whitespace.
rdar://problem/21692337
Swift SVN r32224
Emit a fix-it replacing them with double-quote string literals.
<rdar://problem/21950709> QoI: Parse single-quoted literals like double-quoted literals
Swift SVN r31973
Allow untyped placeholder to take arbitrary type, but default to Void.
Add _undefined<T>() function, which is like fatalError() but has
arbitrary return type. In playground mode, merely warn about outstanding
placeholders instead of erroring out, and transform placeholders into
calls to _undefined(). This way, code with outstanding placeholders will
only crash when it attempts to evaluate such placeholders.
<rdar://problem/21167372> transform EditorPlaceholderExpr into fatalError()
Swift SVN r31481
Teach skipToEndOfInterpolatedExpression() to match quote marks as well
as parentheses in the interpolated expression. This makes expressions
like "hello \(names["bob"])" possible.
Swift SVN r31283
Sink the actual logic for omitting needless words way down into
Basic, so we can re-use it elsewhere. Tie the Clang importer into that
logic, mapping Clang types down to strings appropriately. NFC
Swift SVN r31233
Also, use 'isHexDigit' instead of the built-in 'isxdigit' for consistency,
even though 'isxdigit' is locale-independent.
No intended functionality change.
Swift SVN r31139
