Update error messages to mention the invalid character.
Improve the diagnostic of floating point exponents.
Add tests for error messages when parsing floating point exponents.
Update existing tests for new error messages.
Rephrased error message to indicate which character is unexpected.
Provide error message variations when parsing binary, octal, decimal (default), and hexadecimal integer literals.
Look for unexpected digits in binary and octal integer literals.
Look for unexpected letters in hex integer literals.
Resolves: SR-5236 rdar://problem/32858684
Previously, users of TokenSyntax would always deal with RC<TokenSyntax>
which is a subclass of RawSyntax. Instead, provide TokenSyntax as a
fully-realized Syntax node, that will always exist as a leaf in the
Syntax tree.
This hides the implementation detail of RawSyntax and SyntaxData
completely from clients of libSyntax, and paves the way for future
generation of Syntax nodes.
Maintain inner most string literal mode to determine whether we allow
newline character or not.
* Disallow newline after multiline string in string interpolation. (SR-5171)
* Allow unbalanced `"` in multiline string in string interpolation.
* [Parse] Refactored internal structure of Tokens.def and documented usage.
Added a level of structure to the macro definitions to allow Swift
keywords to be cleanly accessed separately from SIL and Swift keywords
together. Documented structure and usage.
* [Parse] Made use of new guarantees and abstractions in Tokens.def
Used guarantees about undefining macros after import and new
SWIFT_KEYWORD abstraction to simplify usage of the Token.def
imports.
* Gardening
This introduces a few unfortunate things because the syntax is awkward.
In particular, the period and following token in \.[a], \.? and \.! are
token sequences that don't appear anywhere else in Swift, and so need
special handling. This is somewhat compounded by \foo.bar.baz possibly
being \(foo).bar.baz or \(foo.bar).baz (parens around the type), and,
furthermore, needing to distinguish \Foo?.bar from \Foo.?bar.
rdar://problem/31724243
This adds support for SE-0168, multi-line string literals.
Extend the lexer to recognize the new literals. Test cases added.
There are still areas for future diagnostic improvement, such as fixits and notes as to why a multi-line string literal will be malformed. Multi-line literals are explicitly forbidden inside of string interpolation, though this may be relaxed in the future.
The Swift 4 Migrator is invoked through either the driver and frontend
with the -update-code flag.
The basic pipeline in the frontend is:
- Perform some list of syntactic fixes (there are currently none).
- Perform N rounds of sema fix-its on the primary input file, currently
set to 7 based on prior migrator seasons. Right now, this is just set
to take any fix-it suggested by the compiler.
- Emit a replacement map file, a JSON file describing replacements to a
file that Xcode knows how to understand.
Currently, the Migrator maintains a history of migration states along
the way for debugging purposes.
- Add -emit-remap frontend option
This will indicate the EmitRemap frontend action.
- Don't fork to a separte swift-update binary.
This is going to be a mode of the compiler, invoked by the same flags.
- Add -disable-migrator-fixits option
Useful for debugging, this skips the phase in the Migrator that
automatically applies fix-its suggested by the compiler.
- Add -emit-migrated-file-path option
This is used for testing/debugging scenarios. This takes the final
migration state's output text and writes it to the file specified
by this option.
- Add -dump-migration-states-dir
This dumps all of the migration states encountered during a migration
run for a file to the given directory. For example, the compiler
fix-it migration pass dumps the input file, the output file, and the
remap file between the two.
State output has the following naming convention:
${Index}-${MigrationPassName}-${What}.${extension}, such as:
1-FixitMigrationState-Input.swift
rdar://problem/30926261
Add an option to the lexer to go back and get a list of "full"
tokens, which include their leading and trailing trivia, which
we can index into from SourceLocs in the current AST.
This starts the Syntax sublibrary, which will support structured
editing APIs. Some skeleton support and basic implementations are
in place for types and generics in the grammar. Yes, it's slightly
redundant with what we have right now. lib/AST conflates syntax
and semantics in the same place(s); this is a first step in changing
that to separate the two concepts for clarity and also to get closer
to incremental parsing and type-checking. The goal is to eventually
extract all of the syntactic information from lib/AST and change that
to be more of a semantic/symbolic model.
Stub out a Semantics manager. This ought to eventually be used as a hub
for encapsulating lazily computed semantic information for syntax nodes.
For the time being, it can serve as a temporary place for mapping from
Syntax nodes to semantically full lib/AST nodes.
This is still in a molten state - don't get too close, wear appropriate
proximity suits, etc.
ede6bf7a80 increments the buffer pointer to early
when searching for the <# prefix to stop lexing an operator,
so the source buffer is accessed 1 byte off the end.
Thanks ASan and thanks @gparker42!
rdar://problem/28457876
'.<#placeholder#>' is actually an unresolved reference where the name is
an editor placeholder, not the operator '.<' followed by #placeholder#>.
rdar://problem/28457876
RFC 2279 states that, in UTF-8:
"The octet values FE and FF never appear."
RFC 3629 states that, in UTF-8:
"The octet values C0, C1, F5 to FF never appear."
Generalize the check to advance past invalid starting bytes for
a UTF-8 sequence to fix a crash in the lexer.
Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.
Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.
Trivia are now data attached to the ends of tokens, not tokens
themselves.
Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.
Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:
fuzzer => token stream => file1
=> Lexer => token stream => file 2 => diff(file1, file2)
Will arrive in a subsequent commit.
This patch does not change the grammar.
Like cursor-info, range info (""source.request.cursorinfo"") answers some
questions clients have for a code snippet under selection, for instance, the type of a selected
expression. This commit implements this new quest kind and provides two
simple information about the selected code: (1) the kind of the
snippet, currently limited to single-statement and expression; and (2)
the type of the selected expression. Gradually, we will enrich the
response to provide more insight into the selected code snippet.
When in Swift 3 Compatibility Mode we now acceptable a standalone
'$' as an identifier. In all other cases this is now disallowed
and must be surrounded by backticks.
Previously, builtin operators followed by comment block were tokenized as
normal operators (e.g. tok::oper_binary_spaced) instead of dedicated
token(e.g. tok::equal).
That used to cause strange parse errors:
test.swift:1:3: error: use of unresolved operator '='
_ =/* */2
^
silently, have the lexer return it as an unknown token. Enhance the expr parser to
detect these things and squash any expression in progress into an ErrorExpr. This
allows us to silence really bad downstream errors. For example, on:
struct S { func f() {} }
func f() {
_ = S.
}
we formerly produced:
x.swift:5:8: error: expected member name following '.'
x.swift:5:7: error: expected member name or constructor call after type name
x.swift:5:7: note: add arguments after the type to construct a value of the type
x.swift:5:7: note: use '.self' to reference the type object
we now emit just the first one. This fixes:
<rdar://problem/22290244> QoI: "UIColor." gives two issues, should only give one
Lexing for conflict markers is inspired by the way clang handles them with a few
updates of our own. Clang's lexer currently searches for the start of the
conflict marker, attempts to find the divider points, lexes that, then finds the
end. We, unfortunately, cannot be so forgiving because of operator overloads.
Instead, we search for the start and end markers and ignore all text in between.
Even if what is found is not conflict markers, it certainly is not valid Swift either.
* Implement the majority of parsing support for SE-0039.
* Parse old object literals names using new syntax and provide FixIt.
For example, parse "#Image(imageLiteral:...)" and provide a FixIt to
change it to "#imageLiteral(resourceName:...)". Now we see something like:
test.swift:4:9: error: '#Image' has been renamed to '#imageLiteral
var y = #Image(imageLiteral: "image.jpg")
^~~~~~ ~~~~~~~~~~~~
#imageLiteral resourceName
Handling the old syntax, and providing a FixIt for that, will be handled in a separate
commit.
Needs tests. Will be provided in later commit once full parsing support is done.
* Add back pieces of syntax map for object literals.
* Add parsing support for old object literal syntax.
... and provide fixits to new syntax.
Full tests to come in later commit.
* Improve parsing of invalid object literals with old syntax.
* Do not include bracket in code completion results.
* Remove defunct code in SyntaxModel.
* Add tests for migration fixits.
* Add literals to code completion overload tests.
@akyrtzi told me this should be fine.
* Clean up response tests not to include full paths.
* Further adjust offsets.
* Mark initializer for _ColorLiteralConvertible in UIKit as @nonobjc.
* Put attribute in the correct place.
Previously, comments were treated as non-whitespace. Operators
also checked for right-boundedness before detecting comments
which start in the middle of an operator.
This resolves both of these issues so that comments are
consistently treated as whitespace when determining whether
operators are left- or right-bound.
Fixes SR-186 and SR-960 (SE-0037)
The if statements check whether a number literal consists
of just the 0b/0o/0x prefix which would be invalid. However, in all three cases there's a preceding check
that already addresses this issue.
This will be used to help IRGen record protocol requirements
with resilient default implementations in protocol metadata.
To enable testing before all the Sema support is in place, this
patch adds SIL parser, printer and verifier support for default
witness tables.
For now, SILGen emits empty default witness tables for protocol
declarations in resilient modules, and IRGen ignores them when
emitting protocol metadata.
