This meant we weren't producing sema diagnostics for the case, and it didn’t
get full syntactic/semantic highlighting or indentation.
enum CasesWithMissingElement {
case a(Int, String),
case b(Int, String),
}
Resolves rdar://problem/61476844
The SyntaxModel walker would end up visiting the attributes attached to a
PatternBindingDecl twice if it contained more than one VarDecl, hitting the
below assertion on the second visit because the tokens corresponding to the
attribute had already been consumed the first time around:
```
Assertion failed: (0 && "Attribute's TokenNodes already consumed?"), function
handleSpecialDeclAttribute
```
It would also hit the same assertion for attributes on an EnumCaseDecl, but even
when it only had a single child EnumElementDecl. This because when we visited
the EnumCaseDecl and pushed its structure node, we'd consume and emit any tokens
before it's start position. This meant that when we tried to process the
attributes attached to the child EnumElementDecl its tokens had already been
consumed, triggering the assertion.
In both cases the attributes syntactically attach to the parent
PatternBindingDecl or EnumCaseDecl, but in the AST they're accessed via their
child VarDecls or EnumElementDecls.
Resolves rdar://problem/53747546
This was causing the tokens comprising image literals to be output separately,
rather than as a single object literal.
Resolves rdar://problem/55045797
When looking for the SyntaxNode corresponding to a type attribute (like
@escaping), ModelASTWalker would look for one whose range *started* at the type
attribute's source location. It never found one, though, because the
SyntaxNode's range included the @, while the type attribute's source location
pointed to the name *after* the @.
ModelASTWalker was previously constructing SyntaxNodes for EnumElementDecls
manually when visiting their associated EnumCaseDecl so that they would appear
as children rather than siblings. It wasn't actually walking these nodes
though, so missed handling some things, e.g. closures passed as default
argument values. These were also still being visited later, and because the
first visit consumed all the associated TokenNodes, this was triggering an
assertion due to the associated TokenNodes not matching expectations.
This fixes custom attribute syntax highlighting on parameters and functions
(where function builders can be applied). They weren't being walked in
the function position previously and were walked out of source order in the
parameter position.
It also fixes rename of the property wrapper and function builder type
names that can appear in custom attributes, as well as rename of property
wrapper constructors, that can appear after the type names, e.g.
`@Wrapper(initialValue: 10)`. The index now also records these constructor
occurrences, along with implicit occurrences whenever a constructor is
called via default value assignment, e.g. `@Wrapper var foo = 10`, so that
finding calls/references to the constructor includes these locations.
Resolves rdar://problem/49036613
Resolves rdar://problem/50073641
We should use parser to figure out the end position of object literal expression instead
of scanning through token stream, which crashes sourcekitd when the syntax is invalid.
Fixing: rdar://48390913
`@unknown` is so far the only attribute for statement. Handle it
specially in syntax-map.
rdar://problem/47855035 / https://bugs.swift.org/browse/SR-9873
Explict cast expressions (i.e. 'as', 'as!`, 'as?', and 'is') appear twice in
'SequenceExpr'. For instance, 'a as B' is parsed as:
(sequence_expr
(unresolved_declref_expr name='a')
(coerce_expr writtenType='B')
(coerce_expr writtenType='B'))
This patch prevents ModelASTWalker from walking into them twice.
rdar://problem/43135727
Previously, it used to use length from AST. Because AST doesn't necessarily
holds actual source information, it may emits inacculate syntax info
which cause mis-coloring or compiler crash in the worst case. Instead, use
collected token info which contains actual token length.
https://bugs.swift.org/browse/SR-8378
rdar://problem/42653982
* [Parser] Keep source location info for the ownership keywords inside a closure capture list
This allows to do syntax coloring for 'weak'/'unowned' inside a capture list
rdar://42655051
The recommended way forward is to use the SyntaxClassifier on the Swift
side.
By removing the C++ SyntaxClassifier, we can also eliminate the
-force-libsyntax-based-processing option that was used to bootstrap
incremental parsing and would generate the syntax map from a syntax
tree.
Multiline strings (and multiline tokens in general) were not well supported by the existing highlighting logic. Edits
on one line can make tokens appear/disappear on previous and later lines, which broke assumptions in the existing
logic, and left odd ranges of source unhighlighted or out of date. This patch accounts for these changes, and also
changes unterminated multiline (and regular strings) to still be highlighted as strings, so the rest of the
file doesn't look like plain text.
Resolves rdar://problem/32148117.
This fixes a syntax coloring issue when only the innermost " character were highlighted as part of the string, e.g:
""<str>"
This is a string
"</str>""
Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.
Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.
Trivia are now data attached to the ends of tokens, not tokens
themselves.
Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.
Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:
fuzzer => token stream => file1
=> Lexer => token stream => file 2 => diff(file1, file2)
Will arrive in a subsequent commit.
This patch does not change the grammar.
Most keywords can be used as argument labels. When they are they shouldn't be
colored as a keyword. This patch fixes three issues with this:
- keywords as local argument labels (where an external label was also present)
were still colored as keywords
- 'let', 'var', and 'inout' cannot be used as argument labels, but were colored
as identifiers rather than keywords when used like one
- the check to see if a keyword token is being used as an argument label wasn't
quite restrictive enough, e.g. treating the 'let's in 'case (let x, let y):' as
identifiers.
This gave a roughly 40-45% improvement in sourcekitd's incremental
syntax-only parse time in files with a lot of doc comments (test case
was ~6000 lines, with ~780 lines being doc comments). This is on the
critical path for every edit.
While there were a few smaller improvements we could have made to the
original code, ultimately std::regex is slow, and it was better to just
use a custom parser for these simple patterns.
rdar://problem/28809397
Keywords like 'let' can serve as argument labels. When they do so, we should
highlight them as identifiers instead of keywords. However, the check for this
situation seems overly lenient so that when 'let', 'var' appear in conditions
of IfStmt or GuardStmt, they are wrongly highlighted as identifiers too. This
commit strengthens the checking to preserve keywords' identity in these statements.
rdar://28297337
Argument labels are allowed to use keywords, in which case we want to
treat them as identifiers in the syntax map (except for '_'). This
commit moves calculation of that into the original lexing instead of
in the model walker, which makes it much more robust, since the model
walker was only guessing about what was next on the the TokenNodes list.
This fixes a bug where arrays of object literals would only have the
first object correct (the following ones were identifiers), as well as
some incorrect cases where we treated keywords as identifiers.
rdar://problem/27726422
