Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.
Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.
Trivia are now data attached to the ends of tokens, not tokens
themselves.
Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.
Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:
fuzzer => token stream => file1
=> Lexer => token stream => file 2 => diff(file1, file2)
Will arrive in a subsequent commit.
This patch does not change the grammar.
Most keywords can be used as argument labels. When they are they shouldn't be
colored as a keyword. This patch fixes three issues with this:
- keywords as local argument labels (where an external label was also present)
were still colored as keywords
- 'let', 'var', and 'inout' cannot be used as argument labels, but were colored
as identifiers rather than keywords when used like one
- the check to see if a keyword token is being used as an argument label wasn't
quite restrictive enough, e.g. treating the 'let's in 'case (let x, let y):' as
identifiers.
Like cursor-info, range info (""source.request.cursorinfo"") answers some
questions clients have for a code snippet under selection, for instance, the type of a selected
expression. This commit implements this new quest kind and provides two
simple information about the selected code: (1) the kind of the
snippet, currently limited to single-statement and expression; and (2)
the type of the selected expression. Gradually, we will enrich the
response to provide more insight into the selected code snippet.
The underlying type-check was correct, but I forgot to consider it in
the outer code, and embarassingly never tested this case.
rdar://problem/28435922
This gave a roughly 40-45% improvement in sourcekitd's incremental
syntax-only parse time in files with a lot of doc comments (test case
was ~6000 lines, with ~780 lines being doc comments). This is on the
critical path for every edit.
While there were a few smaller improvements we could have made to the
original code, ultimately std::regex is slow, and it was better to just
use a custom parser for these simple patterns.
rdar://problem/28809397
With the previous type-checker change, we end up with some vacuous
infix operator completions, where the right-hand side and result type
are *both* type variables. Suppress these; they aren't useful to the
developer.
This was exposed by building with a recent version of clang. Without this
change, the following tests were failing:
Swift(macosx-x86_64) :: IDE/complete_assignment.swift
Swift(macosx-x86_64) :: IDE/complete_enum_elements.swift
Swift(macosx-x86_64) :: IDE/complete_stmt_controlling_expr.swift
Swift(macosx-x86_64) :: SourceKit/CodeComplete/complete_structure.swift
I did not narrow it down to which uses of DeclFilter were problematic.
The global variables certainly do not seem like a good place to use a
function_ref. rdar://problem/28699882
[SourceKit] Indentation: when the indented line starts with open brace and the
line before starts with a leading declaration keywords, we never add
indentation level on the brace. rdar://28049927
[SyntaxModel] When searching URLs in doc comments, reduce the number of protocol name comparisons by looking ahead more characters, NFC. rdar://28298506
Searching URL in doc comments can be expensive. We used to look for
every colon as an indicator of potential URLs. However, this is not
efficient enough. Suggested by Ben, we further divide protocols into
categories so that most protocols can use "://" as an indicator of its
existence.
Not sure whether this is enough to close the radar, but I believe it is
a valuable performance improvement anyway.
We strip the first input type on instance methods like,
struct S { func foo(T) -> U } // (S)->(T)->U
but we should not do that when it's actually a curried instance method,
such as S.foo.
In most places where we were checking "is<ErrorType>()", we now mean
"any error occurred". The few exceptions are in associated type
inference, code completion, and expression diagnostics, where we might
still work with partial errors.
Type::subst()'s "IgnoreMissing" option was fairly unprincipled, dropping
unsubstituted types into the resulting AST without any indication
whatsoever that anything went wrong. Replace this notion with a new
form of ErrorType that explicitly tracks which substituted type caused
the problem. It's still an ErrorType, but it prints like the
substituted type (which is important for code completion) and allows
us to step back to the substituted type if needed (which is used by
associated type inference). Then, allow Type::subst(), when the new
UseErrorTypes flag is passed, to form partially-substituted types that
contain errors, which both code completion and associated type
inference relied on.
Over time, I hope we can use error-types-with-original-types more
often to eliminate "<<error type>>" from diagnostics and teach
Type::subst() never to return a "null" type. Clients can check
"hasError()" to deal with failure cases rather than checking null.
If we're completing in a context that expects a function type, try to
match methods/functions as function references before trying them as
calls. This means that
func take(_: (Int)->()) {}
func foo(a: Int) {}
take(#^A^#) // completes foo(a:) instead of foo(a: {#value#})
Note: doesn't yet work with generic types.
rdar://problem/28435922
