Store leading a trailing "trivia" around a token, such as whitespace,
comments, doc comments, and escaping backticks. These are syntactically
important for preserving formatting when printing ASTs but don't
semantically affect the program.
Tokens take all trailing trivia up to, but not including, the next
newline. This is important to maintain checks that statements without
semicolon separators start on a new line, among other things.
Trivia are now data attached to the ends of tokens, not tokens
themselves.
Create a new Syntax sublibrary for upcoming immutable, persistent,
thread-safe ASTs, which will contain only the syntactic information
about source structure, as well as for generating new source code, and
structural editing. Proactively move swift::Token into there.
Since this patch is getting a bit large, a token fuzzer which checks
for round-trip equivlence with the workflow:
fuzzer => token stream => file1
=> Lexer => token stream => file 2 => diff(file1, file2)
Will arrive in a subsequent commit.
This patch does not change the grammar.
Most keywords can be used as argument labels. When they are they shouldn't be
colored as a keyword. This patch fixes three issues with this:
- keywords as local argument labels (where an external label was also present)
were still colored as keywords
- 'let', 'var', and 'inout' cannot be used as argument labels, but were colored
as identifiers rather than keywords when used like one
- the check to see if a keyword token is being used as an argument label wasn't
quite restrictive enough, e.g. treating the 'let's in 'case (let x, let y):' as
identifiers.
Like cursor-info, range info (""source.request.cursorinfo"") answers some
questions clients have for a code snippet under selection, for instance, the type of a selected
expression. This commit implements this new quest kind and provides two
simple information about the selected code: (1) the kind of the
snippet, currently limited to single-statement and expression; and (2)
the type of the selected expression. Gradually, we will enrich the
response to provide more insight into the selected code snippet.
The underlying type-check was correct, but I forgot to consider it in
the outer code, and embarassingly never tested this case.
rdar://problem/28435922
This gave a roughly 40-45% improvement in sourcekitd's incremental
syntax-only parse time in files with a lot of doc comments (test case
was ~6000 lines, with ~780 lines being doc comments). This is on the
critical path for every edit.
While there were a few smaller improvements we could have made to the
original code, ultimately std::regex is slow, and it was better to just
use a custom parser for these simple patterns.
rdar://problem/28809397
With the previous type-checker change, we end up with some vacuous
infix operator completions, where the right-hand side and result type
are *both* type variables. Suppress these; they aren't useful to the
developer.
This was exposed by building with a recent version of clang. Without this
change, the following tests were failing:
Swift(macosx-x86_64) :: IDE/complete_assignment.swift
Swift(macosx-x86_64) :: IDE/complete_enum_elements.swift
Swift(macosx-x86_64) :: IDE/complete_stmt_controlling_expr.swift
Swift(macosx-x86_64) :: SourceKit/CodeComplete/complete_structure.swift
I did not narrow it down to which uses of DeclFilter were problematic.
The global variables certainly do not seem like a good place to use a
function_ref. rdar://problem/28699882
[SourceKit] Indentation: when the indented line starts with open brace and the
line before starts with a leading declaration keywords, we never add
indentation level on the brace. rdar://28049927
[SyntaxModel] When searching URLs in doc comments, reduce the number of protocol name comparisons by looking ahead more characters, NFC. rdar://28298506
Searching URL in doc comments can be expensive. We used to look for
every colon as an indicator of potential URLs. However, this is not
efficient enough. Suggested by Ben, we further divide protocols into
categories so that most protocols can use "://" as an indicator of its
existence.
Not sure whether this is enough to close the radar, but I believe it is
a valuable performance improvement anyway.
We strip the first input type on instance methods like,
struct S { func foo(T) -> U } // (S)->(T)->U
but we should not do that when it's actually a curried instance method,
such as S.foo.
In most places where we were checking "is<ErrorType>()", we now mean
"any error occurred". The few exceptions are in associated type
inference, code completion, and expression diagnostics, where we might
still work with partial errors.
Type::subst()'s "IgnoreMissing" option was fairly unprincipled, dropping
unsubstituted types into the resulting AST without any indication
whatsoever that anything went wrong. Replace this notion with a new
form of ErrorType that explicitly tracks which substituted type caused
the problem. It's still an ErrorType, but it prints like the
substituted type (which is important for code completion) and allows
us to step back to the substituted type if needed (which is used by
associated type inference). Then, allow Type::subst(), when the new
UseErrorTypes flag is passed, to form partially-substituted types that
contain errors, which both code completion and associated type
inference relied on.
Over time, I hope we can use error-types-with-original-types more
often to eliminate "<<error type>>" from diagnostics and teach
Type::subst() never to return a "null" type. Clients can check
"hasError()" to deal with failure cases rather than checking null.
If we're completing in a context that expects a function type, try to
match methods/functions as function references before trying them as
calls. This means that
func take(_: (Int)->()) {}
func foo(a: Int) {}
take(#^A^#) // completes foo(a:) instead of foo(a: {#value#})
Note: doesn't yet work with generic types.
rdar://problem/28435922
With a bit of work, we can re-purpose the existing
QualifiedArchetype mangling to cover this case.
This allows us to get rid of a usage of
ArchetypeType::getSelfProtocol(), which we want to remove.
There was a ton of complicated logic here to work around
two problems:
- Same-type constraints were not represented properly in
RequirementReprs, requiring us to store them in strong form
and parse them out when printing type interfaces.
- The TypeBase::getAllGenericArgs() method did not do the
right thing for members of protocols and protocol extensions,
and so instead of simple calls to Type::subst(), we had
an elaborate 'ArchetypeTransformer' abstraction repeated
in two places.
Rewrite this code to use GenericSignatures and
GenericFunctionType instead of old-school GenericParamLists
and PolymorphicFunctionType.
This changes the code completion and AST printer output
slightly. A few of the changes are actually fixes for cases
where the old code didn't handle substitutions properly.
A few others are subjective, for example a generic parameter
list of the form <T : Proto> now prints as <T where T : Proto>.
We can add heuristics to make the output whatever we want
here; the important thing is that now we're using modern
abstractions.
This fixes a source of non-determinism. The IDE/complete_constructor
test would sometimes fail depending on the order in which prior tests
ran, since those prior tests might populate the code completion cache.
Keywords like 'let' can serve as argument labels. When they do so, we should
highlight them as identifiers instead of keywords. However, the check for this
situation seems overly lenient so that when 'let', 'var' appear in conditions
of IfStmt or GuardStmt, they are wrongly highlighted as identifiers too. This
commit strengthens the checking to preserve keywords' identity in these statements.
rdar://28297337
There's a bit of a hack to deal with generic typealiases, but
overall this makes things more logical.
This is the last big refactoring before we can allow constrained
extensions to make generic parameters concrete. All that remains
is a small set of changes to SIL type lowering, and retooling
some diagnostics in Sema.
... in other initializers. We don't restrict it to convenience inits
because you might have forgotten the convenience keyword and the
compiler will give you a fixit.
rdar://problem/20913297
* If "required" or "convenience" is specified, emit only initializers
* If "final" or "open" is specified, don't emit initializers or typealias
* If "typealias" is specified, emit only associated type implementation
* Emit "override" or "required" modifier for initializers
* Emit access modifier for initializers
* Emit designated initializers even if "override" is specified
* Don't emit inheritance clause for associated type implentation
