Instead of mangling class template specializations with the prefix "__CxxTemplateInst," simply set the decl name as the class templates plus the types that it is specialized on (so `vector<Int>` rather than `__CxxTemplateInstNSt3__16vectorIi...`).
This is mainly to improve diagnostics. As a side effect of this change, if anyone copies the name of a class template specializaiton from an error/warning and uses it in source code, the compiler will error (that class templates aren't available in swift) rather than silently passing only to cause serailization failures down the road.
Use the name mangling scheme we've devised for macro expansions to
back the implementation of the macro expansion context's
`getUniqueName` operation. This way, we guarantee that the names
provided by macro expansions don't conflict, as well as making them
demangleable so we can determine what introduced the names.
The "local context" was only used to prevent parsing of closures in a
non-local context, and also string interpolations because they are
similar-ish to closures. However, this isn't something a parser should
decide, so remove this special-case semantic check from the parser and
eliminate the notion of "local context" entirely.
Local discriminators for named entities are currently being set by the
parser, so entities not created by the parser (e.g., that come from
synthesized code) don't get local discriminators. Moreover, there is
no checking to ensure that every named local entity gets a local
discriminator, so some entities would incorrectly get a local
discriminator of 0.
Assign local discriminators as part of setting closure discriminators,
in response to a request asking for the local discriminator, so the
parser does not need to track this information, and all local
declarations---including synthesized ones---get local discriminators.
And add checking to make sure that every entity that needs a local
discriminator gets assigned one.
There are a few interesting cases in here:
* There was a potential mangling collision with local property
wrappers because their generated variables weren't getting local
discriminators
* $interpolation variables introduced for string interpolation weren't
getting local discriminators, they were just wrong.
* "Local rename" when dealing with captures like `[x]` was dependent on
the new delcaration of `x` *not* getting a local discriminator. There
are funny cases involving nesting where it would do the wrong thing.
Rather than set closure discriminators in both the parser (for explicit
closures) and then later as part of contextualizing closures (for
autoclosures), do so via a request that sets all of the discriminators
for a given context.
Otherwise, the closure discriminator will be incremented by one when the closure witht he code completion token is parsed a second time during the second pass, and thus it would receive a different discriminator during the second pass.
This will be necessary to make cursor info completion like to inspect the just parsed source file because the callback from parsing the code completion token won’t be called.
In the Swift grammar, the top-level of a source file is a mix of three
different kinds of "items": declarations, statements, and expressions.
However, the existing parser forces all of these into declarations at
parse time, wrapping statements and expressions in TopLevelCodeDecls,
so the primary API for getting the top-level entities in source files
is based on getting declarations.
Start generalizing the representation by storing ASTNode instances at
the top level, rather than declaration pointers, updating many (but
not all!) uses of this API. The walk over declarations is a (cached)
filter to pick out all of the declarations. Existing parsed files are
unaffected (the parser still creates top-level code declarations), but
the new "macro expansion" source file kind skips creating top-level
code declarations so we get the pure parse tree. Additionally, some
generalized clients (like ASTScope lookup) will now look at the list
of items, so they'll be able to walk into statements and expressions
without the intervening TopLevelCodeDecl.
Over time, I'd like to phase out `getTopLevelDecls()` entirely,
relying on the new `getTopLevelItems()` for parsed content. We can
introduce TopLevelCodeDecls more lazily for semantic walks.
Introduce a new source file kind to describe source files for macro
expansions, and include the macro expression that they expand. This
establishes a "parent" relationship
Also track every kind of auxiliary source file---whether for macro
expansions or other reasons---that is introduced into a module, adding
an operation that allows us to find the source file that contains a
given source location.
Refactors `parseSingleAttrOption()` to create a helper that can parse a single arbitrary `Identifier`. This simplifies the handling of `SwiftNativeObjCRuntimeBaseAttr`, `ObjCRuntimeNameAttr`, and `ProjectedValuePropertyAttr`.
Introduce support for parsing declaration attributes that occur within
example:
#if hasAttribute(frozen)
@frozen
#endif
public struct X { ... }
will apply to "frozen" attribute to the struct `X`, but only when the
compiler supports the "frozen" attribute.
Correctly determining whether a particular `#if` block contains
attributes to be associated with the following declaration vs.
starting a new declaration requires arbitrary lookahead. The parser
will ensure that at least one of the branches of the `#if` contains an
attribute, and that none of the branches contains something that does
not fit the attribute grammar, before committing to parsing the `#if`
clause as part of the declaration attributes. This lookahead does
occur at the top level (e.g., in the parsing of top-level declarations
and code), but should only need to scan past the first `#if` line to
the following token in the common case.
Unlike other `#if` when used to wrap statements or declarations, we
make no attempt to record the `#if` not taken anywhere in the AST.
This reflects a change in attitude in the design of the AST, because
we have found that trying to represent this information there (e.g.,
via `IfConfigDecl`) complicates clients while providing little value.
This information is best kept in the syntax tree, only.
When recovering from a parser error in an expression, we resumed parsing at a '{'. I assume this was because we wanted to continue inside e.g. an if-body if parsing the condition failed, but it's actually causing more issue because when parsing e.g.
```swift
expr + has - error +
functionTakesClosure {
}
```
we continue parsing at the `{` of the trailing closure, which is a completely garbage location to continue parsing.
The motivating example for this change was (in a result builder)
```swift
Text("\(island.#^COMPLETE^#)")
takeTrailingClosure {}
```
Here `Text(…)` has an error (because it contains a code completion token) and thus we skip `takeTrailingClosure`, effectively parsing
```swift
Text(….) {}
```
which the type checker wasn’t very happy with and thus refused to provide code completion. With this change, we completely drop `takeTrailingClosure {}`. The type checker is a lot happier with that.
Previously, when we reached the maximum nesting level, we changed the current token’s kind to an EOF token. A lot of places in the parser are not set up to expect this token change. The intended workaround was to check whether pushing a structure marker failed (which would change the token kind) and bail out parsing if this happened. This was fragile and caused assertion failures in assert builds.
Instead of changing the current token’s kind, and failing to push the structure marker, let the lexer know that it should cut off lexing, essentially making the input buffer stop at the current position. The parser will continue to consume its current token (`Parser.Tok`) and the next token that’s already lexed in the lexer (`Lexer.NextToken`) before reaching the emulated EOF token. Thus two more tokens are parsed than before, but that shouldn’t make much of a difference.
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag
This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.
* Updated tests
Fixe a couple of bugs in libSyntax parsing found by enabling `-verify-syntax-tree` for `%target-build-swift`:
- Fix parsing of the `actor` contextual keyword in actor decls
- Don't build a libSyntax tree when parsing the availability macro
- The availability macro is not part of the source code and doesn't form a valid Swift file, thus creation of a libSyntax tree is completely pointless and will fail
- Add support for parsing `@_originallyDefinedIn` attributes.
- Add support for parsing `#sourceLocation` in member decl lists
- Add support for effectful properties (throwing/async getters/setters)
- Add support for optional types as the base of a key path (e.g. `\TestOptional2?.something`)
- Allow platform restrictions without a version (e.g. `_iOS13Aligned`)
In particular, we were unconditionally dropping argument labels on accessors; now we only do that for property accessors, not subscript accessors.
Doing this unconditionally causes ClangImporter failures when a method parameter is called “subscript” (really!), so this behavior is enabled only by the caller’s request.
For backtracking scopes that are never cancelled, we can completely disable the SyntaxParsingContext, avoiding the creation of deferred nodes which will never get recorded.
Small peformance improvement: Token is larger than a pointer and not
modified in the performance-criticial TokenReceivers, so we can pass
it by reference.
Instead, only reference count the SyntaxArena that the RawSyntax nodes
live in. The user of RawSyntax nodes must guarantee that the SyntaxArena
stays alive as long as the RawSyntax nodes are being accessed.
During parse time, the SyntaxTreeCreator holds on to the SyntaxArena
in which it creates RawSyntax nodes. When inspecting a syntax tree,
the root SyntaxData node keeps the SyntaxArena alive. The change should
be mostly invisible to the users of the public libSyntax API.
This change significantly decreases the overall reference-counting
overhead. Since we were not able to free individual RawSyntax nodes
anyway, performing the reference-counting on the level of the
SyntaxArena feels natural.
Do the same thing that we are already doing for trivia: Since RawSyntax
nodes always live inside a SyntaxArena, we don't need to tail-allocate
an OwnedString to store the token's text. Instead we can just copy it
to the SyntaxArena. If we copy the entire source buffer to the syntax
arena at the start of parsing, this means that no more copies are
required later on. Plus we also avoid ref-counting the OwnedString which
should also increase performance.
This is again a transitional state before SyntaxParsingContext hands
the responsibility over to SyntaxTreeCreator and from there to
SyntaxParseActions.
This is an intermediate state in which the lexer delegates the
responsibility for trivia lexing to the parser. Later, the parser will
delegate this responsibility to SyntaxParsingContext which will hand it
over to SyntaxParseAction, which will only lex the pieces if it is
really necessary to do so.
Code completion used to avoid forming single expression closures/function
bodies when the single expression contained the code completion expression
because a contextual type mismatch could result in types not being applied
to the AST, giving no completions.
Completions that have been migrated to the new solver-based completion
mechanism don't need this behavior, however. Rather than trying to guess
whether the type of completion we're going to end up performing is one of
the ones that haven't been migrated to the solver yet when parsing, instead
just always form single-expression closures/function bodies (like we do for
regular compilation) and undo the transformation if and when we know we're
going to perform a completion kind we haven't migrated yet.
Once all completion kinds are migrated, the undo-ing code can be removed.
For example, the completion below would trigger error recovery within the
closure, which we recover from by skipping to the first inner closure's right
brace. The fact that we recovered though, was not recorded. The closure is
treated as still being an error, triggering another recovery after it that
skips over the 'Thing' token, giving a lone closure expression, rather than a
call.
CreateThings {
Thing { point in
print("hello")
point.#^HERE^#
}
Thing { _ in }
}
This isn't an issue for code completion when the outer closure is a regular
closure, but when it's a function builder, invalid elements result in no types
being applied (no valid solutions) and we end up with no completion results.
The fix here is removing the error status from the parser result after the
initial parser recovery.
