Introduce support for parsing declaration attributes that occur within
example:
#if hasAttribute(frozen)
@frozen
#endif
public struct X { ... }
will apply to "frozen" attribute to the struct `X`, but only when the
compiler supports the "frozen" attribute.
Correctly determining whether a particular `#if` block contains
attributes to be associated with the following declaration vs.
starting a new declaration requires arbitrary lookahead. The parser
will ensure that at least one of the branches of the `#if` contains an
attribute, and that none of the branches contains something that does
not fit the attribute grammar, before committing to parsing the `#if`
clause as part of the declaration attributes. This lookahead does
occur at the top level (e.g., in the parsing of top-level declarations
and code), but should only need to scan past the first `#if` line to
the following token in the common case.
Unlike other `#if` when used to wrap statements or declarations, we
make no attempt to record the `#if` not taken anywhere in the AST.
This reflects a change in attitude in the design of the AST, because
we have found that trying to represent this information there (e.g.,
via `IfConfigDecl`) complicates clients while providing little value.
This information is best kept in the syntax tree, only.
When recovering from a parser error in an expression, we resumed parsing at a '{'. I assume this was because we wanted to continue inside e.g. an if-body if parsing the condition failed, but it's actually causing more issue because when parsing e.g.
```swift
expr + has - error +
functionTakesClosure {
}
```
we continue parsing at the `{` of the trailing closure, which is a completely garbage location to continue parsing.
The motivating example for this change was (in a result builder)
```swift
Text("\(island.#^COMPLETE^#)")
takeTrailingClosure {}
```
Here `Text(…)` has an error (because it contains a code completion token) and thus we skip `takeTrailingClosure`, effectively parsing
```swift
Text(….) {}
```
which the type checker wasn’t very happy with and thus refused to provide code completion. With this change, we completely drop `takeTrailingClosure {}`. The type checker is a lot happier with that.
Previously, when we reached the maximum nesting level, we changed the current token’s kind to an EOF token. A lot of places in the parser are not set up to expect this token change. The intended workaround was to check whether pushing a structure marker failed (which would change the token kind) and bail out parsing if this happened. This was fragile and caused assertion failures in assert builds.
Instead of changing the current token’s kind, and failing to push the structure marker, let the lexer know that it should cut off lexing, essentially making the input buffer stop at the current position. The parser will continue to consume its current token (`Parser.Tok`) and the next token that’s already lexed in the lexer (`Lexer.NextToken`) before reaching the emulated EOF token. Thus two more tokens are parsed than before, but that shouldn’t make much of a difference.
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag
This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.
* Updated tests
Fixe a couple of bugs in libSyntax parsing found by enabling `-verify-syntax-tree` for `%target-build-swift`:
- Fix parsing of the `actor` contextual keyword in actor decls
- Don't build a libSyntax tree when parsing the availability macro
- The availability macro is not part of the source code and doesn't form a valid Swift file, thus creation of a libSyntax tree is completely pointless and will fail
- Add support for parsing `@_originallyDefinedIn` attributes.
- Add support for parsing `#sourceLocation` in member decl lists
- Add support for effectful properties (throwing/async getters/setters)
- Add support for optional types as the base of a key path (e.g. `\TestOptional2?.something`)
- Allow platform restrictions without a version (e.g. `_iOS13Aligned`)
In particular, we were unconditionally dropping argument labels on accessors; now we only do that for property accessors, not subscript accessors.
Doing this unconditionally causes ClangImporter failures when a method parameter is called “subscript” (really!), so this behavior is enabled only by the caller’s request.
For backtracking scopes that are never cancelled, we can completely disable the SyntaxParsingContext, avoiding the creation of deferred nodes which will never get recorded.
Small peformance improvement: Token is larger than a pointer and not
modified in the performance-criticial TokenReceivers, so we can pass
it by reference.
Instead, only reference count the SyntaxArena that the RawSyntax nodes
live in. The user of RawSyntax nodes must guarantee that the SyntaxArena
stays alive as long as the RawSyntax nodes are being accessed.
During parse time, the SyntaxTreeCreator holds on to the SyntaxArena
in which it creates RawSyntax nodes. When inspecting a syntax tree,
the root SyntaxData node keeps the SyntaxArena alive. The change should
be mostly invisible to the users of the public libSyntax API.
This change significantly decreases the overall reference-counting
overhead. Since we were not able to free individual RawSyntax nodes
anyway, performing the reference-counting on the level of the
SyntaxArena feels natural.
Do the same thing that we are already doing for trivia: Since RawSyntax
nodes always live inside a SyntaxArena, we don't need to tail-allocate
an OwnedString to store the token's text. Instead we can just copy it
to the SyntaxArena. If we copy the entire source buffer to the syntax
arena at the start of parsing, this means that no more copies are
required later on. Plus we also avoid ref-counting the OwnedString which
should also increase performance.
This is again a transitional state before SyntaxParsingContext hands
the responsibility over to SyntaxTreeCreator and from there to
SyntaxParseActions.
This is an intermediate state in which the lexer delegates the
responsibility for trivia lexing to the parser. Later, the parser will
delegate this responsibility to SyntaxParsingContext which will hand it
over to SyntaxParseAction, which will only lex the pieces if it is
really necessary to do so.
Code completion used to avoid forming single expression closures/function
bodies when the single expression contained the code completion expression
because a contextual type mismatch could result in types not being applied
to the AST, giving no completions.
Completions that have been migrated to the new solver-based completion
mechanism don't need this behavior, however. Rather than trying to guess
whether the type of completion we're going to end up performing is one of
the ones that haven't been migrated to the solver yet when parsing, instead
just always form single-expression closures/function bodies (like we do for
regular compilation) and undo the transformation if and when we know we're
going to perform a completion kind we haven't migrated yet.
Once all completion kinds are migrated, the undo-ing code can be removed.
For example, the completion below would trigger error recovery within the
closure, which we recover from by skipping to the first inner closure's right
brace. The fact that we recovered though, was not recorded. The closure is
treated as still being an error, triggering another recovery after it that
skips over the 'Thing' token, giving a lone closure expression, rather than a
call.
CreateThings {
Thing { point in
print("hello")
point.#^HERE^#
}
Thing { _ in }
}
This isn't an issue for code completion when the outer closure is a regular
closure, but when it's a function builder, invalid elements result in no types
being applied (no valid solutions) and we end up with no completion results.
The fix here is removing the error status from the parser result after the
initial parser recovery.
We need ClangImporterOptions to be persistent for several scenarios: (1)
when creating a sub-ASTContext to build Swift modules from interfaces; and
(2) when creating a new Clang instance to invoke Clang dependencies scanner.
This change is NFC.
Currently when parsing a SourceFile, the parser
gets handed pointers so that it can write the
interface hash and collected tokens directly into
the file. It can also call `setSyntaxRoot` at
the end of parsing to set the syntax tree.
In preparation for the removal of
`performParseOnly`, this commit formalizes these
values as outputs of `ParseSourceFileRequest`,
ensuring that the file gets parsed when the
interface hash, collected tokens, or syntax tree
is queried.
Sink the `BuildSyntaxTree` and
`CollectParsedTokens` bits into
`SourceFile::ParsingFlags`, with a static method
to get the parsing options from the lang opts.
Also add a parsing flag for enabling the interface
hash, which can be used instead of calling
`enableInterfaceHash`.
Previously it was backtracking for the duration of the whole property body which was preventing re-use of previously parsed nodes for incremental re-parsing.
Add ModuleImplicitImportsRequest, which computes
the modules that should be implicitly imported by
each file of a given module. Use this request in
import resolution to add all the necessary
implicit imports.
The request computes the implicit imports by
consulting the ImplicitImportInfo, which ModuleDecl
can now be created with. This allows us to remove
uses of `SourceFile::addImports` in favor of
adding modules needed to be implicitly imported to
the ImplicitImportInfo.
Code Completion operates on a CompilerInstance that passes a primary
file down for type checking. This means it creates and registers
dependencies in the referenced name trackers. Despite the fact that
those dependencty edges are unused,
because the would-be swiftdeps file is never written to disk,
it is still a dependency source that should participate in the
request-based dependency tracking refactor.
We were always dropping the error status when returning from parseExprImpl. We
were also incorrectly keeping error status after recovering by finding the
right close token in parseList. This change fixes both, and also updates a few
callers of parseList that assumed when they reported a failure parsing an
element the list as a whole would get error status, which isn't true due to
recovery.
