Essentially all ParsedRawSyntaxNode types have a range associated with
them, so let's just store it in a dedicated field with a common getter.
Additionally, this cleans up the recorded storage to just contain an
OpaqueSyntaxNode. When deferred nodes are being handled by
SyntaxParseActions, we can use this OpaqueNode storage to store either
recorded or deferred node data, which is left to be interpreted by the
SyntaxParseAction.
This is a multi-commit effort to push the responsibility of deferred
node handling to the SyntaxParseActions which have more detailed
knowledge of their requirements on deferred nodes and might perform
additional optimisations. For example, the SyntaxTreeCreator can always
create RawSyntax nodes (even for deferred nodes) and decide to simply
not use them, should the deferred nodes not get recorded.
This is again a transitional state before SyntaxParsingContext hands
the responsibility over to SyntaxTreeCreator and from there to
SyntaxParseActions.
Using Parsed*SyntaxBuilder interface and SyntaxParserResult was
unnecessarily complicated. Use SyntaxParsingContext based node creation.
No behavior change.
Previously it was backtracking for the duration of the whole property body which was preventing re-use of previously parsed nodes for incremental re-parsing.
This both addresses a crash during incremental re-parse (rdar://57679731) and generally avoids violating invariants for interacting with the parser library.
Like the last commit, SourceFile is used a lot by Parse and Sema, but
less so by the ClangImporter and (de)Serialization. Split it out to
cut down on recompilation times when something changes.
This commit does /not/ split the implementation of SourceFile out of
Module.cpp, which is where most of it lives. That might also be a
reasonable change, but the reason I was reluctant to is because a
number of SourceFile members correspond to the entry points in
ModuleDecl. Someone else can pick this up later if they decide it's a
good idea.
No functionality change.
Most of AST, Parse, and Sema deal with FileUnits regularly, but SIL
and IRGen certainly don't. Split FileUnit out into its own header to
cut down on recompilation times when something changes.
No functionality change.
The type deduction here will copy the returned ParsedTypeSyntax which
is no longer copy constructible. Use a reference to hold the result
instead which should fix the build on Windows. The returned value of
the `popToken` returns a `ParsedTokenSyntax` which is no longer copy
constructible. Explicitly move the value.
So that we can easily detect 'ParsedSyntaxNode' leaking. When it's
moved, the original node become "null" node. In the destructor of
'ParsedSyntaxNode', assert the node is not "recorded" node.
There were a few places discarding recorded syntax:
- '#<code-complete>' at top-level (this should be parsed as UnknownDecl).
- 'typealias' decl with inheritance clause in protocol decl.
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
SyntaxParseActions::recordToken() et al. may return the same pointer
value for different nodes (e.g. `nullptr`). So we cannot use DenseMap to
associate the node from the explicit syntax parsing actions to libSyntax
node. Instead, use a structure that wraps them.
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
This eliminates the overhead of ParsedRawSyntaxNode needing to do memory management.
If ParsedRawSyntaxNode needs to point to some data the memory is allocated from a bump allocator.
There are also some improvements on how the ParsedSyntaxBuilders work.
Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code.
This decouples the act of parsing from the act of forming a syntax tree representation.
'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree.
To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two.
This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.
