For backtracking scopes that are never cancelled, we can completely disable the SyntaxParsingContext, avoiding the creation of deferred nodes which will never get recorded.
We have finally reached our goal of optimising deferred node creation
for SyntaxTreeCreator. Instead of creating dedicated deferred nodes and
copying the data into a RawSyntax node when recording, we always create
RawSyntax nodes. Recording a deferred node is thus a no-op, since we
have already created a RawSyntax node. Should a deferred node not be
recorded, it stays alive in the SyntaxArena without any reference to it.
While this means, we are leaking some memory for such nodes, most nodes
do get recorded, so the overhead should be fine compared to the
performance benefit.
By now ParsedRawSyntaxNode does not have any knowledge about deferred
node data anymore, which frees up SyntaxParseActions (and, in
particular its sublass SyntaxTreeCreator) to perform optimisations to
more efficiently create and record deferred nodes.
Referencing a string in arbitrary memory is not safe since the source
buffer to which it points may have been freed. Instead copy all strings
into the SyntaxArena. Since RawSyntax nodes retain their arena, they can
be sure that the string won't disappear if it lives in their arena.
To avoid lots of small copies, we copy the entire source buffer once
into the syntax arena and make StringRefs point into that buffer.
Currently when parsing a SourceFile, the parser
gets handed pointers so that it can write the
interface hash and collected tokens directly into
the file. It can also call `setSyntaxRoot` at
the end of parsing to set the syntax tree.
In preparation for the removal of
`performParseOnly`, this commit formalizes these
values as outputs of `ParseSourceFileRequest`,
ensuring that the file gets parsed when the
interface hash, collected tokens, or syntax tree
is queried.
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
This eliminates the overhead of ParsedRawSyntaxNode needing to do memory management.
If ParsedRawSyntaxNode needs to point to some data the memory is allocated from a bump allocator.
There are also some improvements on how the ParsedSyntaxBuilders work.
Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code.
This decouples the act of parsing from the act of forming a syntax tree representation.
'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree.
To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two.
This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.
