When the source code is invalid, this allows us to represent tokens that could not be used to form a valid syntax tree with more fidelity.
This commit does not start using GarbageNodes yet, it just sets everything up for them.
Initializer declarations were missing support for `async`, in part
because they deplicated most of the `FunctionSignature` production.
Instead, use `FunctionSignature` consistently and let the presence of a
result type be a semantic error.
We were only keeping track of `RawSyntax` node IDs to incrementally transfer a syntax tree via JSON. However, AFAICT the incremental JSON transfer option has been superceeded by `SyntaxParseActions`, which are more efficient.
So, let’s clean up and remove the `RawSyntax` node ID and JSON incremental transfer option.
In places that still need a notion of `RawSyntax` identity (like determining the reused syntax regions), use the `RawSyntax`’s pointer instead of the manually created ID.
In `incr_transfer_round_trip.py` always use the code path that uses the `SyntaxParseActions` and remove the transitional code that was still using the incremental JSON transfer but was never called.
- Utilize ignoreToken() to skip tokens while keeping them in syntax tree
- SyntaxParsedResult now holds ParsedRawSyntaxNode and ParserStatus
- Simplify migration support for 'TypeName[]' type
- Use builder for generic argument clause parsing
Replace the inline sed commands with sed scripts to avoid the subshells
on Windows. Additionally, the unicode handling on Windows causes
problems and using the scripts circumvents that problem.
Implement an inline dos2unix as the BSD sed does not support `-b` and on
Windows, sed will convert the line endings.
Instead of creating multiple CodeBlockItemList nodes, that need to get merged and discarded later on, do this:
* Ensure for libSyntax parsing that we parse the whole file
* Create top-level CodeBlockItem nodes that we just directly wrap with a single CodeBlockItemList node at the end
The importance of this change will become more obvious later on when we'll decouple syntax parsing from the formation of libSyntax tree nodes.
Variable declarations are declarations led by either 'var' or 'let'. It
can contain multiple pattern bindings as children.
For patterns, this patch only creates syntax nodes for simple identifier
patterns, e.g. 'a = 3'. The rest of the pattern kinds are still left
unknown (UnknownPattern).
To construct struct syntax, this patch first specialized type
inheritance clause. For protocol's class requirement, we currently
treat it as an unknown type.
This patch also teaches SyntaxParsingContext to collect syntax nodes
from back in place. This is useful to squash multiple decl modifiers
for declarations like function. This is not used for struct declaration
because only accessibility modifier is allowed.
* Serialize TopLevelDecls as an array of raw syntax nodes, instead of one-after-another.
* Add tests for multiple decls
* Remove extra newlines from test inputs
