The existing libSyntax infrastructure uses external python
dictionaries to share logic between C++ and Swift implementations.
This patch teaches trivia kinds to adapt to this infrastructure
as well.
Make the quoted parameters more portable. This is needed in order to
support building swift on Windows. Simplify the declaration of the
flags and do a small formatting clean up. This ensures that the ninja
generation gets the command line invocation correct.
* Implement #warning and #error
* Fix #warning/#error in switch statements
* Fix AST printing for #warning/#error
* Add to test case
* Add extra handling to ParseDeclPoundDiagnostic
* fix dumping
* Consume the right paren even in the failure case
* Diagnose extra tokens on the same line after a diagnostic directive
Introduced SyntaxArena for managing memory and cache.
SyntaxArena holds BumpPtrAllocator as a allocation storage.
RawSyntax is now able to be constructed with normal heap allocation, or
by SyntaxArena. RawSyntax has ManualMemory flag which indicates it's managed by
SyntaxArena. If the flag is true, its Retain()/Release() is no-op thus it's
never destructed by IntrusiveRefCntPtr.
This speedups the memory allocation for RawSyntax.
Also, in Syntax parsing, "token" RawSyntax is reused if:
a) It's not string literal with >16 length; and
b) It doesn't contain random text trivia (e.g. comment).
This reduces the overall allocation cost.
The enhanced SourceKitd requests are EditorOpen and EdtiorReplaceText. In these two requests, the clients can specify a flag "key. enablesyntaxtree = 1" to get a serialize libSyntax tree with the response.
To help this integration, we added a function in SyntaxParsingContext to explicitly finalize the creation of a SourceFileSyntax to incorporate the fact that SourceKit needs the tree before its destroying the parser instance.
To test this integration, we diff the syntax tree serialized from the frontend action and the tree serialized from a SourceKitd response. They should be identical.
With more syntax nodes being specialized, we'd like this
straight-forward way to pinpoint unknown entities. This diagnostics
is only issued in -emit-syntax frontend action and swift-syntax-test
invocation.
This patch reworks the line-directive command line argument for gyb to
take in a python-style format string with `line` and `file` variables
to be substituted.
libSyntax nodes don't maintain absolute source location on each
individual node. Instead, the absolute locations are calculated on
demand with a given root by accumulating the length of all the other
nodes before the target node. This bridging is important for issuing
diagnostics from libSyntax entities.
With the observation that our current implementation of the source
location calculation has multiple bugs, this patch re-implemented this
bridging by using the newly-added syntax visitor. Also, we moved the function
from RawSyntax to Syntax for better visibility.
To test this source location calculation, we added a new action in
swift-syntax-test. This action parses a given file as a
SourceFileSyntax, calculates the absolute location of the
EOF token in the SourceFileSyntax, and dump the buffer from the start
of the input file to the absolute location of the EOF. Finally, we compare
the dump with the original input to ensure they are identical.
This allows the root context to have a separate place to keep track of
the global data that each sub-context can access to, for instance,
SourceFile, DiagnosticEngine, etc.
A string interpolation expression is composed of { OpenQuote, Segments,
CloseQuote }. To represent OpenQuote, CloseQuote and StringSegment, we have to
introduce new token kinds correspondingly.
This patch adds a python function to syntax node gyb support called
"check_child_condition". Given a child's definition, this function
generate a C++ closure to check whether a given syntax node can satisfy
the condition of the child node. This function recursively generates code
for node choices too, therefore we don't need to hard code the
condition checking for node choices.
Some structures of syntax nodes can have children choices, e.g. a
dictionary expression can either contain a single ':' token or a list of
key-value pairs.
This patch gives the existing code generation infrastructure a way to
specify such node choices. Node choices are specified under a child
declaration with two constraints: a choice cannot be declared as
optional, and a choice cannot have further recursive choices.
Since we don't have too many node structures with choices, part of the
SyntaxFactory code for these nodes is manually typed.
This patch also teaches AccessorBlock to use node choices.
