Having this be a single buffer hardcoded in the SourceManager and set
by all clients is silly. SourceFiles with the 'Main' kind are allowed
to have hashbang lines (`#!`), other files are not. And anyone
manually setting up a Lexer can decide for themselves.
No intended behavioral change.
* Reject bad string interpolations
String interpolations with multiple comma-separate expressions or argument labels were being incorrectly accepted.
* Tweak error name to match message
* Diagnose empty interpolations more clearly
* Don’t double-diagnose parse errors
Fixes a test at Parse/recovery.swift:799 which the previous commit broke.
* Fix incorrect test RUN: line
A previous version of this test used FileCheck instead of -verify, and the run line wasn’t properly corrected to use -verify.
* Update comment
* Add more argument label tests
Ensures that we don’t get different results from an initializer that doesn’t exist or doesn’t take a String.
* Resolve the SR-7958 crasher test
We now diagnose the error and remove the label before it has an opportunity to crash.
Before this patch, we have one flag (KeepSyntaxInfo) to turn on two syntax
functionalities of parser: (1) collecting parsed tokens for coloring and
(2) building syntax trees. Since sourcekitd is the only consumer of either of these
functionalities, sourcekitd by default always enables such flag.
However, empirical results show (2) is both heavier and less-frequently
needed than (1). Therefore, separating the flag to two flags makes more
sense, where CollectParsedToken controls (1) and BuildSyntaxTree
controls (2).
CollectingParsedToken is always enabled by sourcekitd because
formatting and syntax-coloring need it; however BuildSyntaxTree should
be explicitly switched on by sourcekitd clients.
resolves: rdar://problem/37483076
Introduced SyntaxArena for managing memory and cache.
SyntaxArena holds BumpPtrAllocator as a allocation storage.
RawSyntax is now able to be constructed with normal heap allocation, or
by SyntaxArena. RawSyntax has ManualMemory flag which indicates it's managed by
SyntaxArena. If the flag is true, its Retain()/Release() is no-op thus it's
never destructed by IntrusiveRefCntPtr.
This speedups the memory allocation for RawSyntax.
Also, in Syntax parsing, "token" RawSyntax is reused if:
a) It's not string literal with >16 length; and
b) It doesn't contain random text trivia (e.g. comment).
This reduces the overall allocation cost.
With more syntax nodes being specialized, we'd like this
straight-forward way to pinpoint unknown entities. This diagnostics
is only issued in -emit-syntax frontend action and swift-syntax-test
invocation.
This has three principal advantages:
- It gives some additional type-safety when working
with known accessors.
- It makes it significantly easier to test whether a declaration
is an accessor and encourages the use of a common idiom.
- It saves a small amount of memory in both FuncDecl and its
serialized form.
This allows the root context to have a separate place to keep track of
the global data that each sub-context can access to, for instance,
SourceFile, DiagnosticEngine, etc.
A string interpolation expression is composed of { OpenQuote, Segments,
CloseQuote }. To represent OpenQuote, CloseQuote and StringSegment, we have to
introduce new token kinds correspondingly.
Tuple expression essentially has the same underlying structure as
function call arguments in libSyntax. However, we separate them as
different libSyntax kinds for better usability.
Different from AST, libSyntax currently allows single-child,
label-free tuple expressions (represented as ParenExpr in AST). This is
subject to change if we need to adopt the same differentiation in
libSyntax in the future.
Avoid heap-allocated memory for syntax parsing context.
Add more assertions to ensure syntax nodes are created only at the top of context stack.
Allow syntax parsing context to delay the specifying of context kind and target syntax kind.
This commit also adds ArrayExpr and DictionaryExpr to the libSyntax nodes
family. Also, it refactors the original parser code for these two
expressions to better fit to the design of SyntaxParsingContext.
This commit has also fixed two crashers.
This commit teaches parser to parse two libSyntax nodes: FunctionCallArgument and
FunctionCallArgumentList. Along with the change, some libSyntax parsing infrastructure changes
as well: (1) parser doesn't directly insert token into the buffer for libSyntax node creation;
instead, when creating a simple libSyntax node like integer literal expression, parser should indicate the location of the last token in the node; (2) implicit libSyntax nodes like empty
statement list must contain a source location indicating where the implicit nodes should appear
(immediately before the token at the given location).
This commit teaches parser to generate code block syntax node. As a support for this,
SyntaxParsingContext can be created by a single syntax kind, indicating the whole context
should be parsed into a node of that given syntax. Another change is to bridge created syntax
node with the given context kind. For instance, if a statement context results into an expression
node, the expression node will be bridged to a statement by wrapping it with a ExpressionStmt
node.
* Re-apply "libSyntax: Ensure round-trip printing when we build syntax tree from parser incrementally. (#12709)"
* Re-apply "libSyntax: Root parsing context should hold a reference to the current token in the parser, NFC."
* Re-apply "libSyntax: avoid copying token text when lexing token syntax nodes, NFC. (#12723)"
* Actually fix the container-overflow issue.
Since all parsing contexts need a reference to the current token of the
parser, we should pass the token reference to the root context. Therefore, the derived
sub-contexts can just copy it while being spawned.
