Commit Graph

8 Commits

Author SHA1 Message Date
Alex Hoppen
e43bad2c71 [libSyntax] Store the token's text in the SyntaxArena
Do the same thing that we are already doing for trivia: Since RawSyntax
nodes always live inside a SyntaxArena, we don't need to tail-allocate
an OwnedString to store the token's text. Instead we can just copy it
to the SyntaxArena. If we copy the entire source buffer to the syntax
arena at the start of parsing, this means that no more copies are
required later on. Plus we also avoid ref-counting the OwnedString which
should also increase performance.
2021-02-10 09:50:12 +01:00
Alex Hoppen
5e1ba8b16e [libSyntax] Store raw trivia inside RawSyntax and only lex into pieces when requested 2021-02-05 08:15:54 +01:00
Alex Hoppen
803499e165 [libSyntax] Require RawSyntax to always live inside a SyntaxArena
This way, we will later be able to store additional information about
the node inside the same arena with a guarantee that they will always be
alive as long as the node is alive.

These additional information will include
a) the token's text (which can be a StringRef into a copy of the source
   code that lives inside the SyntaxArena)
b) the token's unparsed trivia, which can be decomposed into pieces when
   needed.
2021-02-01 10:34:44 +01:00
Alex Hoppen
8bb1167e21 [libSyntax] Restructure RawSyntax to more closely resemble the SwiftSyntax implementation 2021-01-29 13:08:12 +01:00
moatom
2e95a0d265 Fix include guards 2019-06-02 12:10:43 +09:00
Alex Hoppen
ac512d4341 [libSyntax] Add a reference counted version of OwnedString
We cannot use unowned strings for token texts of incrementally parsed
syntax trees since the source buffer to which reused nodes refer will
have been freed for reused nodes. Always copying the token text whenever
OwnedString is passed is too expensive. A reference counted copy of the
string allows us to keep the token's string alive across incremental
parses while eliminating unnecessary copies.
2018-08-13 15:37:53 -07:00
Alex Hoppen
9d59cd286b [incrParse] Add a stable id to the syntax nodes
The id is meant to be stable across incremental parses
2018-07-13 16:56:03 -07:00
Dexin Li
e0f8b27117 [Syntax]Add a deserializer that convert json to libSyntax tree (#15203) 2018-03-16 15:22:04 -07:00