[libSyntax] Add a reference counted version of OwnedString

We cannot use unowned strings for token texts of incrementally parsed
syntax trees since the source buffer to which reused nodes refer will
have been freed for reused nodes. Always copying the token text whenever
OwnedString is passed is too expensive. A reference counted copy of the
string allows us to keep the token's string alive across incremental
parses while eliminating unnecessary copies.
This commit is contained in:
Alex Hoppen
2018-08-13 11:52:16 -07:00
parent a03749a743
commit ac512d4341
10 changed files with 132 additions and 273 deletions

View File

@@ -312,15 +312,15 @@ swift::tokenizeWithTrivia(const LangOptions &LangOpts, const SourceManager &SM,
syntax::AbsolutePosition RunningPos;
tokenize(
LangOpts, SM, BufferID, Offset, EndOffset,
Diags,
LangOpts, SM, BufferID, Offset, EndOffset, Diags,
CommentRetentionMode::AttachToNextToken, TriviaRetentionMode::WithTrivia,
/*TokenizeInterpolatedString=*/false,
/*SplitTokens=*/ArrayRef<Token>(),
[&](const Token &Tok, const Trivia &LeadingTrivia,
const Trivia &TrailingTrivia) {
auto Text = OwnedString::makeRefCounted(Tok.getText());
auto ThisToken =
RawSyntax::make(Tok.getKind(), Tok.getText(), LeadingTrivia.Pieces,
RawSyntax::make(Tok.getKind(), Text, LeadingTrivia.Pieces,
TrailingTrivia.Pieces, SourcePresence::Present);
auto ThisTokenPos = ThisToken->accumulateAbsolutePosition(RunningPos);