[Syntax] Abolish 'backtick' trivia

- Stop producing 'backtick' trivia for escaping identifier token. '`'s
  are now parts of the token text
- Adjust and simplify C++ libSyntax APIs
- Add 'is_deprecated' property to Trivia.py to attribute SwiftSyntax
  APIs

rdar://problem/54810608
This commit is contained in:
Rintaro Ishizaki
2019-09-09 10:21:06 -07:00
parent 19f5ac15c9
commit 8edea315cd
12 changed files with 42 additions and 63 deletions

View File

@@ -319,7 +319,7 @@ swift::tokenizeWithTrivia(const LangOptions &LangOpts, const SourceManager &SM,
/*SplitTokens=*/ArrayRef<Token>(),
[&](const Token &Tok, const ParsedTrivia &LeadingTrivia,
const ParsedTrivia &TrailingTrivia) {
CharSourceRange TokRange = Tok.getRangeWithoutBackticks();
CharSourceRange TokRange = Tok.getRange();
SourceLoc LeadingTriviaLoc =
TokRange.getStart().getAdvancedLoc(-LeadingTrivia.getLength());
SourceLoc TrailingTriviaLoc =
@@ -328,7 +328,7 @@ swift::tokenizeWithTrivia(const LangOptions &LangOpts, const SourceManager &SM,
LeadingTrivia.convertToSyntaxTrivia(LeadingTriviaLoc, SM, BufferID);
Trivia syntaxTrailingTrivia =
TrailingTrivia.convertToSyntaxTrivia(TrailingTriviaLoc, SM, BufferID);
auto Text = OwnedString::makeRefCounted(Tok.getText());
auto Text = OwnedString::makeRefCounted(Tok.getRawText());
auto ThisToken =
RawSyntax::make(Tok.getKind(), Text, syntaxLeadingTrivia.Pieces,
syntaxTrailingTrivia.Pieces,