Commit Graph

952 Commits

Author SHA1 Message Date
Harlan
5e02d2a877 Implement #warning and #error (#14048)
* Implement #warning and #error

* Fix #warning/#error in switch statements

* Fix AST printing for #warning/#error

* Add to test case

* Add extra handling to ParseDeclPoundDiagnostic

* fix dumping

* Consume the right paren even in the failure case

* Diagnose extra tokens on the same line after a diagnostic directive
2018-02-03 18:07:05 -05:00
Rintaro Ishizaki
2134f9f768 [Parse] Move SyntaxParsingContext to Parse (#14360)
This is only for the Parser.
Also, this resolves layering violation where libAST and libSyntax
depends on each other.
2018-02-03 10:43:52 +09:00
Rintaro Ishizaki
0780c529c4 [Syntax] Unify RawSyntax and RawTokenSyntax using union and TrailingObjects
It better matches with SwiftSyntax model.

Using TrailingObjects reduces the number of heap allocation which
gains 18% performance improvement.
2018-01-18 14:49:46 +09:00
Xi Ge
4c172037e8 [libSyntax][SourceKit] Integrating libSyntax representation of a source file with several SourceKitd syntax requests (#14000)
The enhanced SourceKitd requests are EditorOpen and EdtiorReplaceText. In these two requests, the clients can specify a flag "key. enablesyntaxtree = 1" to get a serialize libSyntax tree with the response.

To help this integration, we added a function in SyntaxParsingContext to explicitly finalize the creation of a SourceFileSyntax to incorporate the fact that SourceKit needs the tree before its destroying the parser instance.

To test this integration, we diff the syntax tree serialized from the frontend action and the tree serialized from a SourceKitd response. They should be identical.
2018-01-17 21:13:22 -08:00
John McCall
7f0f8830cd Split AccessorDecl out from FuncDecl. NFC.
This has three principal advantages:

- It gives some additional type-safety when working
  with known accessors.

- It makes it significantly easier to test whether a declaration
  is an accessor and encourages the use of a common idiom.

- It saves a small amount of memory in both FuncDecl and its
  serialized form.
2018-01-12 14:20:27 -05:00
Xi Ge
22ce6934bd libSyntax: parse string interpolation expression. (#13708)
A string interpolation expression is composed of { OpenQuote, Segments,
CloseQuote }. To represent OpenQuote, CloseQuote and StringSegment, we have to
introduce new token kinds correspondingly.
2018-01-03 20:41:34 -08:00
Rintaro Ishizaki
01f42c89d3 Merge pull request #13631 from omochi/lex-escape-backtick
[Parse] Lexer build backtick trivia around espaced identifier token
2017-12-29 12:36:00 +09:00
omochimetaru
bc88330740 [Parse] Lexer build backtick trivia around espaced identifier token 2017-12-29 00:22:49 +09:00
Rintaro Ishizaki
48eb60702e [Syntax] Stop using SyntaxBuilders for TypeIdentifier
Since we haven't handled error cases yet, Error type identifier (e.g.
'Array.+') used to case round trip problem.
2017-12-27 13:18:17 +09:00
Rintaro Ishizaki
39746a9e58 [Parse] Inline skipExtraTopLevelRBraces()
Removed from 'parseTopLevel()'. There's no particular reason to diagnose
it in 'parseTopLevel()'.
2017-12-25 23:32:55 +09:00
Rintaro Ishizaki
7aea64ded7 Merge pull request #13598 from rintaro/syntax-stmt-part2
[Syntax] Parse "if", "guard" and "for-in"  statement nodes
2017-12-23 15:51:46 +09:00
Rintaro Ishizaki
02769d77c2 [Parse] Factor out StmtConditionElement parsing 2017-12-22 18:28:04 +09:00
omochimetaru
24509a0bde [Parse] Change LeadingTrivia type to Trivia 2017-12-20 14:09:47 +09:00
Rintaro Ishizaki
181333ce0f [Lexer] Lex conflict marker as a trivia 2017-12-19 09:24:33 +09:00
Rintaro Ishizaki
023379cde5 Merge pull request #13495 from omochi/lexer-refactor-skip-to-eol
[Parse] Refactor Lexer's skipToEndOfline
2017-12-19 09:12:07 +09:00
Xi Ge
fe1a4ca0d2 libSyntax: create libSyntax node for var decl accessors. 2017-12-18 15:16:14 -08:00
omochimetaru
5de598f34a [Parse] use skipHashbang in lexTrivia 2017-12-18 18:22:04 +09:00
omochimetaru
aeb9ba6f96 [Parse] use skipSlashSlashComment in lexTrivia 2017-12-18 18:22:04 +09:00
omochimetaru
f7136ae635 [Parse] delete skipUpToEndOfLine 2017-12-18 18:22:04 +09:00
omochimetaru
ed58c152bf [Parse] Improve Lexer's UTF-8 BOM handling (#13483)
* Add BOM handling testcases
* Add ContentStart to Lexer for BOM handling
2017-12-18 17:22:11 +09:00
Rintaro Ishizaki
7b4fc5b196 [Syntax] Restore Lexer state by restoring LeadingTrivia
Instead of re-lexing Trivias. Trivia might contain skipped garbage which
might be re-lexed as tokens.
2017-12-09 14:48:21 +09:00
Rintaro Ishizaki
60bfa893b9 [Parse] Make PersistentParserState to hold ParserPosition
instead of PersistentParserState::ParserPos.
2017-12-09 13:58:50 +09:00
Rintaro Ishizaki
574d39bcb6 [Parse] Split ParserPosition declaration to dedicated header file 2017-12-09 13:58:50 +09:00
Rintaro Ishizaki
724052d266 [Parse] Rename Lexer::State to LexerState 2017-12-09 13:58:50 +09:00
Rintaro Ishizaki
5571e5cc76 [Lexer] Clear trivia at the top of lexImpl()
To make sure we only parse trivia for the current token.
2017-12-08 12:08:05 +09:00
Rintaro Ishizaki
d767dc39ba [Lexer] Improve implementation of lexTrivia 2017-12-08 11:36:20 +09:00
Xi Ge
fec040d95e libSyntax: support generic parameter clause. (#13286)
This patch also performs minor refactoring to align syntax parsing
context with the right scope. We start to support the generic clauses
because they are necessary pieces to construct struct or
function syntax node.
2017-12-05 19:34:55 -08:00
Rintaro Ishizaki
d46073dd75 [libSyntax] Backtracking restarts from leading trivia position
When reading syntax.
2017-12-04 10:46:03 -08:00
Rintaro Ishizaki
c0ceb7a943 [Parse] Simplify parsing name for function declarations (#13141) 2017-12-05 02:04:01 +09:00
Rintaro Ishizaki
0bcaf909b2 [Parse] Gardening: remove unused function
Remove 2 convenient functions:
* parseTypeSimple()
* parseTypeSimpleOrComposition()
2017-11-30 15:57:58 +09:00
Rintaro Ishizaki
d389a812e2 [libSyntax] Support parsing postfix type nodes
* OptionalType (?)
* ImplicitlyUnwrappedOptional (!)
* MetatypeType (.Type, .Protocol)
2017-11-30 14:31:21 +09:00
Rintaro Ishizaki
5923e4fee4 [libSyntax] Support parsing collection types 2017-11-30 11:49:39 +09:00
Rintaro Ishizaki
640a11d11e [libSyntax] Return type-identifier as SyntaxParserResult 2017-11-29 09:58:05 +09:00
Rintaro Ishizaki
e7cfae0ba9 [libSyntax] Support parsing type-identifier 2017-11-29 09:57:59 +09:00
Rintaro Ishizaki
d1b0d137da [Parse] Set appropriate token kind when consumeStartingCharacterOfCurrentToken() 2017-11-29 09:37:22 +09:00
Xi Ge
e560170a60 libSyntax: Parse member access expression. (#13119)
* libSyntax: Parse member access expression.

This patch uses createNodeInPlace from syntax parsing context API to
merge an expression with its suffix to create recursive nodes such as
member access expression.

Meanwhile, this patch breaks down a signed integer or float literal to a
prefix operator expression. This expression consists of two parts: an
operator and the following expression. This makes literals like "+1" or
"-1" no different from other prefix unary expressions such as "!true".
2017-11-28 16:29:50 -08:00
Rintaro Ishizaki
0a401b381c [Syntax] Rewrite SyntaxParsingContext
Read RawSyntaxToken along with Parser::consumeToken()

* Single Lexer pass
* Backtracking support
* Split token support
2017-11-18 15:35:46 +09:00
Rintaro Ishizaki
a78fda0720 [Syntax] Always lex Trivia when SF.shouldKeepSyntaxInfo()
For backward compatibility, Don't lex comments as trailing trivias.
2017-11-17 14:56:49 +09:00
Rintaro Ishizaki
40b195d98c [Syntax] Get rid of fullLex
Defer (Token, Trivia) -> RawTokenSyntax conversion from Lexer to Parser.
This is a part of effort for consolidating Syntax and AST parsing.
2017-11-17 14:56:49 +09:00
Xi Ge
0d9745f6eb libSyntax: teach parser to parse dictionary and array literals. (#12821)
This commit also adds ArrayExpr and DictionaryExpr to the libSyntax nodes
family. Also, it refactors the original parser code for these two
expressions to better fit to the design of SyntaxParsingContext.

This commit has also fixed two crashers.
2017-11-09 09:00:43 -08:00
Xi Ge
0390d452a6 libSyntax: parse function argument syntax node. (#12803)
This commit teaches parser to parse two libSyntax nodes: FunctionCallArgument and
FunctionCallArgumentList. Along with the change, some libSyntax parsing infrastructure changes
as well: (1) parser doesn't directly insert token into the buffer for libSyntax node creation;
instead, when creating a simple libSyntax node like integer literal expression, parser should indicate the location of the last token in the node; (2) implicit libSyntax nodes like empty
statement list must contain a source location indicating where the implicit nodes should appear
(immediately before the token at the given location).
2017-11-07 15:59:00 -08:00
Xi Ge
75db3c1db8 Re-apply libSyntax patches after fixing ASAN issue (#12730)
* Re-apply "libSyntax: Ensure round-trip printing when we build syntax tree from parser incrementally. (#12709)"

* Re-apply "libSyntax: Root parsing context should hold a reference to the current token in the parser, NFC."

* Re-apply "libSyntax: avoid copying token text when lexing token syntax nodes, NFC. (#12723)"

* Actually fix the container-overflow issue.
2017-11-03 13:25:33 -07:00
Xi Ge
7ebf66ed2d libSyntax: forward declare libSyntax entities in several header files, NFC. (#12735) 2017-11-02 20:55:18 -07:00
Xi Ge
4d1249aa82 Revert "libSyntax: Ensure round-trip printing when we build syntax tree from parser incrementally. (#12709)"
This reverts commit 0d98c4c5df.
2017-11-02 14:44:26 -07:00
Xi Ge
0d98c4c5df libSyntax: Ensure round-trip printing when we build syntax tree from parser incrementally. (#12709) 2017-11-01 20:29:30 -07:00
Slava Pestov
93c80da77c Parse: __shared and __owned should be contextual keywords
This was a source compatibility regression, someone actually had
an identifier named __shared.
2017-10-24 20:45:50 -07:00
Xi Ge
844aeae2d5 Re-apply "libSyntax: create a basic infrastructure for generating libSyntax entities by using Parser." (#12538) 2017-10-20 22:58:28 -07:00
Greg Parker
48a6b9d464 Revert "libSyntax: create a basic infrastructure for generating libSyntax entities by using Parser."
This reverts commit ee7a06276d.
It causes build failures like "'swift/Syntax/SyntaxNodes.h' file not found".
2017-10-19 17:11:48 -07:00
Xi Ge
ee7a06276d libSyntax: create a basic infrastructure for generating libSyntax entities by using Parser. 2017-10-18 17:02:00 -07:00
Xi Ge
2ba1ca2d8f IDE: simplify some code. NFC (#11935)
* IDE: simplify some code. NFC

* add assert.
2017-09-14 18:19:48 -07:00