Commit Graph

727 Commits

Author SHA1 Message Date
Rintaro Ishizaki
da95961de2 [SyntaxParse] Fix a crash in TokenReceiver
rdar://problem/55267292
2019-09-11 10:51:07 -07:00
Rintaro Ishizaki
a7d6612813 Merge pull request #27046 from rintaro/syntaxparse-refactor-type
[SyntaxParse] Refactor parsing facilities
2019-09-11 06:43:42 +02:00
Rintaro Ishizaki
a9af910664 [SyntaxParse] Add ignoreToken() function to ignore tokens 2019-09-10 10:29:02 -07:00
Rintaro Ishizaki
8edea315cd [Syntax] Abolish 'backtick' trivia
- Stop producing 'backtick' trivia for escaping identifier token. '`'s
  are now parts of the token text
- Adjust and simplify C++ libSyntax APIs
- Add 'is_deprecated' property to Trivia.py to attribute SwiftSyntax
  APIs

rdar://problem/54810608
2019-09-09 11:49:25 -07:00
Doug Gregor
6d390e44d9 Allow delayed body parsing in parseDelayedDecl().
When performing delayed decl parsing (e.g., for code completion),
allow other function bodies to be delayed.
2019-09-03 11:41:19 -07:00
Doug Gregor
d7f6768a9d Eliminate DelayedParsingCallbacks.
DelayedParsingCallbacks only had one implementation, for code
completion, which is only used to determine which bodies to skip and
which to delay. Inline that logic into the parser's delay logic and
remove DelayedParsingCallbacks entirely.
2019-09-02 19:03:07 -07:00
Doug Gregor
d8b745db77 Add a request to lazily parse function bodies.
Rework the lazy function body parsing mechanism to use the
request-evaluator, so that asking for the body of a function will
initiate parsing. Clean up a number of callers to
AbstractFunctionDecl::getBody() that don't actually need the body, so
we don't perform unnecessary parsing.

This change does not delay parsing of function bodies in the general
case; rather, it sets up the infrastructure to always delay parsing of
function bodies.
2019-08-31 16:23:08 -07:00
eeckstein
903eeb220b Revert "Add a request to lazily parse function bodies." 2019-08-31 12:29:01 +02:00
Doug Gregor
2ab05a6835 Add a request to lazily parse function bodies.
Rework the lazy function body parsing mechanism to use the
request-evaluator, so that asking for the body of a function will
initiate parsing. Clean up a number of callers to
AbstractFunctionDecl::getBody() that don't actually need the body, so
we don't perform unnecessary parsing.

This change does not delay parsing of function bodies in the general
case; rather, it sets up the infrastructure to always delay parsing of
function bodies.
2019-08-30 16:48:23 -07:00
Doug Gregor
11969847c4 Simplify lazy parsing of nominal and extension members.
Lazy parsing for the members of nominal types and extensions depends
only on information already present in
`IterableDeclContext`. Eliminate the use of PersistentParserState as
an intermediary and have the member-parsing request construct a new
`Parser` instance itself to handle parsing. Make this possible even
for ill-formed nominal types/extensions to simplify the code path.

Eliminate `LazyMemberParser` and all of its uses, because it was only
present for lazy member parsing, which no longer needs it.
2019-08-30 08:29:43 -07:00
Rintaro Ishizaki
cb308b7e53 Revert "Revert "[Parser] Decouple the parser from AST creation (part 2)""
This reverts commit 8ad3cc8a82.
2019-08-27 14:36:41 -07:00
Rintaro Ishizaki
8ad3cc8a82 Revert "[Parser] Decouple the parser from AST creation (part 2)" 2019-08-27 12:28:48 -07:00
Jan Svoboda
77924c4b84 [Parser] Decouple the parser from AST creation (part 2)
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
2019-08-26 19:10:51 +02:00
David Ungar
a40b694801 Use token start locations 2019-08-21 22:36:15 -07:00
David Ungar
d84d541e99 De-lazify 2019-08-21 12:02:25 -07:00
David Ungar
0dbf7e6ab6 Parser changes for lazy ASTScopes 2019-08-21 09:45:45 -07:00
David Ungar
fe5e4ee96f Correct comment for Parser::parseMatchingToken. 2019-08-13 09:54:54 -07:00
Rintaro Ishizaki
94f0c14675 [SyntaxParse] Return dedicated pointer from HiddenLibSyntaxAction
SyntaxParseActions::recordToken() et al. may return the same pointer
value for different nodes (e.g. `nullptr`). So we cannot use DenseMap to
associate the node from the explicit syntax parsing actions to libSyntax
node. Instead, use a structure that wraps them.
2019-07-31 16:14:32 -07:00
Jan Svoboda
f0395a469a Revert "Revert "[Parser] Decouple the parser from AST creation (part 1)"" 2019-07-30 04:34:09 +00:00
Brent Royal-Gordon
c66a8be4eb Revert "[Parser] Decouple the parser from AST creation (part 1)" 2019-07-09 14:07:43 -07:00
Jan Svoboda
5aee188716 Rename SyntaxTransformer to ASTGen 2019-07-08 17:16:35 +02:00
Jan Svoboda
44d7769238 [Parser] Decouple the parser from AST creation (part 1)
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
2019-06-28 14:28:19 +02:00
Saleem Abdulrasool
731c31f9a5 MSVC: litter the code with llvm_unreachable (NFC)
Add `llvm_unreachable` to mark covered switches which MSVC does not
analyze correctly and believes that there exists a path through the
function without a return value.
2019-06-01 19:02:46 -07:00
Sam Lazarus
3d9b6396a1 Merge pull request #24059 from sl/sl/sr-10293
Allow var / let as parameter names but provide a warning and fixit to add backticks.
2019-04-26 09:30:23 -04:00
Sam Lazarus
ede8127adf Test: Add and update tests for allowing var and let as argument labels 2019-04-26 04:08:27 -04:00
Sam Lazarus
2a38b48eea Parse / AST: Allow let / var as argument labels with a warning.
The diagnostic is now a warning and the new message alerts the user that
though it is valid to have let and var as argument label names,
they are interpreted as argument labels, not keywords.
2019-04-26 04:08:01 -04:00
Doug Gregor
32b0245187 [Parser] Consistently use consumeIdentifier() for normal identifiers.
consumeIdentifier() provides the general way in which we consume an
identifier token and fill in an Identifier. Use it consistently in the
parser.
2019-04-23 11:31:58 -07:00
Harlan Haskins
f5fc6f0c57 [Lexer] Handle SwiftInterface files as well as SIL
Previously, the Lexer kept a single flag whether we’re lexing Swift or SIL. Instead, keep track if we’re parsing Swift, SIL, or a Swiftinterface file. .swiftinterface files allow $-prefixed identifiers anywhere.
2019-01-22 11:02:36 -08:00
Slava Pestov
83472bca29 Frontend: Only enable delayed member parsing for non-primary files 2019-01-18 00:15:53 -05:00
Slava Pestov
fb0fbc099b Parse: Rename DisableDelayedParsing to DelayBodyParsing and change polarity 2019-01-18 00:15:41 -05:00
Argyrios Kyrtzidis
c7ac859310 [Parse] Optimize syntax parsing: Speed-up Lexer::lexTrivia()
Introduce ParsedTrivia which is a more efficient structure to use during lexing than syntax::Trivia.
2019-01-17 12:10:27 -08:00
Xi Ge
6057a60aca [Parser] Expose a flag to allow users explicitly disable delayed parsing. NFC
Discussed with @dcci, this patch is necessary to fix an lldb test failure. rdar://38396444
2019-01-09 14:32:41 -08:00
Argyrios Kyrtzidis
4806fb1cef [ParsedRawSyntaxNode] Explicitely specify if a ParsedRawSyntaxNode is null using a DataKind
Avoid implicitely assuming 'null' node if its OpaqueSyntaxNode is null, there should be no interpretation
of OpaqueSyntaxNode values, a SyntaxParseActions implementation should be able to return null pointers as OpaqueSyntaxNode.
2019-01-07 19:56:37 -08:00
Argyrios Kyrtzidis
ab7427723e [Parse/Syntax] Refactoring to decouple the parser from syntax tree creation
Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code.
This decouples the act of parsing from the act of forming a syntax tree representation.
'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree.
To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two.

This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.
2019-01-07 19:52:59 -08:00
Argyrios Kyrtzidis
66be4a58b4 [Parser] Introduce ParserUnit::parse() function to avoid duplication of code 2018-12-16 09:52:15 -08:00
Adrian Prantl
ff63eaea6f Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

      for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
2018-12-04 15:45:04 -08:00
Marc Rasi
bf18697b4f parsing, typechecking, and SILGen for #assert
`#assert` is a new static assertion statement that will let us write
tests for the new constant evaluation infrastructure that we are working
on. `#assert` works by lowering to a `Builtin.poundAssert` SIL
instruction. The constant evaluation infrastructure will look for these
SIL instructions, const-evaluate their conditions, and emit errors if
the conditions are non-constant or false.

This commit implements parsing, typechecking and SILGen for `#assert`.
2018-11-07 16:34:17 -08:00
Brent Royal-Gordon
9bd1a26089 Implementation for SE-0228: Fix ExpressibleByStringInterpolation (#20214)
* [CodeCompletion] Restrict ancestor search to brace

This change allows ExprParentFinder to restrict certain searches for parents to just AST nodes within the nearest surrounding BraceStmt. In the string interpolation rework, BraceStmts can appear in new places in the AST; this keeps code completion from looking at irrelevant context.

NFC in this commit, but keeps code completion from crashing once TapExpr is introduced.

* Remove test relying on ExpressibleByStringInterpolation being deprecated

Since soon enough, it won’t be anymore.

* [AST] Introduce TapExpr

TapExpr allows a block of code to to be inserted between two expressions, accessing and potentially mutating the result of its subexpression before giving it to its parent expression. It’s roughly equivalent to this function:

  func _tap<T>(_ value: T, do body: (inout T) throws -> Void) rethrows -> T {
    var copy = value
    try body(&copy)
    return copy
  }

Except that it doesn’t use a closure, so no variables are captured and no call frame is (even notionally) added.

This commit does not include tests because nothing in it actually uses TapExpr yet. It will be used by string interpolation.

* SE-0228: Fix ExpressibleByStringInterpolation

This is the bulk of the implementation of the string interpolation rework. It includes a redesigned AST node, new parsing logic, new constraints and post-typechecking code generation, and new standard library types and members.

* [Sema] Rip out typeCheckExpressionShallow()

With new string interpolation in place, it is no longer used by anything in the compiler.

* [Sema] Diagnose invalid StringInterpolationProtocols

StringInterpolationProtocol informally requires conforming types to provide at least one method with the base name “appendInterpolation” with no (or a discardable) return value and visibility at least as broad as the conforming type’s. This change diagnoses an error when a conforming type does not have a method that meets those criteria.

* [Stdlib] Fix map(String.init) source break

Some users, including some in the source compatibility suite, accidentally used init(stringInterpolationSegment:) by writing code like `map(String.init)`. Now that these intializers have been removed, the remaining initializers often end up tying during overload resolution. This change adds several overloads of `String.init(describing:)` which will break these ties in cases where the compiler previously selected `String.init(stringInterpolationSegment:)`.

* [Sema] Make callWitness() take non-mutable arrays

It doesn’t actually need to mutate them.

* [Stdlib] Improve floating-point interpolation performance

This change avoids constructing a String when interpolating a Float, Double, or Float80. Instead, we write the characters to a fixed-size buffer and then append them directly to the string’s storage.

This seems to improve performance for all three types, but especially for Double and Float80, which cannot always fit into a small string when stringified.

* [NameLookup] Improve MemberLookupTable invalidation

In rare cases usually involving generated code, an overload added by an extension in the middle of a file would not be visible below it if the type had lazy members and the same base name had already been referenced above the extension. This change essentially dirties a type’s member lookup table whenever an extension is added to it, ensuring the entries in it will be updated.

This change also includes some debugging improvements for NameLookup.

* [SILOptimizer] XFAIL dead object removal failure

The DeadObjectRemoval pass in SILOptimizer does not currently remove reworked string interpolations as well as the old design because their effects cannot be described by @_effects(readonly). That causes a test failure on Linux. This change temporarily silences that test. The SILOptimizer issue has been filed as SR-9008.

* Confess string interpolation’s source stability sins

* [Parser] Parse empty interpolations

Previously, the parser had an odd asymmetry which caused the same function to accept foo(), but reject “\()”. This change fixes the issue.

Already tested by test/Parse/try.swift, which uses this construct in one of its throwing interpolation tests.

* [Sema] Fix batch-mode-only lazy var bug

The temporary variable used by string interpolation needs to be recontextualized when it’s inserted into a synthesized getter. Fixes a compilation failure in Alamofire.

I’ll probably follow up on this bug a bit more after merging.
2018-11-02 19:16:03 -07:00
Christopher Ian Stern
469a0f3eb0 fix stack overflow on deeply nested parens [SR-4866] (#19631)
Make sure StructureMarkerRAII checks structure nesting level on all paths.  Previously swift crashed with  no diagnostic on deeply nested '('. Now we print an error when more than 256 parens deep, just as we always have for '['.

fixes SR-4866
2018-10-10 13:42:18 +09:00
Rintaro Ishizaki
143f55a6e5 [Lexer] Add formStringLiteralToken dedicated for forming string literal 2018-09-19 18:58:54 +09:00
Xi Ge
0a03259f4a Parser: disable lexer diagnostics during delayed parsing.
The lexer diagnostics have already been emitted during token skipping.
2018-09-11 16:30:57 -07:00
Xi Ge
b4356d4f93 Merge pull request #19104 from nkcsgexi/parser-laziness
Parser: lazily parse members in extension decls.
2018-09-06 15:24:17 -07:00
John Holdsworth
4da8cbe655 Implement SE-0200 (extended escaping in string literals)
Supports string literals like #"foo"\n"bar"#.
2018-09-06 15:19:52 -07:00
Brent Royal-Gordon
df22ea1bfb Revert "[Parse] Implementation for SE-200 (raw strings)" 2018-09-06 12:22:41 -07:00
swift-ci
192587c98a Merge pull request #17668 from johnno1962a/master 2018-09-06 11:47:36 -07:00
Xi Ge
24b0eac9a4 Parser: parse members in extension decls incrementally. 2018-09-05 17:00:39 -07:00
John Holdsworth
dc96342368 Response to xwu's review 2018-09-02 11:37:02 +01:00
Rintaro Ishizaki
2a1ab7d8e2 Merge pull request #19046 from rintaro/parse-refactor-getsetdecl
[Parse] refactor get/set block parsing
2018-09-01 00:32:57 +09:00
Rintaro Ishizaki
4797a7caf6 Merge pull request #18919 from rintaro/ide-complete-pound
[CodeCompletion] Implement completion for # directives
2018-08-31 10:50:40 +09:00
Rintaro Ishizaki
51b7168d05 [Parse] Refactor get/set accessor parsing
* Use 'parseAbstractFunctionBody()' for accessors as well. This
  simplifies the implementation, and makes 'parseAbstractFunctionBody()'
  the single point of parsing body of every 'AbstructFunctionDecl' types.
2018-08-29 18:01:21 +09:00