Commit Graph

557 Commits

Author SHA1 Message Date
Jan Svoboda
5aee188716 Rename SyntaxTransformer to ASTGen 2019-07-08 17:16:35 +02:00
Jan Svoboda
44d7769238 [Parser] Decouple the parser from AST creation (part 1)
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
2019-06-28 14:28:19 +02:00
Saleem Abdulrasool
731c31f9a5 MSVC: litter the code with llvm_unreachable (NFC)
Add `llvm_unreachable` to mark covered switches which MSVC does not
analyze correctly and believes that there exists a path through the
function without a return value.
2019-06-01 19:02:46 -07:00
Sam Lazarus
3d9b6396a1 Merge pull request #24059 from sl/sl/sr-10293
Allow var / let as parameter names but provide a warning and fixit to add backticks.
2019-04-26 09:30:23 -04:00
Sam Lazarus
ede8127adf Test: Add and update tests for allowing var and let as argument labels 2019-04-26 04:08:27 -04:00
Sam Lazarus
2a38b48eea Parse / AST: Allow let / var as argument labels with a warning.
The diagnostic is now a warning and the new message alerts the user that
though it is valid to have let and var as argument label names,
they are interpreted as argument labels, not keywords.
2019-04-26 04:08:01 -04:00
Doug Gregor
32b0245187 [Parser] Consistently use consumeIdentifier() for normal identifiers.
consumeIdentifier() provides the general way in which we consume an
identifier token and fill in an Identifier. Use it consistently in the
parser.
2019-04-23 11:31:58 -07:00
Harlan Haskins
f5fc6f0c57 [Lexer] Handle SwiftInterface files as well as SIL
Previously, the Lexer kept a single flag whether we’re lexing Swift or SIL. Instead, keep track if we’re parsing Swift, SIL, or a Swiftinterface file. .swiftinterface files allow $-prefixed identifiers anywhere.
2019-01-22 11:02:36 -08:00
Slava Pestov
83472bca29 Frontend: Only enable delayed member parsing for non-primary files 2019-01-18 00:15:53 -05:00
Slava Pestov
fb0fbc099b Parse: Rename DisableDelayedParsing to DelayBodyParsing and change polarity 2019-01-18 00:15:41 -05:00
Argyrios Kyrtzidis
c7ac859310 [Parse] Optimize syntax parsing: Speed-up Lexer::lexTrivia()
Introduce ParsedTrivia which is a more efficient structure to use during lexing than syntax::Trivia.
2019-01-17 12:10:27 -08:00
Xi Ge
6057a60aca [Parser] Expose a flag to allow users explicitly disable delayed parsing. NFC
Discussed with @dcci, this patch is necessary to fix an lldb test failure. rdar://38396444
2019-01-09 14:32:41 -08:00
Argyrios Kyrtzidis
4806fb1cef [ParsedRawSyntaxNode] Explicitely specify if a ParsedRawSyntaxNode is null using a DataKind
Avoid implicitely assuming 'null' node if its OpaqueSyntaxNode is null, there should be no interpretation
of OpaqueSyntaxNode values, a SyntaxParseActions implementation should be able to return null pointers as OpaqueSyntaxNode.
2019-01-07 19:56:37 -08:00
Argyrios Kyrtzidis
ab7427723e [Parse/Syntax] Refactoring to decouple the parser from syntax tree creation
Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code.
This decouples the act of parsing from the act of forming a syntax tree representation.
'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree.
To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two.

This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.
2019-01-07 19:52:59 -08:00
Argyrios Kyrtzidis
66be4a58b4 [Parser] Introduce ParserUnit::parse() function to avoid duplication of code 2018-12-16 09:52:15 -08:00
Adrian Prantl
ff63eaea6f Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

      for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
2018-12-04 15:45:04 -08:00
Marc Rasi
bf18697b4f parsing, typechecking, and SILGen for #assert
`#assert` is a new static assertion statement that will let us write
tests for the new constant evaluation infrastructure that we are working
on. `#assert` works by lowering to a `Builtin.poundAssert` SIL
instruction. The constant evaluation infrastructure will look for these
SIL instructions, const-evaluate their conditions, and emit errors if
the conditions are non-constant or false.

This commit implements parsing, typechecking and SILGen for `#assert`.
2018-11-07 16:34:17 -08:00
Brent Royal-Gordon
9bd1a26089 Implementation for SE-0228: Fix ExpressibleByStringInterpolation (#20214)
* [CodeCompletion] Restrict ancestor search to brace

This change allows ExprParentFinder to restrict certain searches for parents to just AST nodes within the nearest surrounding BraceStmt. In the string interpolation rework, BraceStmts can appear in new places in the AST; this keeps code completion from looking at irrelevant context.

NFC in this commit, but keeps code completion from crashing once TapExpr is introduced.

* Remove test relying on ExpressibleByStringInterpolation being deprecated

Since soon enough, it won’t be anymore.

* [AST] Introduce TapExpr

TapExpr allows a block of code to to be inserted between two expressions, accessing and potentially mutating the result of its subexpression before giving it to its parent expression. It’s roughly equivalent to this function:

  func _tap<T>(_ value: T, do body: (inout T) throws -> Void) rethrows -> T {
    var copy = value
    try body(&copy)
    return copy
  }

Except that it doesn’t use a closure, so no variables are captured and no call frame is (even notionally) added.

This commit does not include tests because nothing in it actually uses TapExpr yet. It will be used by string interpolation.

* SE-0228: Fix ExpressibleByStringInterpolation

This is the bulk of the implementation of the string interpolation rework. It includes a redesigned AST node, new parsing logic, new constraints and post-typechecking code generation, and new standard library types and members.

* [Sema] Rip out typeCheckExpressionShallow()

With new string interpolation in place, it is no longer used by anything in the compiler.

* [Sema] Diagnose invalid StringInterpolationProtocols

StringInterpolationProtocol informally requires conforming types to provide at least one method with the base name “appendInterpolation” with no (or a discardable) return value and visibility at least as broad as the conforming type’s. This change diagnoses an error when a conforming type does not have a method that meets those criteria.

* [Stdlib] Fix map(String.init) source break

Some users, including some in the source compatibility suite, accidentally used init(stringInterpolationSegment:) by writing code like `map(String.init)`. Now that these intializers have been removed, the remaining initializers often end up tying during overload resolution. This change adds several overloads of `String.init(describing:)` which will break these ties in cases where the compiler previously selected `String.init(stringInterpolationSegment:)`.

* [Sema] Make callWitness() take non-mutable arrays

It doesn’t actually need to mutate them.

* [Stdlib] Improve floating-point interpolation performance

This change avoids constructing a String when interpolating a Float, Double, or Float80. Instead, we write the characters to a fixed-size buffer and then append them directly to the string’s storage.

This seems to improve performance for all three types, but especially for Double and Float80, which cannot always fit into a small string when stringified.

* [NameLookup] Improve MemberLookupTable invalidation

In rare cases usually involving generated code, an overload added by an extension in the middle of a file would not be visible below it if the type had lazy members and the same base name had already been referenced above the extension. This change essentially dirties a type’s member lookup table whenever an extension is added to it, ensuring the entries in it will be updated.

This change also includes some debugging improvements for NameLookup.

* [SILOptimizer] XFAIL dead object removal failure

The DeadObjectRemoval pass in SILOptimizer does not currently remove reworked string interpolations as well as the old design because their effects cannot be described by @_effects(readonly). That causes a test failure on Linux. This change temporarily silences that test. The SILOptimizer issue has been filed as SR-9008.

* Confess string interpolation’s source stability sins

* [Parser] Parse empty interpolations

Previously, the parser had an odd asymmetry which caused the same function to accept foo(), but reject “\()”. This change fixes the issue.

Already tested by test/Parse/try.swift, which uses this construct in one of its throwing interpolation tests.

* [Sema] Fix batch-mode-only lazy var bug

The temporary variable used by string interpolation needs to be recontextualized when it’s inserted into a synthesized getter. Fixes a compilation failure in Alamofire.

I’ll probably follow up on this bug a bit more after merging.
2018-11-02 19:16:03 -07:00
Christopher Ian Stern
469a0f3eb0 fix stack overflow on deeply nested parens [SR-4866] (#19631)
Make sure StructureMarkerRAII checks structure nesting level on all paths.  Previously swift crashed with  no diagnostic on deeply nested '('. Now we print an error when more than 256 parens deep, just as we always have for '['.

fixes SR-4866
2018-10-10 13:42:18 +09:00
Rintaro Ishizaki
143f55a6e5 [Lexer] Add formStringLiteralToken dedicated for forming string literal 2018-09-19 18:58:54 +09:00
Xi Ge
0a03259f4a Parser: disable lexer diagnostics during delayed parsing.
The lexer diagnostics have already been emitted during token skipping.
2018-09-11 16:30:57 -07:00
Xi Ge
b4356d4f93 Merge pull request #19104 from nkcsgexi/parser-laziness
Parser: lazily parse members in extension decls.
2018-09-06 15:24:17 -07:00
John Holdsworth
4da8cbe655 Implement SE-0200 (extended escaping in string literals)
Supports string literals like #"foo"\n"bar"#.
2018-09-06 15:19:52 -07:00
Brent Royal-Gordon
df22ea1bfb Revert "[Parse] Implementation for SE-200 (raw strings)" 2018-09-06 12:22:41 -07:00
swift-ci
192587c98a Merge pull request #17668 from johnno1962a/master 2018-09-06 11:47:36 -07:00
Xi Ge
24b0eac9a4 Parser: parse members in extension decls incrementally. 2018-09-05 17:00:39 -07:00
John Holdsworth
dc96342368 Response to xwu's review 2018-09-02 11:37:02 +01:00
Rintaro Ishizaki
2a1ab7d8e2 Merge pull request #19046 from rintaro/parse-refactor-getsetdecl
[Parse] refactor get/set block parsing
2018-09-01 00:32:57 +09:00
Rintaro Ishizaki
4797a7caf6 Merge pull request #18919 from rintaro/ide-complete-pound
[CodeCompletion] Implement completion for # directives
2018-08-31 10:50:40 +09:00
Rintaro Ishizaki
51b7168d05 [Parse] Refactor get/set accessor parsing
* Use 'parseAbstractFunctionBody()' for accessors as well. This
  simplifies the implementation, and makes 'parseAbstractFunctionBody()'
  the single point of parsing body of every 'AbstructFunctionDecl' types.
2018-08-29 18:01:21 +09:00
Alex Hoppen
66374a14ea [libSyntax] Make RawSyntax nodes hold a strong reference to their arena
This allows an elegant design in which we can still allocate RawSyntax
nodes using a bump allocator but are able to automatically free that
buffer once the last RawSyntax node within that buffer is freed.

This also resolves a memory leak of RawSyntax nodes that was caused by
ParserUnit not freeing its underlying ASTContext.
2018-08-24 08:39:54 -07:00
Rintaro Ishizaki
5ef5f5ed84 [CodeCompletion] Implement completion for # directives
rdar://problem/29976235
2018-08-24 12:24:54 +09:00
John Holdsworth
7866093ea5 Extend token boundary to include delimiter 2018-08-17 02:12:01 +01:00
Alex Hoppen
b9676772a7 [libSyntax] Partly revert #18698 to fix ASAN bot 2018-08-15 09:30:37 -07:00
Alex Hoppen
093f9179b0 Merge pull request #18698 from ahoppen/leak-fix
[libSyntax] Resolve a memory leak leaking the syntax tree
2018-08-14 14:11:37 -07:00
Alex Hoppen
05ae109478 [Parser] Delete ASTContext of ParserUnit
This resolves a memory leak.
2018-08-14 07:42:51 -07:00
Alex Hoppen
ac512d4341 [libSyntax] Add a reference counted version of OwnedString
We cannot use unowned strings for token texts of incrementally parsed
syntax trees since the source buffer to which reused nodes refer will
have been freed for reused nodes. Always copying the token text whenever
OwnedString is passed is too expensive. A reference counted copy of the
string allows us to keep the token's string alive across incremental
parses while eliminating unnecessary copies.
2018-08-13 15:37:53 -07:00
Huon Wilson
507d3625bc [Parse] Factor out "find instance of token on current line" logic. NFC. 2018-08-09 16:37:32 +10:00
Jordan Rose
fc9ea1e329 Add Lexer::IsHashbangAllowed, drop SourceManager::getHashbangBufferID (#18534)
Having this be a single buffer hardcoded in the SourceManager and set
by all clients is silly. SourceFiles with the 'Main' kind are allowed
to have hashbang lines (`#!`), other files are not. And anyone
manually setting up a Lexer can decide for themselves.

No intended behavioral change.
2018-08-07 08:25:05 -07:00
Rintaro Ishizaki
1aacb8fefb Merge pull request #17788 from rintaro/parse-identifier-drop3
[Parse] Drop Swift3 support for '$', 'throws', and 'rethrows' as identifier
2018-07-25 19:36:39 +09:00
Alex Hoppen
fcc5a6b424 [libSyntax] Enable incremental parsing for syntax tree based syntax coloring 2018-07-17 14:17:58 -07:00
Rintaro Ishizaki
abc44ef44c [Parse] Drop Swift3 support for 'throws', 'rethrows' as identifier
Swift3 used to parse 'throws' and 'rethrows' as identififers.

Related a1760161c5
2018-07-06 17:59:23 +09:00
John Holdsworth
14213b84bd Revised implementation for raw strings 2018-07-02 12:54:06 +01:00
Robert Widmann
dfe42d2e55 Revert "Reject bad string interpolations (#17074)"
This reverts commit fc23f3404d.
2018-06-13 15:06:33 -07:00
Brent Royal-Gordon
fc23f3404d Reject bad string interpolations (#17074)
* Reject bad string interpolations

String interpolations with multiple comma-separate expressions or argument labels were being incorrectly accepted.

* Tweak error name to match message

* Diagnose empty interpolations more clearly

* Don’t double-diagnose parse errors

Fixes a test at Parse/recovery.swift:799 which the previous commit broke.

* Fix incorrect test RUN: line

A previous version of this test used FileCheck instead of -verify, and the run line wasn’t properly corrected to use -verify.

* Update comment

* Add more argument label tests

Ensures that we don’t get different results from an initializer that doesn’t exist or doesn’t take a String.

* Resolve the SR-7958 crasher test

We now diagnose the error and remove the label before it has an opportunity to crash.
2018-06-12 18:46:52 -07:00
Alex Hoppen
8c9e2e07ec [incParse] Make the SytnaxParsingCache operate on the leading trivia's start 2018-05-22 08:52:35 -07:00
Alex Hoppen
de9737c946 [incrParse] Support incremental parsing for edited files 2018-05-22 08:52:33 -07:00
Slava Pestov
8bf90f21d4 AST: Remove unneeded isDefinition and needsCapture methods from ValueDecl 2018-05-13 22:42:47 -07:00
David Zarzycki
f0c106c0e3 [AST] NFC: Avoid pointer indirection with ASTContext Impl storage
Most of this change is mechanical: `Impl` -> `getImpl()`.
2018-05-10 05:50:56 -04:00
Huon Wilson
0de6c9ee6b [Parse] std::function -> llvm::function_ref for some non-escaping params. 2018-05-01 08:29:06 +10:00