Commit Graph

1233 Commits

Author SHA1 Message Date
Brent Royal-Gordon
c66a8be4eb Revert "[Parser] Decouple the parser from AST creation (part 1)" 2019-07-09 14:07:43 -07:00
Jan Svoboda
5aee188716 Rename SyntaxTransformer to ASTGen 2019-07-08 17:16:35 +02:00
Jan Svoboda
44d7769238 [Parser] Decouple the parser from AST creation (part 1)
Instead of creating the AST directly in the parser (and libSyntax or
SwiftSyntax via SyntaxParsingContext), make Parser to explicitly create
a tree of ParsedSyntaxNodes. Their OpaqueSyntaxNodes can be either
libSyntax or SwiftSyntax. If AST is needed, it can be generated from the
libSyntax tree.
2019-06-28 14:28:19 +02:00
Rintaro Ishizaki
302052cd22 [Parse] 'super' requires following l-square 2019-06-27 16:50:29 -07:00
Rintaro Ishizaki
257b61744e [CodeCompletion] Eliminate super related completion
In parser, 'parseExprPostfixSuffix()' can parse postfix expression for
'super'. 'parseExprSuper()' doesn't need to parse them.

In code-completion, 'completeExprSuper()' and 'completeExprSuperDot()'
can be consolidated to 'completePostfixExpr()' and 'completeDotExpr()'.
2019-06-26 09:59:47 -07:00
Xi Ge
2509f50e45 Syntax: introduce a token kind for single quote to preserve round-trip printing of syntax tree
For invalid code, lexer is forgiving enough to allow single quote '\'' as the starting point
of string literals. Later, parser assumes the string literals are always using double quote "\"", and passes
such knowledge to SwiftSyntax side, leading to the round-trip failure we observed in the radar.
This commit fixes the issue by introducing another token kind for single quote.

rdar://51071021
2019-06-12 17:16:16 -07:00
mishal_shah
1e38fc3030 Update master to build with Xcode 11 beta, macOS 10.15, iOS 13, tvOS 13, and watchOS 6 SDKs 2019-06-03 22:50:02 -07:00
David Ungar
92fd649b92 Use CloseQuoteBegin 2019-05-20 10:41:47 -07:00
David Ungar
7a8b3c8383 Add trailing quote location to interpolated string literal. 2019-05-19 15:26:34 -07:00
Rintaro Ishizaki
1b688d5e21 [CodeCompletion] Implement call signature completion for subscript
rdar://problem/28874899
2019-05-17 14:20:28 -07:00
Rintaro Ishizaki
1e711850bb Merge pull request #24805 from rintaro/ide-completion-unresolvedarg-rdar50696432
[CodeCompletion] Call argument completion for implicit member expression
2019-05-15 18:49:14 -07:00
Rintaro Ishizaki
5b5d342995 [CodeCompletion] Enable call signature completion for unresolved member
rdar://problem/50696432
2019-05-15 17:31:19 -07:00
Jan Svoboda
b0f6168660 [libSyntax] Represent raw string delimiters 2019-05-15 09:15:03 +02:00
Brent Royal-Gordon
82928fd821 Merge pull request #24464 from brentdax/charmed-interpolations
[Parse][Sema] Improve interpolation parsing and construction
2019-05-14 17:49:46 -07:00
kitasuke
e09a34129a Consume toke for string literal expr in a better way
Fix incorrect indent
2019-05-14 08:52:42 +09:00
kitasuke
b3a4ab6744 Revert removal of invalid interpolation check 2019-05-14 00:33:28 +09:00
kitasuke
a7d33aa751 Fix to make minimum AST nodes for StringLiteralExpr 2019-05-14 00:32:58 +09:00
kitasuke
9155ed88e0 Use StringLiteralExpr for invalid InterpolatedStringLiteralExpr 2019-05-12 11:23:16 +09:00
kitasuke
ef413e491c Use StringLiteralExpr node if it's a single literal segment 2019-05-12 11:23:08 +09:00
kitasuke
b583fef8e2 Consolidate StringInterpolationExpr to StringLiteralExpr 2019-05-12 11:23:00 +09:00
Brent Royal-Gordon
4f1e05cbeb [AST] Give appendInterpolation refs a SourceLoc
This change permits UnresolvedDotExpr to have both a name and a base that are implicit, but a valid DotLoc, and to treat that DotLoc as the node’s location. It then changes the generation of string interpolation code so that `$stringInterpolation.appendInterpolation` references have a DotLoc corresponding to the backslash in the string literal.

This makes it possible for `ExprContextAnalyzer` in IDE to correctly detect when you are code-completing in a string interpolation and treat it as an `appendInterpolation` call.
2019-05-09 15:29:21 -07:00
Brent Royal-Gordon
da74978a8d [Parse] Parse string interpolations as args
Now that we manipulate the argument list to correct strange interpolations in Sema, we can parse interpolations directly as argument lists, simplifying the parser.

By itself, this refactoring causes a code completion regression; a subsequent commit will fix that.
2019-05-09 15:29:21 -07:00
Brent Royal-Gordon
a4e5bcea6a Move strange interpolation fix to PreCheckExpression
This defers diagnosis until a stage where #if conditions have definitely been evaluated, at the cost of a slightly more complex implementation. We’ll gain some of that complexity back in a subsequent refactoring. Fixes SR-9937.
2019-05-09 15:29:21 -07:00
kitasuke
5debb929da Rename StringInterpolationSegments to StringLiteralSegments 2019-05-06 09:37:37 +08:00
kitasuke
94815fae26 Revert "Revert "SR-10469 Unknown syntax for interpolated string literal""
This reverts commit 18dd11ab21.
2019-04-30 22:35:29 +01:00
Mishal Shah
18dd11ab21 Revert "SR-10469 Unknown syntax for interpolated string literal" 2019-04-29 23:09:58 -07:00
kitasuke
81ed9cc2f9 Merge branch 'master' into SR-10469-func-call-args_list 2019-04-26 06:29:02 +02:00
Doug Gregor
7f293f66b3 [Parser] Allow use of $ declarations in all modes.
Allow the use of declarations whose names start with $ in all
modes. However, normal code cannot define new entities with names that
start with $: only the implementation can do that, e.g., for property
delegates.
2019-04-23 11:31:58 -07:00
Doug Gregor
32b0245187 [Parser] Consistently use consumeIdentifier() for normal identifiers.
consumeIdentifier() provides the general way in which we consume an
identifier token and fill in an Identifier. Use it consistently in the
parser.
2019-04-23 11:31:58 -07:00
kitasuke
abd6dcfece Change expression kind from Expr to FunctionCallArgumentList 2019-04-23 17:54:26 +09:00
Rintaro Ishizaki
82b5f2a195 [CodeCompletion] Find parsed expression from typechecked decl context
Instead of re-typechecking parsed expression, find typechecked
expression that corresponds to the parsed expression from the
typechecked decl context, because the sub expressions of the parsed
expression can be weirdly mutated/replaced by decl context typechecking.

rdar://problem/48141174
2019-02-28 18:39:46 -08:00
Davide Italiano
49b2f4eba7 [DebuggerSupport] Unbreak closures in the expression parser.
The new trick is that of leavign an unresolved identifier for
the expression parser to fill in.

<rdar://problem/47982630>
2019-02-12 16:36:47 -08:00
Argyrios Kyrtzidis
049c3cd7bd Merge pull request #21955 from akyrtzi/syntax-parser-opt
[Parser] Optimize syntactic parsing for libSyntax
2019-01-17 16:09:56 -08:00
Xi Ge
758e4a28c2 SyntaxNodes: subsume DotSelfExpr and ImplicitMemberExpr by MemberAccessExpr.
Per our discussion in another PR, having these three similar nodes at
the same time  may unnecessarily complicate client side.
2019-01-17 14:15:53 -08:00
Argyrios Kyrtzidis
57c1f72cc3 [Parse] Optimize syntax parsing: Make ParsedRawSyntaxNode a POD type
This eliminates the overhead of ParsedRawSyntaxNode needing to do memory management.
If ParsedRawSyntaxNode needs to point to some data the memory is allocated from a bump allocator.

There are also some improvements on how the ParsedSyntaxBuilders work.
2019-01-17 12:17:28 -08:00
Argyrios Kyrtzidis
c7ac859310 [Parse] Optimize syntax parsing: Speed-up Lexer::lexTrivia()
Introduce ParsedTrivia which is a more efficient structure to use during lexing than syntax::Trivia.
2019-01-17 12:10:27 -08:00
Argyrios Kyrtzidis
9e67e7389f [Parse] Ensure that the tokens formed for interpolation anchors have proper ranges from the input source 2019-01-07 19:56:37 -08:00
Argyrios Kyrtzidis
1f1eab638f [Parse/Syntax] Replace ParsedSyntaxRecorder::record* calls from the parser with ParsedSyntaxRecorder::make*
Doing a "direct ParsedSyntaxRecorder::record[some syntax]" call from the parser is not a good idea due to possibility
of being in a backtracking context when the call is made. Replace them with "ParsedSyntaxRecorder::make[some syntax]"
which will implicitly check for backtracking and create a recorded or deferred node accordingly.
2019-01-07 19:56:36 -08:00
Argyrios Kyrtzidis
ab7427723e [Parse/Syntax] Refactoring to decouple the parser from syntax tree creation
Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code.
This decouples the act of parsing from the act of forming a syntax tree representation.
'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree.
To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two.

This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.
2019-01-07 19:52:59 -08:00
Ankit Aggarwal
45290837b4 Merge pull request #21110 from aciidb0mb3r/swiftpm-manifest-version
Extend @available to support PackageDescription
2019-01-07 12:20:23 -08:00
Rintaro Ishizaki
22652f9e88 [Parse] Eliminate backtracking in collection expression parsing
Parsing collection literal expression used to take exponential time
depending on the nesting level of the first element.

Stop using 'parseList()' because using it complicates libSyntax parsing.

rdar://problem/45221238 / https://bugs.swift.org/browse/SR-9220
rdar://problem/38913395 / https://bugs.swift.org/browse/SR-7283
2018-12-25 11:05:23 +09:00
Ankit Aggarwal
92d09f4e19 Extend @available to support PackageDescription
<rdar://problem/46548531> Extend @available to support PackageDescription

This introduces a new private availability kind "_PackageDescription" to
allow availability testing by an arbitary version that can be passed
using a new command-line flag "-swiftpm-manifest-version". The semantics
are exactly same as Swift version specific availability. In longer term,
it maybe possible to remove this enhancement once there is
a language-level availability support for 3rd party libraries.

Motivation:

Swift packages are configured using a Package.swift manifest file. The
manifest file uses a library called PackageDescription, which contains
various settings that can be configured for a package. The new additions
in the PackageDescription APIs are gated behind a "tools version" that
every manifest must declare. This means, packages don't automatically
get access to the new APIs. They need to update their declared tools
version in order to use the new API. This is basically similar to the
minimum deployment target version we have for our OSes.

This gating is important for allowing packages to maintain backwards
compatibility. SwiftPM currently checks for API usages at runtime in
order to implement this gating. This works reasonably well but can lead
to a poor experience with features like code-completion and module
interface generation in IDEs and editors (that use sourcekit-lsp) as
SwiftPM has no control over these features.
2018-12-08 09:38:40 +05:30
Adrian Prantl
ff63eaea6f Remove \brief commands from doxygen comments.
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.

Patch produced by

      for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
2018-12-04 15:45:04 -08:00
Rintaro Ishizaki
5e69941fda [CodeCompletion] Propagate completion status of parseGenericArguments()
To properly handle code-completion in generic argument list.

rdar://problem/39312854
2018-12-03 12:34:59 +09:00
Arnold Schwaighofer
b102c7f6b4 Parser/Sema/SILGen changes for @_dynamicReplacement(for:)
Dynamic replacements are currently written in extensions as

extension ExtendedType {
  @_dynamicReplacement(for: replacedFun())
  func replacement() { }
}

The runtime implementation allows an implementation in the future where
dynamic replacements are gather in a scope and can be dynamically
enabled and disabled.

For example:

dynamic_extension_scope CollectionOfReplacements {
  extension ExtentedType {
    func replacedFun() {}
  }

  extension ExtentedType2 {
    func replacedFun() {}
  }
}

CollectionOfReplacements.enable()
CollectionOfReplacements.disable()
2018-11-06 09:58:36 -08:00
Brent Royal-Gordon
9bd1a26089 Implementation for SE-0228: Fix ExpressibleByStringInterpolation (#20214)
* [CodeCompletion] Restrict ancestor search to brace

This change allows ExprParentFinder to restrict certain searches for parents to just AST nodes within the nearest surrounding BraceStmt. In the string interpolation rework, BraceStmts can appear in new places in the AST; this keeps code completion from looking at irrelevant context.

NFC in this commit, but keeps code completion from crashing once TapExpr is introduced.

* Remove test relying on ExpressibleByStringInterpolation being deprecated

Since soon enough, it won’t be anymore.

* [AST] Introduce TapExpr

TapExpr allows a block of code to to be inserted between two expressions, accessing and potentially mutating the result of its subexpression before giving it to its parent expression. It’s roughly equivalent to this function:

  func _tap<T>(_ value: T, do body: (inout T) throws -> Void) rethrows -> T {
    var copy = value
    try body(&copy)
    return copy
  }

Except that it doesn’t use a closure, so no variables are captured and no call frame is (even notionally) added.

This commit does not include tests because nothing in it actually uses TapExpr yet. It will be used by string interpolation.

* SE-0228: Fix ExpressibleByStringInterpolation

This is the bulk of the implementation of the string interpolation rework. It includes a redesigned AST node, new parsing logic, new constraints and post-typechecking code generation, and new standard library types and members.

* [Sema] Rip out typeCheckExpressionShallow()

With new string interpolation in place, it is no longer used by anything in the compiler.

* [Sema] Diagnose invalid StringInterpolationProtocols

StringInterpolationProtocol informally requires conforming types to provide at least one method with the base name “appendInterpolation” with no (or a discardable) return value and visibility at least as broad as the conforming type’s. This change diagnoses an error when a conforming type does not have a method that meets those criteria.

* [Stdlib] Fix map(String.init) source break

Some users, including some in the source compatibility suite, accidentally used init(stringInterpolationSegment:) by writing code like `map(String.init)`. Now that these intializers have been removed, the remaining initializers often end up tying during overload resolution. This change adds several overloads of `String.init(describing:)` which will break these ties in cases where the compiler previously selected `String.init(stringInterpolationSegment:)`.

* [Sema] Make callWitness() take non-mutable arrays

It doesn’t actually need to mutate them.

* [Stdlib] Improve floating-point interpolation performance

This change avoids constructing a String when interpolating a Float, Double, or Float80. Instead, we write the characters to a fixed-size buffer and then append them directly to the string’s storage.

This seems to improve performance for all three types, but especially for Double and Float80, which cannot always fit into a small string when stringified.

* [NameLookup] Improve MemberLookupTable invalidation

In rare cases usually involving generated code, an overload added by an extension in the middle of a file would not be visible below it if the type had lazy members and the same base name had already been referenced above the extension. This change essentially dirties a type’s member lookup table whenever an extension is added to it, ensuring the entries in it will be updated.

This change also includes some debugging improvements for NameLookup.

* [SILOptimizer] XFAIL dead object removal failure

The DeadObjectRemoval pass in SILOptimizer does not currently remove reworked string interpolations as well as the old design because their effects cannot be described by @_effects(readonly). That causes a test failure on Linux. This change temporarily silences that test. The SILOptimizer issue has been filed as SR-9008.

* Confess string interpolation’s source stability sins

* [Parser] Parse empty interpolations

Previously, the parser had an odd asymmetry which caused the same function to accept foo(), but reject “\()”. This change fixes the issue.

Already tested by test/Parse/try.swift, which uses this construct in one of its throwing interpolation tests.

* [Sema] Fix batch-mode-only lazy var bug

The temporary variable used by string interpolation needs to be recontextualized when it’s inserted into a synthesized getter. Fixes a compilation failure in Alamofire.

I’ll probably follow up on this bug a bit more after merging.
2018-11-02 19:16:03 -07:00
Ben Langmuir
f8636b462e Merge pull request #20237 from rintaro/ide-codecompletion-inbetween
[CodeCompletion] Don't parse postfix if the primary ends with CC token
2018-11-02 08:53:20 -07:00
Rintaro Ishizaki
14105792ab [CodeCompletion] Don't parse postfix if the primary ends with CC token
For example, given:

  [.foo(), .bar()]

If user want to insert another element in between:

  [.foo(), <HERE> .bar()]

'.bar()' is probably not a part of the inserting element. Moreover, having
the suffix doesn't help type inference in any way.
2018-11-02 17:10:11 +09:00
John McCall
cf511445e2 Basic support for Builtin.IntegerLiteral. 2018-10-31 18:42:34 -04:00
Rintaro Ishizaki
74356d655c [CodeCompletion] Implement context type analysis at subscript position
This improves completion for index part of subscript expression.

rdar://problem/38920581
2018-10-30 10:58:32 +09:00