`libInternalSwiftSyntaxParser.dylib` currently doesn’t link against `SwiftExperimentalStringProcessing`, so it can’t use the regex lexing functions defined within. This caused SwiftSyntax to fail if the source code contained regex literals.
Implement a fallback regex lexing function in C++ and use it for SwiftSyntax parsing.
rdar://93580240
Co-authored-by: Rintaro Ishizaki <rishizaki@apple.com>
value types.
This change simply calls into Parser::startsParameterName, which already
supports parsing 'any' and 'some', instead of repeating the code without
checking for contextual keywords.
Previously we would only check for a starting
character of `)` when performing a tentative
lex of a regex literal. Expand this to cover the
entire range of the regex literal, ensuring to
take escapes and custom character classes into
account.
Treat a prefix operator containing `/` the same as
the unapplied infix operator case, where we
tentatively lex. This means that we bail if there
is no closing `/` or the starting character is
invalid. This leaves binary operator containing
`/` in expression position as the last place where
we know that we definitely have a regex literal.
This is unfortunately needed to ensure we correctly
re-lex regex literal tokens correctly, which is
needed for diagnostic logic to correctly compute
source ranges.
rdar://92469692
The code completion might occur inside an attriubte that isn’t part of the AST because it’s missing a `VarDecl` that it could be attached to. In these cases, record the `CustomAttr` and type check it standalone, pretending it was part of a `DeclContext`.
This also fixes a few issues where code completion previously wouldn’t find the attribute constructor call and thus wasn’t providing code completion inside the property wrapper.
rdar://92842803
If the first type has a code completion token, don't record a same type constraint because otherwise if we have
```swift
K.#^COMPLETE^# == Foo
```
we parse this as
```
K == Foo
```
and thus simplify `K` to `Foo`. But we didn't want to state that `K` is `Foo` but that `K` has a member of type `Foo`.
rdar://77458518
Teach the lexer not to consider `/` an operator
character when attempting to re-lex a regex
literal. This allows us to split off a prefix
operator.
Previously this was done after-the-fact in the
parser, but that didn't cover the unapplied infix
operator case, and didn't form a `tok::amp_prefix`
for `foo(&/.../)`, which led to a suboptimal
diagnostic.
This also now means we'll split an operator for
cases such as `foo(!/^/)` rather than treating it
as an unapplied infix operator.
rdar://92469917
The `SWIFT_COMPILER_VERSION` define is used to stamp a vendor’s version number into a Swift compiler binary. It can be queried from Swift code using `#if _compiler_version` and from Clang by using a preprocessor definition called `__SWIFT_COMPILER_VERSION`. These are unsupported compiler-internal features used primarily by Apple Swift.
In Swift 1.0 through 5.5, Apple Swift used a scheme for `SWIFT_COMPILER_VERSION` where the major version matched the embedded clang (e.g. 1300 for Apple Clang 13.0.0) and the minor version was ignored. Starting in Swift 5.6, Apple Swift started using major and minor version numbers that matched the Swift.org version number. This makes them easier to understand, but it means that version 1300.0.x was followed by version 5.6.x. Not only did version numbers go backwards, but also the old logic to ignore minor versions was now a liability, because it meant you would not be able to target a change to 5.7.x compilers but not 5.6.x compilers.
This commit addresses the problem by:
* Modifying the existing `#if _compiler_version(string-literal)` feature so it transforms the major version into a major and minor that will compare correctly to new version numbers. For instance, “1300.*” is transformed into “1.300”, which will compare correctly to a “5.6” or “5.7” version even if it doesn’t really capture the fact that “1300” was a Swift 5.5 compiler. As a bonus, this allows you to use the feature to backwards-compatibly test new compilers using the existing feature: “5007.*” will be seen by compilers before 5.7 as an unknown future version, but will be seen by 5.7 compilers as targeting them.
* Modifying the `__SWIFT_COMPILER_VERSION` clang define similarly so that, to preprocessor conditions written for the old scheme, a 5.7 compiler will appear to have major version 5007.
* Adding a new variant of `#if _compiler_version` with the same syntax as `#if swift` and `#if compiler`—that is, taking a comparison operator and a bare set of dotted version numbers, rather than a string literal. Going forward, this will be how version checks are written once compatibility with compilers before this change is no longer a concern.
These changes are only lightly tested because tests have to work without any compiler version defined (the default in most configurations), but I’ve tested what I can.
Fixes rdar://89841295.
This fixes:
* An issue where the diagnostic messages were leaked
* Diagnose at correct position inside the regex literal
To do this:
* Introduce 'Parse' SwiftCompiler module that is a bridging layer
between '_CompilerRegexParser' and C++ libParse
* Move libswiftParseRegexLiteral and libswiftLexRegexLiteral to 'Parse'
Also this change makes 'SwiftCompilerSources/Package.swift' be configured
by CMake so it can actually be built with 'swift-build'.
rdar://92187284
Queue up diagnostics when lexing, waiting until
`Lexer::lex` is called before emitting them. This
allows us to re-lex without having to deal with
previously invalid tokens.
When recovering from a parser error in an expression, we resumed parsing at a '{'. I assume this was because we wanted to continue inside e.g. an if-body if parsing the condition failed, but it's actually causing more issue because when parsing e.g.
```swift
expr + has - error +
functionTakesClosure {
}
```
we continue parsing at the `{` of the trailing closure, which is a completely garbage location to continue parsing.
The motivating example for this change was (in a result builder)
```swift
Text("\(island.#^COMPLETE^#)")
takeTrailingClosure {}
```
Here `Text(…)` has an error (because it contains a code completion token) and thus we skip `takeTrailingClosure`, effectively parsing
```swift
Text(….) {}
```
which the type checker wasn’t very happy with and thus refused to provide code completion. With this change, we completely drop `takeTrailingClosure {}`. The type checker is a lot happier with that.
This patch improves the error message emitted when the capture list
contains an item that is a sub-field of a struct/class/etc....
If the closure capture did not include `weak` at the beginning, the
presence of a period would cause the if-chain to fall through the
identifier checking, resulting in an error message about expecting a
`weak` keyword. Instead, I've opted to accept the period at that stage
of parsing so that we can fall through to a better error message.
For the following code
```
{ [self.field] in ... }
```
instead of emitting
`expected 'weak', 'unowned', or no specifier in capture list`,
we now emit
`fields may only be captured by assigning to a specific name`
with a fix-it that changes the code to
```
{ [ field = self.field ] in ... }
```
As the _MatchingEngine module no longer contains the matching engine, this patch renames this module to describe its role more accurately. Because this module primarily contains the AST and the regex parsing logic, I propose we rename it to "_RegexParser".
Also renames the ExperimentalRegex module in SwiftCompilerSources to _RegexParser for consistency. This would prevent errors if sources in _RegexParser used qualified lookup with the module name.
