Always parse macro expansions, regardless of language mode, and
eliminate the fallback path for very, very, very old object literals
like `#Color`. Instead, check for the feature flag for macro
declaration and at macro expansion time, since this is a semantic
restriction.
While here, refactor things so the vast majority of the macro-handling
logic still applies even if the Swift Swift parser is disabled. Only
attempts to expand the macro will fail. This allows us to enable the
macro-diagnostics test everywhere.
If `buildBlock` is also unavailable, or the
builder itself is unavailable, continue to solve
using `buildPartialBlock` to get better
diagnostics.
This behavior technically differs from what is
specified in SE-0348, but only affects the invalid
case where no builder methods are available to use.
In particular, this improves diagnostics for
RegexComponentBuilder when the deployment target
is too low. Previously we would try to solve using
`buildBlock` (as `buildPartialBlock` is unavailable),
but RegexComponentBuilder only defines `buildBlock`
for the empty body case, leading to unhelpful
diagnostics that ultimately preferred not to use
the result builder at all.
rdar://97533700
Instead of failing constraint generation by returning `nullptr` for an `ErrorExpr` or returning a null type when a type fails to be resolved, return a fresh type variable. This allows the constraint solver to continue further and produce more meaningful diagnostics.
Most importantly, it allows us to produce a solution where previously constraint generation for a syntactic element had failed, which is required to type check multi-statement closures in result builders inside the constraint system.
For code such as the following:
```
let r = Regex {
/abc/
}
```
If RegexBuilder has not been imported, emit a
specialized diagnostic and fix-it to add
`import RegexBuilder` to the file.
Unfortunately we're currently prevented from
emitting the specialized diagnostic in cases where
the builder contains references to RegexBuilder
types, such as:
```
let r = Regex {
Capture {
/abc/
}
}
```
This is due to the fact that we bail from CSGen
due to the reference to `Capture` being turned
into an `ErrorExpr`. We ought to be able to
handle solving in the presence of such errors, but
for now I'm leaving it as future work.
rdar://93176036
Ban space and tab as the last character of a
`/.../` regex literal, unless escaped with a
backslash. This matches the banning of space and
tab as the first character, and helps avoid breaking
source in even more cases.
While skipping, if we encounter a token that looks
like it could be the start of a `/.../` regex
literal, fall back to parsing the function or type
body normally, as such a token could become a
regex literal. As such, it could treat `{` and
`}` as literal, or otherwise have contents that
would be lexically invalid Swift.
To avoid falling back in too many cases, we apply
the existing regex literal heuristics. Cases that
pass the heuristic fall back to regular parsing.
Cases that fail the heuristic are further checked
to make sure they wouldn't contain an unbalanced
`{` or `}`, but otherwise are allowed to be
skipped. This allows us to continue skipping for
most occurrences of infix and prefix `/`.
This is meant as a lower risk workaround to fix the
the issue, we ought to go back to handling regex
literals in the lexer.
Resolves rdar://95354010
Previously we would only check for a starting
character of `)` when performing a tentative
lex of a regex literal. Expand this to cover the
entire range of the regex literal, ensuring to
take escapes and custom character classes into
account.
Treat a prefix operator containing `/` the same as
the unapplied infix operator case, where we
tentatively lex. This means that we bail if there
is no closing `/` or the starting character is
invalid. This leaves binary operator containing
`/` in expression position as the last place where
we know that we definitely have a regex literal.
This is unfortunately needed to ensure we correctly
re-lex regex literal tokens correctly, which is
needed for diagnostic logic to correctly compute
source ranges.
rdar://92469692
Teach the lexer not to consider `/` an operator
character when attempting to re-lex a regex
literal. This allows us to split off a prefix
operator.
Previously this was done after-the-fact in the
parser, but that didn't cover the unapplied infix
operator case, and didn't form a `tok::amp_prefix`
for `foo(&/.../)`, which led to a suboptimal
diagnostic.
This also now means we'll split an operator for
cases such as `foo(!/^/)` rather than treating it
as an unapplied infix operator.
rdar://92469917
This fixes:
* An issue where the diagnostic messages were leaked
* Diagnose at correct position inside the regex literal
To do this:
* Introduce 'Parse' SwiftCompiler module that is a bridging layer
between '_CompilerRegexParser' and C++ libParse
* Move libswiftParseRegexLiteral and libswiftLexRegexLiteral to 'Parse'
Also this change makes 'SwiftCompilerSources/Package.swift' be configured
by CMake so it can actually be built with 'swift-build'.
rdar://92187284
Because we don't form a type-checked call to the
Regex initializer in the AST, we need to explicitly
handle the availability checking for `Regex<Output>`
and the initializer we're implicitly calling.
rdar://92156542
Friend PR: apple/swift-experimental-string-processing#282.
Also remove the `-enable-experimental-pairwise-build-block` flag when building regex modules as the feature is already on by default.
We add a new flag to disable the implicit import of `_StringProcessing`, similar to `-disable-implicit-concurrency-module-import`. We need this to build `_RegexParser` when `-enable-experimental-string-processing` is enabled by default, because `_StringProcessing` currently imports `_RegexParser` publicly (non-implementation-only).
Instead of returning a parser error, which results
in generic parser recovery that skips until the
next decl, return an `ErrorExpr` so we can
continue parsing.
Update the lexing code for the replacement of the
`'/.../'` and `'|...|'` delimiters with `#/.../#`
and `#|...|#` respectively, in addition to
allowing the `re'...'` delimiter.
Applies swift-experimental-string-processing#68 in regex literal type inference. Regex literals with captures will have type `Regex<Tuple{n}<Substring, {Captures...}>>`. This is a temporary thing that allows us to define generic constraints on captures. We will switch back to native tuples once we have variadic generics.
When parsing a regular expression literal, accept a serialized capture structure from the regex parser. During type checking, decode it and form Swift types.
Examples:
```swift
'/(.)(.)/' // ==> `Regex<(Substring, Substring)>`
'/(?<label>.)(.)/' // ==> `Regex<(label: Substring, Substring)`
'/((.))*((.)?)/' //==> `Regex<([Substring], [Substring], Substring, Substring?)>`
```
Also:
- Fix a bug where a regex literal parsing error is not returning an error parser result.
Note:
- This needs to land after apple/swift-experimental-string-processing#92 and after `dev/4` tag has been created.
- See apple/swift-experimental-string-processing#92 for regex parser changes and the capture structure encoding.
- The `RegexLiteralParsingFn` `CaptureStructureOut` pointer type change from `char *` to `void *` will not break builds due to implicit pointer conversion (SE-0324) and unchanged ABI.
Resolves rdar://83253511.
