ASTGen always builds with the host Swift compiler, without requiring
bootstrapping, and is enabled in more places. Move the regex literal
parsing logic there so it is enabled in more host environments, and
makes use of CMake's Swift support. Enable all of the regex literal
tests when ASTGen is built, to ensure everything is working.
Remove the "AST" and "Parse" Swift modules from SwiftCompilerSources,
because they are no longer needed.
Because we don't form a type-checked call to the
Regex initializer in the AST, we need to explicitly
handle the availability checking for `Regex<Output>`
and the initializer we're implicitly calling.
rdar://92156542
Update the lexing code for the replacement of the
`'/.../'` and `'|...|'` delimiters with `#/.../#`
and `#|...|#` respectively, in addition to
allowing the `re'...'` delimiter.
Applies swift-experimental-string-processing#68 in regex literal type inference. Regex literals with captures will have type `Regex<Tuple{n}<Substring, {Captures...}>>`. This is a temporary thing that allows us to define generic constraints on captures. We will switch back to native tuples once we have variadic generics.
When parsing a regular expression literal, accept a serialized capture structure from the regex parser. During type checking, decode it and form Swift types.
Examples:
```swift
'/(.)(.)/' // ==> `Regex<(Substring, Substring)>`
'/(?<label>.)(.)/' // ==> `Regex<(label: Substring, Substring)`
'/((.))*((.)?)/' //==> `Regex<([Substring], [Substring], Substring, Substring?)>`
```
Also:
- Fix a bug where a regex literal parsing error is not returning an error parser result.
Note:
- This needs to land after apple/swift-experimental-string-processing#92 and after `dev/4` tag has been created.
- See apple/swift-experimental-string-processing#92 for regex parser changes and the capture structure encoding.
- The `RegexLiteralParsingFn` `CaptureStructureOut` pointer type change from `char *` to `void *` will not break builds due to implicit pointer conversion (SE-0324) and unchanged ABI.
Resolves rdar://83253511.
This reverts commit a67a0436f7, reversing
changes made to 9965df76d0.
This commit or the earlier commit this commit is based on (#40531) broke the
incremental bot.
Update the lexing implementation to defer to the
regex library, which will pass back the pointer
from to resume lexing, and update the emission to
call the new `Regex(_regexString:version:)`
overload, that will accept the regex string with
delimiters.
Because this uses the library's lexing
implementation, the delimiters are now `'/.../'`
and `'|...|'` instead of plain `'...'`.
- Checkout apple/swift-experimental-string-processing using a tag.
- Build `_MatchingEngine` as part of libswift (`ExperimentalRegex`) using sources from the package.
- Parse regex literals using the parser from `_MatchingEngine`.
- Build both `_MatchingEngine` and `_StringProcessing` as part of core libs using sources from the package.
- Use `Regex<DynamicCaptures>` as the default regex type until we finalize apple/swift-experimental-string-processing#68.
- Frontend: Implicitly import `_StringProcessing` when frontend flag `-enable-experimental-string-processing` is set.
- Type checker: Set a regex literal expression's type as `_StringProcessing.Regex<(Substring, DynamicCaptures)>`. `(Substring, DynamicCaptures)` is a temporary `Match` type that will help get us to an end-to-end working system. This will be replaced by actual type inference based a regex's pattern in a follow-up patch (soon).
- SILGen: Lower a regex literal expression to a call to `_StringProcessing.Regex.init(_regexString:)`.
- String processing runtime: Add `Regex`, `DynamicCaptures` (matching actual APIs in apple/swift-experimental-string-processing), and `Regex(_regexString:)`.
Upcoming:
- Build `_MatchingEngine` and `_StringProcessing` modules with sources from apple/swift-experimental-string-processing.
- Replace `DynamicCaptures` with inferred capture types.
