ASTGen always builds with the host Swift compiler, without requiring
bootstrapping, and is enabled in more places. Move the regex literal
parsing logic there so it is enabled in more host environments, and
makes use of CMake's Swift support. Enable all of the regex literal
tests when ASTGen is built, to ensure everything is working.
Remove the "AST" and "Parse" Swift modules from SwiftCompilerSources,
because they are no longer needed.
Friend PR: apple/swift-experimental-string-processing#282.
Also remove the `-enable-experimental-pairwise-build-block` flag when building regex modules as the feature is already on by default.
Update the lexing code for the replacement of the
`'/.../'` and `'|...|'` delimiters with `#/.../#`
and `#|...|#` respectively, in addition to
allowing the `re'...'` delimiter.
Applies swift-experimental-string-processing#68 in regex literal type inference. Regex literals with captures will have type `Regex<Tuple{n}<Substring, {Captures...}>>`. This is a temporary thing that allows us to define generic constraints on captures. We will switch back to native tuples once we have variadic generics.
When parsing a regular expression literal, accept a serialized capture structure from the regex parser. During type checking, decode it and form Swift types.
Examples:
```swift
'/(.)(.)/' // ==> `Regex<(Substring, Substring)>`
'/(?<label>.)(.)/' // ==> `Regex<(label: Substring, Substring)`
'/((.))*((.)?)/' //==> `Regex<([Substring], [Substring], Substring, Substring?)>`
```
Also:
- Fix a bug where a regex literal parsing error is not returning an error parser result.
Note:
- This needs to land after apple/swift-experimental-string-processing#92 and after `dev/4` tag has been created.
- See apple/swift-experimental-string-processing#92 for regex parser changes and the capture structure encoding.
- The `RegexLiteralParsingFn` `CaptureStructureOut` pointer type change from `char *` to `void *` will not break builds due to implicit pointer conversion (SE-0324) and unchanged ABI.
Resolves rdar://83253511.
Update the lexing implementation to defer to the
regex library, which will pass back the pointer
from to resume lexing, and update the emission to
call the new `Regex(_regexString:version:)`
overload, that will accept the regex string with
delimiters.
Because this uses the library's lexing
implementation, the delimiters are now `'/.../'`
and `'|...|'` instead of plain `'...'`.
- Checkout apple/swift-experimental-string-processing using a tag.
- Build `_MatchingEngine` as part of libswift (`ExperimentalRegex`) using sources from the package.
- Parse regex literals using the parser from `_MatchingEngine`.
- Build both `_MatchingEngine` and `_StringProcessing` as part of core libs using sources from the package.
- Use `Regex<DynamicCaptures>` as the default regex type until we finalize apple/swift-experimental-string-processing#68.
Rintaro Ishizaki
Richard Wei
Hamish Knight
Richard Wei
Mishal Shah