ASTGen always builds with the host Swift compiler, without requiring
bootstrapping, and is enabled in more places. Move the regex literal
parsing logic there so it is enabled in more host environments, and
makes use of CMake's Swift support. Enable all of the regex literal
tests when ASTGen is built, to ensure everything is working.
Remove the "AST" and "Parse" Swift modules from SwiftCompilerSources,
because they are no longer needed.
Ban space and tab as the last character of a
`/.../` regex literal, unless escaped with a
backslash. This matches the banning of space and
tab as the first character, and helps avoid breaking
source in even more cases.
While skipping, if we encounter a token that looks
like it could be the start of a `/.../` regex
literal, fall back to parsing the function or type
body normally, as such a token could become a
regex literal. As such, it could treat `{` and
`}` as literal, or otherwise have contents that
would be lexically invalid Swift.
To avoid falling back in too many cases, we apply
the existing regex literal heuristics. Cases that
pass the heuristic fall back to regular parsing.
Cases that fail the heuristic are further checked
to make sure they wouldn't contain an unbalanced
`{` or `}`, but otherwise are allowed to be
skipped. This allows us to continue skipping for
most occurrences of infix and prefix `/`.
This is meant as a lower risk workaround to fix the
the issue, we ought to go back to handling regex
literals in the lexer.
Resolves rdar://95354010
