We only want to path normalize a string (and replace backslashes with
forward slashes if on Windows) if it's actually a path. But checking if
there's a path with that name doesn't work if the path is a substring of
an actual path. Additionally check whether the dirname points to
something on the file system. If it does, it's likely meant to be
interpreted as a path.
This adds sanitizing of the expansion of the `%t` temp dir to
PathSanitizingFileCheck. Because the expansion of this path is different
for each test case, lit.cfg cannot use the expanded version. Instead it
relies on lit expanding the `%t` substring. This requires path
normalization to occur in PathSanitizingFileCheck instead of lit.cfg.
All strings matching the expansion of `%t` are now replaced with
`TMP_DIR`. This is especially useful when source files are created in
`%t` using `split-file`.
Also sorts the patterns to sanitize based on length, to guarantee that
patterns that are substrings of some other pattern are always tested
after the longer patterns has already been tested. Otherwise the longer
pattern may never match.
The backtracing code will warn you if you attempt to forcibly enable
backtracing for a privileged executable. This is apparently upsetting
the Driver/filelists.swift test.
Since we want to force it on for tests, so that we will definitely get
backtraces, add an option to suppress warning messages, and turn that
on for tests as well.
rdar://144497613
The `__future__` we relied on is now, where the 3 specific things are
all included [since Python 3.0](https://docs.python.org/3/library/__future__.html):
* absolute_import
* print_function
* unicode_literals
* division
These import statements are no-ops and are no longer necessary.
The regular expression engine escaped the strings differently across
python 2 and 3. Using a raw string makes this simpler to understand
and obsoletes the comment. This change also now properly allows the
replacement to occur in the same way on 2 and 3.
Adjust the regex match to do a better job of sanitizing the paths. This
improves the test coverage pass rate with Python 3. Furthermore, handle
the unicode conversion properly that breaks with Python 3. This further
improves the Python 3 test coverage pass rate.
Python 3 uses the concepts of text and (binary) data instead of Unicode
strings and 8-bit strings. This change makes it explicit in Python 2 and
3 that this utility works with binary data instead of strings.
* print(bytes(r'(/|\\\\|\\\\\\\\)', encoding="ascii")) => b'(/|\\\\\\\\|\\\\\\\\\\\\\\\\)'
* print(bytes(r'(/|\\\\)', encoding="ascii")) => b'(/|\\\\\\\\)'
* print(bytes(r'/', encoding="ascii")) => b'/'
* print(bytes(r'\\/', encoding="ascii")) => b'\\\\/'
When absolute paths that need to be substituted by PathSanitizingFileCheck contained characters that could be part of regexp syntax, substitution didn't actually take place.
For example, substitution like BUILD_DIR='/Users/johnsmith/swift-source/Ninja-RelWithDebInfoAssert+swift-DebugAssert' didn't result in anything because of the '+' in the path.
This is an attempt to fix it.
This commit also adds the --dry-run flag to PathSanitizingFileCheck which was used to detect and debug this issue, but why not let it stay.
Admit defeat and realize that one simply cannot win against Windows. Use
the pattern with both separators in the tests and remove the hack in
PathSanitizingFileCheck because it created false positives, some of them
could not have been easily fixed.
Sadly lit substitutions or other tricks in lit.cfg will not work,
because CHECK lines are not processed by lit, but by FileCheck.
There were several Unix specific things in the ParseableInterface tests:
the Bash subcommand was used, a Python tool was invoked without an
explicit interpreter, and Unix path separators are used in CHECK lines.
The Bash subcommand is replaced by a Lit substitution. Both swiftmodule
and swiftdoc had substitutions, so swiftinterface gets one which should
about the subcommand.
The Python tool is invoked with an explicit interpreter.
The Unix paths, instead of adding the ugly {{\\|/}} pattern everywhere
are fixed in the PathSanitizing tool instead, that gets a new capability
in Windows compatibility mode where all the Windows path found in the
input are transformed into Windows path with only forward slashes (Unix
slashes) in the output, which can be checked textually against the CHECK
lines.
In pch-bridging-header-deps test, the PathSanitizingFileCheck is used to
check against YAML escaped paths. This works for Unix paths, since the
slash doesn't need to be escaped, but doesn't work for Windows paths,
because the path separator is escaped, and the replacement done by
PathSanitizingFileCheck only looks for exact matches.
To avoid changes in how Darwin/Linux execute the tests, this commit adds
two options in PathSanitizingFileCheck: Windows compatibility makes the
given paths match both forward and backward slashes; in the additional
YAML compatibility is enabled, escaped backward slashes will also be
matched.
The Windows compatibility is enabled for all the test in case the tests
are running on Windows (change done in lit.cfg), while the YAML
compatibility is only enabled for those tests that might need it (like
the PCH bridging header one). This is in order to not allow possible
empty path components in tests that do not deal with YAML escaped paths.
'%FileCheck' removes absolute paths of the source and build directory
from the input. Overwhelming majority of tests don't intend to match
these paths.
Also add a substitution '%raw-FileCheck' that does not sanitize the
input.
