This is meant to support parallel CI runs on the same Linux bot, so that
they don't share the module cache and reduce (hopefully) the likelihood
of issues related to invalid signatures.
When the option is enabled, the environment variable `XDG_CACHE_HOME`
is explicitly set in `build-script` and should be inherited by all the
child processes.
Currently we don't check the operating system before setting the
variable (since it should be a noop for other OSes).
Addresses rdar://73887745
Having the test directory match the module we are testing means we can have scripts in the top level of utils/build_swift which can also have tests. As part of this re-structure the test utilties have been simplified somewhat and all tests no longer use a custom TestCase, rather the standard one exposed by the unittest module.
This commit is not meant to completely test update-checkout, but rather create
some scaffolding for testing update-checkout so we can create starter bugs to
fill out the rest of the functionality. Once we have enough testing in place, we
can start refactoring/simplifying update-checkout.
Design
------
This is just a standard python unittest test suite except that the tests expect
an environment variable (UPDATECHECKOUT_TEST_WORKSPACE_DIR) to be set that
specifies the directory that the unittests have for creating mock git repos
during setup/teardown. lit invokes the test by calling the unittests with the
environment variable set to the appropriate temporary directory.
In this temporary directory, each test creates a pristine set of "fake" remote
repos and a test-config.json file that can be passed to update-checkout to work
with these "fake" remote repos. This allows each test that we write to test
various update-checkout functionalities against a pristime set of git repos. I
choose the git clone test, just b/c it was really simple.
NOTE: One can also run the tests locally using the script
test_update_checkout.sh that uses /tmp/workspace as the workspace directory.
This makes update_checkout into a true python module that we can add nosetests
for, readme, and also split up without issue into multiple small files.
I also added a small stub test directory and support for running tests for
update_checkout during validation-testing. So once we add some tests, everything
is setup correctly in terms of the swift build for testing purposes.
Moved `lit` test that runs unit tests for benchmark scripts from `validation-test/Python` to `test/benchmark` directory. Run the tests for benchmark infrustructure with single lit invocation:
````
swift-source$ ./llvm/utils/lit/lit.py -sv ${SWIFT_BUILD_DIR}/test-macosx-x86_64/benchmark
````
Documented the invocation of benchmark infrastructure tests in README.md
* Implemented a wrapper module around the standard argparse package, exposing the same interface with some extras on top, including a new builder type with expressive DSL for constructing complex argument parsers.
* Fixed imports in build_swift/argparse/__init__.py to make flake8 happy.
* More re-formmating to meet the exacting standards of the python_lint script.
* Added doc-strings to all the modules in the build_swift argparse overlay.
* Implemented a new BoolType for the argparse module which handles boolean-like values and replaces the hard-coded boolean values in the _ToggleAction class.
* Fixed the mess of imports in the tests sub-package to favor relative imports, so now the unit-tests will actually run as expected. The README has also been updated with a better command for executing the unit-test suite.
* Updated the add_positional method on the ArgumentParser builder class to only take a single action or default to the store action.
* Cleaned up the set_defaults method.
* Added validation test to run the build_swift unit-tests.
* Updated validation-test for the build_swift unit-test suite to use %utils.
* Fixed hard-coded default values in the expected_options module used for generating argument parser tests.
* Updated the comment in the Python validation test to run the build_swift unit-tests.
compare_perf_test.py is now covered with unit tests and public methods are documented in the implementation.
Minor refactoring to better conform to Python conventions:
* classes declared in new style
* proper private method prefix of single underscore
* replacing map with list comprehension where it was clearer
Unit test are executed as part of validation-test.
.gitignore was modified to ignore .coverage and htmlcov artifacts generated by the coverage.py package
The bug reducer makes assumptions about the layout of the swift build directory
that may not be true in non-Ninja builds. Two examples of cmake generators that
would cause problems are VisualStudio and Xcode. I do not have time right now to
add such support, hence the disabling of the test.
rdar://30311943
Currently it supports random bug finding and opt pipeline reduction. At some
point in the future it will support:
1. Reducing of optimizer crasher test cases.
2. Reducing optimizer miscompile test cases.
But those are for another time, this is just a first step. Patches welcome = ).
Check out llvm's bugpoint for inspiration.
It also has a basic test of pass pipeline reduction that is triggered via the
validation testing in lit. One thing to note here is that the tool right now
assumes darwin. This can be fixed in the future.
`utils/line-directive` is used when building Swift (see
`cmake/modules/AddSwift.cmake`), and includes some doctests. In order
to make sure it continues to behave as intended, add its tests to
`validation-test`.
https://bugs.swift.org/browse/SR-237 calls for `build-script` and
`build-script-impl` to be merged. This commit takes another step towards
that goal by moving the logic that finds the path to the `clang` and
`clang++` executables up into Python-land.
Rather than simply moving all of the logic into `utils/build-script`,
this commit moves relevant functions into a new Python module, named
`swift_build_support`. This has several benefits:
- The logic can be tested. Whereas `build-script-impl` needed to be run
in order to verify its behavior, the logic extracted out of it into
`swift_build_support` can be tested in isolation.
- The logic can be split up into several files without polluting the
`utils` directory, which now contains many different files that are
unrelated to `build-script`.
