The manipulation of host-test and skip-android-host was a little
different than the equivalent skip-ios-host and similar variables. These
changes make them closer and allows executing only the compiler tests,
but skip the test that need an Android device to run.
- Disables the upload command of the tests if the subset is the
non-executable tests. The non-executable test do not need to be
uploaded, and in the case of Android, a device doesn't need to be
connected, so trying to connect to one will fail.
- Fix a problem where the swift_interpreter feature was removed without
first checking if it was really added.
- Only enable the host tests (the compiler tests) in the Android CI
preset (there's no device attached to that server, but currently only
the Linux tests were being executed, which doesn't make a lot of
sense).
- Move the decision about which platform support device/host tests into
the platform themselves, which allows Android to have device/host
tests. Also modify a little bit the logic around enabling/disabling
the test suite to allow running only the host tests of a platform.
- Fix the suffix name for the target of non-executable tests in a couple
of places.
Stress tests are, by definition, stressful. They intentionally burn a
lot of resources by using randomness to hopefully surface state machine
bugs. Additionally, many stress tests are multi-threaded these days and
they may attempt to use all of the available CPUs to better uncover
bugs. In isolation, this is not a problem, but the test suite as a whole
assumes that individual tests are single threaded and therefore running
multiple stress tests at once can quickly spiral out of control.
This change formalizes stress tests and then treats them like long
tests, i.e. tested via 'check-swift-all' and otherwise opt-in.
Finally, with this change, the CI build bots might need to change if
they are still only testing 'validation' instead of all of the tests.
I see three options:
1) Run all of the tests. -- There are very few long tests left these
days, and the additional costs seems small relative to the cost of
the whole validation test suite before this change.
2) Continue checking 'validation', now sans stress tests.
3) Check 'validation', *then* the stress tests. If the former doesn't
pass, then there is no point in the latter, and by running the stress
tests separately, they stand a better chance of uncovering bugs and
not overwhelming build bot resources.
