Stress tests are, by definition, stressful. They intentionally burn a
lot of resources by using randomness to hopefully surface state machine
bugs. Additionally, many stress tests are multi-threaded these days and
they may attempt to use all of the available CPUs to better uncover
bugs. In isolation, this is not a problem, but the test suite as a whole
assumes that individual tests are single threaded and therefore running
multiple stress tests at once can quickly spiral out of control.
This change formalizes stress tests and then treats them like long
tests, i.e. tested via 'check-swift-all' and otherwise opt-in.
Finally, with this change, the CI build bots might need to change if
they are still only testing 'validation' instead of all of the tests.
I see three options:
1) Run all of the tests. -- There are very few long tests left these
days, and the additional costs seems small relative to the cost of
the whole validation test suite before this change.
2) Continue checking 'validation', now sans stress tests.
3) Check 'validation', *then* the stress tests. If the former doesn't
pass, then there is no point in the latter, and by running the stress
tests separately, they stand a better chance of uncovering bugs and
not overwhelming build bot resources.
This API is documented in its headers to only allow being called once
for a particular domain, so we have to make sure our check for an
existing provider is synchronized with the setting.
rdar://problem/27541751
Huge thanks to John for noting that 'consume' didn't provide the
guarantees we wanted, and to Michael G. for getting a TSan build up
and running to identify/verify this race.
It's possible that this is overlay strict, and that we only need to
look at the domain to ensure that the code and userInfo are
visible. However, TSan seems to prefix the form in this patch, so
we'll be more conservative for now.
Fixes rdar://problem/27541751.
Yuta Saito
Kuba (Brecka) Mracek
David Zarzycki
Mikio Takeuchi
Jordan Rose
Doug Gregor
Nicole Jacque
Joe Groff
Manav Gabhawala
Slava Pestov
Max Moiseev
Dmitri Gribenko