Commit Graph

10 Commits

Author SHA1 Message Date
David Zarzycki
283713d61d [Testing] Formalize stress tests
Stress tests are, by definition, stressful. They intentionally burn a
lot of resources by using randomness to hopefully surface state machine
bugs. Additionally, many stress tests are multi-threaded these days and
they may attempt to use all of the available CPUs to better uncover
bugs. In isolation, this is not a problem, but the test suite as a whole
assumes that individual tests are single threaded and therefore running
multiple stress tests at once can quickly spiral out of control.

This change formalizes stress tests and then treats them like long
tests, i.e. tested via 'check-swift-all' and otherwise opt-in.

Finally, with this change, the CI build bots might need to change if
they are still only testing 'validation' instead of all of the tests.
I see three options:

1) Run all of the tests. -- There are very few long tests left these
   days, and the additional costs seems small relative to the cost of
   the whole validation test suite before this change.
2) Continue checking 'validation', now sans stress tests.
3) Check 'validation', *then* the stress tests. If the former doesn't
   pass, then there is no point in the latter, and by running the stress
   tests separately, they stand a better chance of uncovering bugs and
   not overwhelming build bot resources.
2018-03-20 21:45:28 -04:00
Mikio Takeuchi
3a46ecb7f0 Use UNSUPPORTED to skip race tests 2017-02-27 12:17:53 +09:00
Mikio Takeuchi
dfeca11403 Disable race tests if nonatomic reference counting is used 2017-02-27 12:17:53 +09:00
Greg Parker
9666d8d39a [test] Add a timeout to runRaceTest(). Use it to limit test AtomicInt.swift. (#4815)
[test] Add a timeout to runRaceTest(). Use it to limit test AtomicInt.swift.

This cuts AtomicInt.swift's execution time from several hours to
about ten minutes on slow hardware and slow build configurations.
2016-09-19 21:46:25 -07:00
Dmitri Gribenko
d175b3b66d Migrate FileCheck to %FileCheck in tests 2016-08-10 23:52:02 -07:00
Robert Widmann
35fe7c7ad6 Cleanup validation tests after SE-0089 2016-07-28 18:01:44 -07:00
Dmitri Gribenko
227cfa1695 StdlibUnittest: add a simple API for race tests 2016-05-20 14:23:31 -07:00
Manav Gabhawala
7928140f79 [SE-0046] Implements consistent function parameter labels by discarding extraneous parameter names and adding _ where necessary 2016-04-06 20:21:58 -04:00
Slava Pestov
49c54870c1 Serialization: Auto-linking recursively walks modules imported from -sil-serialize-all modules 2016-04-01 12:21:36 -07:00
Jordan Rose
40f57149f6 Move StdlibUnittest tests to their own directory.
StdlibUnittest.swift became StdlibUnittest/Common.swift;
all others just dropped the "StdlibUnittest" prefix.
2016-03-18 16:19:30 -07:00