The initial version of the debugger testing transform instruments
assignments in a way that allows the debugger to sanity-check its
expression evaluator.
Given an assignment expression of the form:
```
a = b
```
The transform rewrites the relevant bits of the AST to look like this:
```
{ () -> () in
a = b
checkExpect("a", stringForPrintObject(a))
}()
```
The purpose of the rewrite is to make it easier to exercise the
debugger's expression evaluator in new contexts. This can be automated
by having the debugger set a breakpoint on checkExpect, running `expr
$Varname`, and comparing the result to the expected value generated by
the runtime.
While the initial version of this testing transform only supports
instrumenting assignments, it should be simple to teach it to do more
interesting rewrites.
There's a driver script available in SWIFT_BIN_DIR/lldb-check-expect to
simplfiy the process of launching and testing instrumented programs.
rdar://36032055
When enabled, send a notification before/after every "compilation",
which for now means `performSema`. This piggy-backs and modifies some
existing code that we had for "tracing" operations in sourcekitd that
unfortunately was untested. At least now some of the basic parts are
tested via the new notifications.
Part of rdar://38438512
Stress tests are, by definition, stressful. They intentionally burn a
lot of resources by using randomness to hopefully surface state machine
bugs. Additionally, many stress tests are multi-threaded these days and
they may attempt to use all of the available CPUs to better uncover
bugs. In isolation, this is not a problem, but the test suite as a whole
assumes that individual tests are single threaded and therefore running
multiple stress tests at once can quickly spiral out of control.
This change formalizes stress tests and then treats them like long
tests, i.e. tested via 'check-swift-all' and otherwise opt-in.
Finally, with this change, the CI build bots might need to change if
they are still only testing 'validation' instead of all of the tests.
I see three options:
1) Run all of the tests. -- There are very few long tests left these
days, and the additional costs seems small relative to the cost of
the whole validation test suite before this change.
2) Continue checking 'validation', now sans stress tests.
3) Check 'validation', *then* the stress tests. If the former doesn't
pass, then there is no point in the latter, and by running the stress
tests separately, they stand a better chance of uncovering bugs and
not overwhelming build bot resources.
lldb-dotest does not forward the arguments from DOTEST_EXTRA properly.
For now, just build the lldb-dotest target to ensure the cmake build has
the right dependencies built. Invoke dotest.py manually as before.
Using lldb-dotest to drive testing solves the short-term problem of
there being missing test dependencies when we invoke dotest.py for cmake
builds.
Long term, we really want to delete as much lldb-specific logic from
build-script as possible and replace all of it with a single call to
lldb-dotest.
Even with this first cut, there are some nice simplifications
(build-script no longer needs to know how to find an out-of-tree
debugserver, etc).
* LLDB assertions are on by default, like swift assertions
* LLDB assertions can be enabled/disabled globally with the --assertions
and --no-assertions options
Partially addresses: rdar://38524846
The lldb bots should run the full lldb test suite, not just the tests
reserved for pull-request testing.
This does not affect PR testing.
rdar://38462589
