When listing benchmarks with `--list` parameter, present the tags in format that is actually accepted by the `--tags` and `--skip-tags` parameters.
Changes the `--list` output from
````
Enabled Tests,Tags
AngryPhonebook,[TestsUtils.BenchmarkCategory.validation, TestsUtils.BenchmarkCategory.api, TestsUtils.BenchmarkCategory.String]
...
````
into
````
Enabled Tests,Tags
AngryPhonebook,[String, api, validation]
…
````
Today, one can not completely disable a benchmark depending on the platform
without changing the source of main.swift. We would like to be able to disable
benchmarks locally in a benchmark's file without needing to modify the rest of
the infrastructure. The closest that one can get to such behavior is to just
conditionally compile out the file locally. But one still will have the test
run.
This commit adds support for not-running the benchmark on specific
platforms. This in combination with conditional compilation of benchmark bodies,
allows us to not have to comment out module's in main.swift or have to
conditionally compile testinfo.
rdar://40541972
Add substring-view-oriented array append benchmarks. Put a
getString/getSubstring call into the innermost loop to prevent some
constant folding towards triviality.
The key thing here is that by providing one of these closures, a benchmark can
inject the initialization/deinitialization of its internal data structures,
outside of the time period where timing is occurring.
The intention is that this will provide us the framework for as we annotate
tests with BenchmarkInfo, to move initialization work out of benchmarks.
It will also allow for more complex benchmarks to be written such as ones that
perform bulk reads from a pipe (my interest in this).
*NOTE* We always prefer a registered benchmark if we have one.
I am going to use BenchmarkInfo to solve the "create data for benchmark while we
are already timing" problem. I am going to add a field to BenchmarkInfo that if
it is not-null is called before we start measuring time. This closure can be
used to initialize any global data structures/etc.
But to do this, I need to be able to combine the registered and legacy
not-registered benchmarks.
In order to minimize impact of results checking on test performance, this removes the @autoclosure for error message.
Added new version of `CheckResults` that takes only `resultsMatch: Bool` - rest of the parameters are defaulted to `StaticString`s for method and file name, plus line number. Old method was deprecated, but left in place as tool for debugging failing checks. All tests were move to use the new method.
