Added support for running benchmarks using substring filters. Positional arguments prefixed with a single + or - sign are interpreted as benchmark name filters.
Excecutes all benchmarks whose names include any of the strings prefixed with a plus sign but none of the strings prefixed with a minus sign.
Support for gathering a minimal number of samples per benchmark, using the optional `--min-samples` argument, which overrides the automatically computed number of samples per `sample-time` if this is lower.
adds a benchmark that imitates SwiftNIO's ChannelPipeline. Recently,
Swift master regressed on this benchmark, therefore I believe adding it
to Swift provides value as it seems to be different enough to the existing
benchmarks.
Generalize CSVParsing so that it can be ran across all of String's
views, and add corresponding benchmarks.
Note: This does change the profile of the original benchmark, which
was an odd mixture of String and UTF16View.
Now one can on Darwin/Linux build the benchmarks via swiftpm from build-script by passing in:
```
build-script $NORMAL_ARGS --install-swift --install-swiftpm --install-llbuild --toolchain-benchmarks --swiftpm --llbuild
```
This is done using the infrastructure that BenL added for sourcekit-lsp.
Lowered the default sample cap from 2k to 200. (This doesn’t effect manually specified `--num-samples` argument in the driver.)
Swift benchmarks have pretty constant performance profile over time. It’s more beneficial to get multiple independent measurements faster, than more samples from the same run.
The use of `--verbose` parameter was affecting the reported memory usage (`--memory`), because it front-loads initialization of string interpolation and printing.
By always computing the configuration string and always calling print, the baseline memory measurement no longer includes this constant overhead.
These should be audited since some might not actually need to be
@inlinable, but for now:
- Anything public and @inline(__always) is now also @inlinable
- Anything @usableFromInline and @inline(__always) is now @inlinable
This adds optional `legacyFactor` to the `BenchmarkInfo`, which allows for linear modification of constants that unnecesarily inflate the base workload of benchmarks, while maintaining the continuity of log-term benchmark tracking.
For example, if a benchmark uses `for _ in N*10_000` in its run function, we could lower this to `for _ in N*1_000` and adding a `legacyFactor: 10` to its `BenchmarkInfo`.
Note that this doesn’t affect the real measurements gathered from the `--verbose` output. The `BenchmarkDoctor` has been slightly adjusted to work with these real samples, therefore `Benchmark_Driver check` will not flag these benchmarks for slow run time reported in the summary, if their real runtimes fall into the recommended range.
