Commit Graph

492 Commits

Author SHA1 Message Date
Andrew Trick
3560ad2203 Register AnyHashableWithAClass benchmark. 2017-09-22 21:25:46 -07:00
Andrew Trick
e7edd270e5 Register PolymorphicCalls benchmark. 2017-09-22 21:11:30 -07:00
Andrew Trick
a6dcdfda29 Register SevenBoom benchmark. 2017-09-22 21:06:51 -07:00
Andrew Trick
d38e8da6a2 Register the LinkedList benchmark. 2017-09-22 20:51:46 -07:00
Andrew Trick
c0ffc0d451 Register the Exclusivity benchmark. 2017-09-22 20:43:23 -07:00
Joe Shajrawi
ca6ccb4864 Benchmark categorization support 2017-09-22 17:39:04 -07:00
Andrew Trick
78b072b4da Add a CPU microbenchmarks for isUniquelyReferenced. 2017-09-22 16:38:49 -07:00
Andrew Trick
2e60052d1d ObjectAllocation is a .cpubench. 2017-09-22 14:52:47 -07:00
Andrew Trick
d9af4c5f45 Introduce BenchmarkInfo and BenchmarkCategories. 2017-09-22 12:49:59 -07:00
Andrew Trick
aaece4f1f2 Remove benchmark CMakeLists and main.swift from gyb.
When a benchmark is added, it's name will be manually added to CMakeLists and
main.swift.

Adding a benchmark no longer requires discovering a nonobvious script.

If the build works locally, then it will work everywhere else.

Note that generate_harness.py should now be renamed to generate_benchmarks.py,
but I think it's name may be hard-coded somewhere in the compiler tests. Anyway,
this script should go away as soon as someone fixes the cmake configuration to
automatically run gyb.
2017-09-22 12:49:59 -07:00
Erik Eckstein
d6f6774214 benchmarks: add a benchmark for string enum construction from a string raw value. 2017-09-18 17:50:24 -07:00
Dave Abrahams
31341e6753 Merge pull request #10976 from d-ronnqvist/reduce-with-inout
[stdlib] Implement SE-0171: Reduce with inout
2017-07-23 19:57:25 -07:00
David Rönnqvist
bc235afcf8 [stdlib] Update reduce benchmark to compare the two forms
The ReduceInto benchmark performs three tasks using both `reduce(_:_)` and `reduce(into:_:)` so that their performance can be compared:

1. Summing an array, reducing to `Int`
2. Filtering an array, reducing to `[Int]`
3. Counting letter frequencies, reducing to `[Character: Int]`
2017-07-18 17:42:27 +02:00
Erik Eckstein
ffa38bbf84 Revert "Add benchmarks for a number of NSStringAPI functions"
This reverts commit cf73e560ad.

The benchmarks are causing leaks.
Reverted until this is investigated.
2017-07-17 17:38:49 -07:00
David Rönnqvist
e15ea5fcf3 [stdlib] Implement reduce with inout (SE-0171)
Implement and document `reduce(into:_:)`, with a few notes:

 - The `initial` parameter was renamed `initialResult` to match the first parameter in `reduce(_:_:)`.
 - The unnamed `combining` parameter was renamed `updateAccumulatingResult` to try and resemble the naming of the closure parameter in `reduce(_:_:)`.
 - The closure throws and `reduce(into:_)` re-throws.
 - This documentation mentions that `reduce(into:_)` is preferred over `reduce(_:_:)` when the result is a copy-on-write type and an example where the result is a dictionary.

Add benchmarks for reduce with accumulation into a scalar, an array, and a dictionary.

Update expected error message in closures test (since there are now two `reduce` methods, the diagnostic is different).
2017-07-14 23:24:42 +02:00
Philippe Hausler
cf73e560ad Add benchmarks for a number of NSStringAPI functions 2017-07-13 11:33:35 -07:00
Andrew Trick
b76794ce2b Benchmarks for memory exclusivity enforcement.
Three blind microbenchmarks:
- global access
- materializeForSet class property
- nested independent access
2017-06-29 14:31:46 -07:00
Michael Ilseman
bd5189c25a [String] Grapheme fast paths for punctuation: 5-8x speedup.
Many strings use non-sub-300 punctuation characters (e.g. unicode
hyphen, CJK quotes, etc). This can cause switching between fast and
slow paths for grapheme breaking. Add in fast-paths for general
punctuation characters and CJK punctuation and symbol characters.

This results in about a 5-8x speedup for heavily (unicode) punctuated
Latiny and CJKy workloads.
2017-06-27 19:18:51 -07:00
Dave Abrahams
cb4c656bc2 [stdlib] Add benchmarks for Character's unicodeScalars view
In local testing, these benchmark results change as follows due to f212fac717:

<details open>
  <summary>Regression (20)</summary>

TEST                                              | OLD    | NEW    | DELTA   | SPEEDUP
---                                               | ---    | ---    | ---     | ---
CharIteration_utf16_unicodeScalars                | 33074  | 79510  | +140.4% | **0.42x**
CharIteration_utf16_unicodeScalars_Backwards      | 73826  | 115515 | +56.5%  | **0.64x**

</details>

<details open>
  <summary>Improvement (43)</summary>

TEST                                              | OLD    | NEW    | DELTA   | SPEEDUP
---                                               | ---    | ---    | ---     | ---
CharIndexing_korean_unicodeScalars                | 118425 | 10360  | -91.3%  | **11.43x**
CharIndexing_tweet_unicodeScalars_Backwards       | 226366 | 19850  | -91.2%  | **11.40x**
CharIndexing_tweet_unicodeScalars                 | 240596 | 21178  | -91.2%  | **11.36x**
CharIndexing_japanese_unicodeScalars_Backwards    | 136265 | 12032  | -91.2%  | **11.33x**
CharIndexing_chinese_unicodeScalars               | 92226  | 8146   | -91.2%  | **11.32x**
CharIndexing_russian_unicodeScalars               | 100908 | 8948   | -91.1%  | **11.28x**
CharIndexing_japanese_unicodeScalars              | 145414 | 12895  | -91.1%  | **11.28x**
CharIndexing_ascii_unicodeScalars                 | 121438 | 10779  | -91.1%  | **11.27x**
CharIndexing_russian_unicodeScalars_Backwards     | 94190  | 8367   | -91.1%  | **11.26x**
CharIndexing_korean_unicodeScalars_Backwards      | 110175 | 9803   | -91.1%  | **11.24x**
CharIndexing_chinese_unicodeScalars_Backwards     | 86479  | 7715   | -91.1%  | **11.21x**
CharIndexing_ascii_unicodeScalars_Backwards       | 113255 | 10132  | -91.1%  | **11.18x**
CharIteration_ascii_unicodeScalars_Backwards      | 114873 | 13701  | -88.1%  | **8.38x**
CharIteration_chinese_unicodeScalars_Backwards    | 85778  | 10411  | -87.9%  | **8.24x**
CharIteration_russian_unicodeScalars_Backwards    | 94504  | 11471  | -87.9%  | **8.24x**
CharIteration_japanese_unicodeScalars_Backwards   | 136231 | 16569  | -87.8%  | **8.22x**
CharIteration_tweet_unicodeScalars_Backwards      | 222907 | 27165  | -87.8%  | **8.21x**
CharIteration_korean_unicodeScalars_Backwards     | 110132 | 13443  | -87.8%  | **8.19x**
CharIteration_korean_unicodeScalars               | 79540  | 11859  | -85.1%  | **6.71x**
CharIteration_russian_unicodeScalars              | 68209  | 10211  | -85.0%  | **6.68x**
CharIteration_japanese_unicodeScalars             | 98016  | 14690  | -85.0%  | **6.67x**
CharIteration_tweet_unicodeScalars                | 161177 | 24227  | -85.0%  | **6.65x**
CharIteration_chinese_unicodeScalars              | 61702  | 9278   | -85.0%  | **6.65x**
CharIteration_ascii_unicodeScalars                | 81049  | 12218  | -84.9%  | **6.63x**

</details>
2017-06-27 17:25:41 -07:00
Erik Eckstein
fadd2d856f benchmarks: add a getString() utility function.
To be used to prevent constant propagation for benchmark input values.
2017-06-21 17:21:51 -07:00
Erik Eckstein
f45bddcaa6 benchmarks: Convert the GlobalClass benchmark to a SILOptimizer test
The purpose of the benchmark was to check a very specific ARC optimization, which is better tested with a lit test.
Beside that, the benchmark was broken anyway.
2017-06-21 14:32:29 -07:00
Erik Eckstein
f657fc5a53 benchmarks: add a getInt() utility function.
To be used to prevent constant propagation for benchmark input values.
2017-06-19 20:30:20 -07:00
Nate Cook
220614c016 [benchmark] Add benchmark for Dictionary(group:by:) 2017-06-15 11:45:33 -05:00
Pavol Vaskovic
f244b54e68 Fix SR-4780
SR-4780 Can not run performance tests that are not in precommit suite

Modified driver to honor command line arguments when listing enabled tests. Fixed interaction between filters (positional arguments) and --run-all option.

Benchmark_Driver lists available benchmarks with --run-all option when benchmarks or filters are specified.
2017-06-07 21:13:50 +02:00
Ben Cohen
ac18e4297a Benchmarks for equating/comparing substrings (#10030)
* Benchmarks for equating/comparing substrings

* Update main.swift

* Regnerate benchmarks harness file
2017-06-01 14:52:58 -07:00
Michael Ilseman
6ee5745a3a [benchmark] Add Russian StringWalk variant (off by default) 2017-05-31 11:34:42 -07:00
Dave Abrahams
010412ced2 [stdlib] Add Substring comparison benchmarks
This exercise reveals lots of unimplemented cases, which, when implemented, will
cause us to uncomment the disabled code.
2017-05-24 20:56:26 -07:00
Joe Shajrawi
a408803345 Revert "[stdlib] Add Substring comparison benchmarks" 2017-05-24 20:20:55 -07:00
Dave Abrahams
dc184b5e96 [stdlib] Update main.swift for added benchmarks 2017-05-24 19:01:32 -07:00
Pavol Vaskovic
05b977d54b Disable HashQuadratic benchmark
Moving HashQuadratic performance test from “precommit” suite to “other”, so that it does not slow down the testing until we have a fix for SR-3268.
2017-05-18 07:29:37 +02:00
Michael Ilseman
fb5734c24f Merge pull request #9575 from milseman/unihan_fasterhan
[stdlib] String: Walk Chinese/Japanese faster: 2x/4x forwards/backwards
2017-05-14 13:50:15 -07:00
Dave Abrahams
5a0db2f389 [stdlib] Add initial Substring benchmarks 2017-05-12 19:59:41 -07:00
Michael Ilseman
075c6e6a04 [benchmark] Add more (off-by-default) StringWalk benchmarks.
Gyb up StringWalk, to avoid the code explosion. Add in benchmarks for
walking Chinese, Japanese, and Korean text.
2017-05-12 17:34:09 -07:00
eeckstein
087b84e800 Merge pull request #9330 from palimondo/check-results-error-msg
[benchmark] CheckResults with auto-generated error message
2017-05-12 08:47:40 -07:00
Slava Pestov
02dbc0f34f Merge pull request #9193 from palimondo/SR-4572
[benchmark] SR-4572 Remove jinja2 dependency from test harness generation
2017-05-11 15:16:11 -07:00
practicalswift
764ce41c9a [gardening] Fix a vs an typos. 2017-05-09 20:47:27 +02:00
Dave Abrahams
e32fb8badb Remove interpolated strings from benchmark CheckResults
This call was in many cases skewing the benchmark results.

Note: Intentionally staging this in without removing the old overload initially.
2017-05-05 10:25:02 -07:00
Pavol Vaskovic
dcd98cc04f CheckResults with auto-generated error message
In order to minimize impact of results checking on test performance, this removes the @autoclosure for error message.

Added new version of `CheckResults` that takes only `resultsMatch: Bool` - rest of the parameters are defaulted to `StaticString`s for method and file name, plus line number. Old method was deprecated, but left in place as tool for debugging failing checks. All tests were move to use the new method.
2017-05-05 14:08:13 +02:00
Erik Eckstein
fc4b5fa25d benchmarks: add a blackHole function, which can be used to prevent the optimizer to remove a result of a computation.
It’s similar to the _blackHole function in the stdlib unit tests.
2017-05-03 09:05:44 -07:00
Pavol Vaskovic
68fde69535 Applied DRYer. Made python lint happy. 2017-05-02 21:49:59 +02:00
Pavol Vaskovic
bdffb35d17 SR-4572 Remove jinja2 dependency from benchmark test harness generation 2017-05-02 16:20:30 +02:00
Erik Eckstein
994c04f943 benchmarks: add another benchmark for appending multiple array elements.
In contrast to the other similar benchmarks, this benchmark starts with a new array each iteration.
2017-04-26 15:06:05 -07:00
Erik Eckstein
7914e09aa8 benchmarks: Add a benchmark for COW existentials 2017-04-26 14:54:16 -07:00
Michael Ilseman
517ac2a66f [benchmark] Add in some (default off) mixed-string micro benchmarks
Adds in a benchmark to see how quickly we can traverse mixed latin and
emoji strings. Default disabled and part of the stringTests suite.
2017-04-24 14:57:42 -07:00
Arnold Schwaighofer
c9aa05c3c6 [benchmark] Move existential tests out of precommit. They are not very stable. 2017-04-20 13:11:20 -07:00
Michael Ilseman
edf8e6e5e6 [benchmark] Move work into the harness generator
Move the improvements to main.swift into the harness generator and
template itself. Run the generator.
2017-04-19 14:59:35 -07:00
Michael Ilseman
9ffc2f073a [benchmark] Add String test suite (off by default)
Starts a String perf test suite called stringTests which is
off-by-default, but allows us to place some lower-level targeted
benchmarking.
2017-04-19 13:31:34 -07:00
Michael Ilseman
c1ba0bcd6a [benchmark] Speed up compilation time of suite
This speeds up the compilation time of main.swift in the benchmark
suite from taking a minute or two to about a second. NFC.
2017-04-19 13:30:35 -07:00
swift-ci
bf08d01610 Merge pull request #8641 from palimondo/sequence-benchmarks 2017-04-14 00:35:02 -07:00
Pavol Vaskovic
1c9cc3815a Performance tests for Sequence methods: DropFirst, DropWhile, PrefixWhile 2017-04-12 01:15:02 +02:00