Commit Graph

1638 Commits

Author SHA1 Message Date
Dave Abrahams
31341e6753 Merge pull request #10976 from d-ronnqvist/reduce-with-inout
[stdlib] Implement SE-0171: Reduce with inout
2017-07-23 19:57:25 -07:00
David Rönnqvist
bc235afcf8 [stdlib] Update reduce benchmark to compare the two forms
The ReduceInto benchmark performs three tasks using both `reduce(_:_)` and `reduce(into:_:)` so that their performance can be compared:

1. Summing an array, reducing to `Int`
2. Filtering an array, reducing to `[Int]`
3. Counting letter frequencies, reducing to `[Character: Int]`
2017-07-18 17:42:27 +02:00
Erik Eckstein
ffa38bbf84 Revert "Add benchmarks for a number of NSStringAPI functions"
This reverts commit cf73e560ad.

The benchmarks are causing leaks.
Reverted until this is investigated.
2017-07-17 17:38:49 -07:00
David Rönnqvist
e15ea5fcf3 [stdlib] Implement reduce with inout (SE-0171)
Implement and document `reduce(into:_:)`, with a few notes:

 - The `initial` parameter was renamed `initialResult` to match the first parameter in `reduce(_:_:)`.
 - The unnamed `combining` parameter was renamed `updateAccumulatingResult` to try and resemble the naming of the closure parameter in `reduce(_:_:)`.
 - The closure throws and `reduce(into:_)` re-throws.
 - This documentation mentions that `reduce(into:_)` is preferred over `reduce(_:_:)` when the result is a copy-on-write type and an example where the result is a dictionary.

Add benchmarks for reduce with accumulation into a scalar, an array, and a dictionary.

Update expected error message in closures test (since there are now two `reduce` methods, the diagnostic is different).
2017-07-14 23:24:42 +02:00
Philippe Hausler
cf73e560ad Add benchmarks for a number of NSStringAPI functions 2017-07-13 11:33:35 -07:00
Andrew Trick
b76794ce2b Benchmarks for memory exclusivity enforcement.
Three blind microbenchmarks:
- global access
- materializeForSet class property
- nested independent access
2017-06-29 14:31:46 -07:00
Michael Ilseman
bd5189c25a [String] Grapheme fast paths for punctuation: 5-8x speedup.
Many strings use non-sub-300 punctuation characters (e.g. unicode
hyphen, CJK quotes, etc). This can cause switching between fast and
slow paths for grapheme breaking. Add in fast-paths for general
punctuation characters and CJK punctuation and symbol characters.

This results in about a 5-8x speedup for heavily (unicode) punctuated
Latiny and CJKy workloads.
2017-06-27 19:18:51 -07:00
Dave Abrahams
cb4c656bc2 [stdlib] Add benchmarks for Character's unicodeScalars view
In local testing, these benchmark results change as follows due to f212fac717:

<details open>
  <summary>Regression (20)</summary>

TEST                                              | OLD    | NEW    | DELTA   | SPEEDUP
---                                               | ---    | ---    | ---     | ---
CharIteration_utf16_unicodeScalars                | 33074  | 79510  | +140.4% | **0.42x**
CharIteration_utf16_unicodeScalars_Backwards      | 73826  | 115515 | +56.5%  | **0.64x**

</details>

<details open>
  <summary>Improvement (43)</summary>

TEST                                              | OLD    | NEW    | DELTA   | SPEEDUP
---                                               | ---    | ---    | ---     | ---
CharIndexing_korean_unicodeScalars                | 118425 | 10360  | -91.3%  | **11.43x**
CharIndexing_tweet_unicodeScalars_Backwards       | 226366 | 19850  | -91.2%  | **11.40x**
CharIndexing_tweet_unicodeScalars                 | 240596 | 21178  | -91.2%  | **11.36x**
CharIndexing_japanese_unicodeScalars_Backwards    | 136265 | 12032  | -91.2%  | **11.33x**
CharIndexing_chinese_unicodeScalars               | 92226  | 8146   | -91.2%  | **11.32x**
CharIndexing_russian_unicodeScalars               | 100908 | 8948   | -91.1%  | **11.28x**
CharIndexing_japanese_unicodeScalars              | 145414 | 12895  | -91.1%  | **11.28x**
CharIndexing_ascii_unicodeScalars                 | 121438 | 10779  | -91.1%  | **11.27x**
CharIndexing_russian_unicodeScalars_Backwards     | 94190  | 8367   | -91.1%  | **11.26x**
CharIndexing_korean_unicodeScalars_Backwards      | 110175 | 9803   | -91.1%  | **11.24x**
CharIndexing_chinese_unicodeScalars_Backwards     | 86479  | 7715   | -91.1%  | **11.21x**
CharIndexing_ascii_unicodeScalars_Backwards       | 113255 | 10132  | -91.1%  | **11.18x**
CharIteration_ascii_unicodeScalars_Backwards      | 114873 | 13701  | -88.1%  | **8.38x**
CharIteration_chinese_unicodeScalars_Backwards    | 85778  | 10411  | -87.9%  | **8.24x**
CharIteration_russian_unicodeScalars_Backwards    | 94504  | 11471  | -87.9%  | **8.24x**
CharIteration_japanese_unicodeScalars_Backwards   | 136231 | 16569  | -87.8%  | **8.22x**
CharIteration_tweet_unicodeScalars_Backwards      | 222907 | 27165  | -87.8%  | **8.21x**
CharIteration_korean_unicodeScalars_Backwards     | 110132 | 13443  | -87.8%  | **8.19x**
CharIteration_korean_unicodeScalars               | 79540  | 11859  | -85.1%  | **6.71x**
CharIteration_russian_unicodeScalars              | 68209  | 10211  | -85.0%  | **6.68x**
CharIteration_japanese_unicodeScalars             | 98016  | 14690  | -85.0%  | **6.67x**
CharIteration_tweet_unicodeScalars                | 161177 | 24227  | -85.0%  | **6.65x**
CharIteration_chinese_unicodeScalars              | 61702  | 9278   | -85.0%  | **6.65x**
CharIteration_ascii_unicodeScalars                | 81049  | 12218  | -84.9%  | **6.63x**

</details>
2017-06-27 17:25:41 -07:00
Pavol Vaskovic
9c51a48917 Fix: Run benchmarks just once 2017-06-27 18:47:35 +02:00
Erik Eckstein
2c0b69d241 benchmarks: Make sure there is no constant propagation in the StringBuilder benchmarks.
This fixes the StringAdder benchmark, where the string addition was completely constant folded.
It has only minimal effect on the other benchmarks in this file, because the optimizer didn't constant fold the strings there anyway.
2017-06-21 17:21:51 -07:00
Erik Eckstein
fadd2d856f benchmarks: add a getString() utility function.
To be used to prevent constant propagation for benchmark input values.
2017-06-21 17:21:51 -07:00
swift-ci
42649b36b0 Merge pull request #10468 from eeckstein/convert-benchmark 2017-06-21 17:10:35 -07:00
Erik Eckstein
f45bddcaa6 benchmarks: Convert the GlobalClass benchmark to a SILOptimizer test
The purpose of the benchmark was to check a very specific ARC optimization, which is better tested with a lit test.
Beside that, the benchmark was broken anyway.
2017-06-21 14:32:29 -07:00
Erik Eckstein
e7360897d2 benchmarks: fix ByteSwap benchmark
*) Use getInt to prevent loop hoisting of the benchmark functions
*) Reduce the overhead of CheckResult by moving it out of the loop
*) Prevent inlining of the actual benchmark functions
2017-06-21 13:37:20 -07:00
Erik Eckstein
8abe785300 benchmarks: Fix the BitCount benchmark
*) Increase the iteration count, so that we cat a time value > 100 for better accuracy
*) Use getInt to prevent constant propgation and loop hoisting
2017-06-19 20:30:20 -07:00
Erik Eckstein
f657fc5a53 benchmarks: add a getInt() utility function.
To be used to prevent constant propagation for benchmark input values.
2017-06-19 20:30:20 -07:00
Nate Cook
c37c6f4929 Merge pull request #10306 from natecook1000/nc-grouping-test
[benchmark] Fix the Dictionary(group:by:) test
2017-06-16 22:12:17 -05:00
Nate Cook
ced26b8565 [benchmark] Fix the Dictionary(group:by:) test
When testing quadratic behavior, remember to repeat small amounts of the
behavior several times, not to increase the N, which is squared. 🙄
2017-06-16 18:33:15 -05:00
swift-ci
a4e73cd697 Merge pull request #10265 from lplarson/compare-branch 2017-06-15 10:56:27 -07:00
Nate Cook
220614c016 [benchmark] Add benchmark for Dictionary(group:by:) 2017-06-15 11:45:33 -05:00
Luke Larson
6944c20e13 Benchmark_Driver: Support custom baseline branches
Support specifying a baseline branch to compare the current results
against. Previously, the master branch was hardcoded.

Fixes: rdar://problem/32751587
2017-06-14 15:28:52 -07:00
Pavol Vaskovic
97d6f8dc5e Support for running benchmarks by ordinal number
Add support for running benchmarks by reffering to them by their ordinal number in `Benchmark_Driver`, as is supported by `Benchmark_O`(`Onone`, `Ounchecked`).

Updated documentation to reflect this.
2017-06-12 20:50:00 +02:00
Pavol Vaskovic
f244b54e68 Fix SR-4780
SR-4780 Can not run performance tests that are not in precommit suite

Modified driver to honor command line arguments when listing enabled tests. Fixed interaction between filters (positional arguments) and --run-all option.

Benchmark_Driver lists available benchmarks with --run-all option when benchmarks or filters are specified.
2017-06-07 21:13:50 +02:00
Mishal Shah
8d6385607b Merge pull request #10035 from palimondo/got-you-covered
Added documentation and test coverage.
2017-06-05 12:57:01 -07:00
Pavol Vaskovic
686c761992 One more typo fix. 2017-06-04 18:48:19 +02:00
Pavol Vaskovic
e7b243cad7 Fixed false statement in documentation. 2017-06-04 18:40:20 +02:00
Pavol Vaskovic
dea7d8fe77 Consistent --output; Improved coverage: main()
Coverage at 99% according to coverage.py

* `compare_perf_tests.py` now always outputs the same format to stdout as is written to `--output` file
* Added integration test for the main() function
* Added tests for console output (and suppressed it leaking during testing)
* Fixed file name in test’s file header
2017-06-04 18:31:06 +02:00
Pavol Vaskovic
9265a71ac6 Improved coverage: ReportFormatter
Coverage at 87% according to coveragy.py

Also fixed spelling errors in documentation.
2017-06-02 02:28:44 +02:00
Ben Cohen
ac18e4297a Benchmarks for equating/comparing substrings (#10030)
* Benchmarks for equating/comparing substrings

* Update main.swift

* Regnerate benchmarks harness file
2017-06-01 14:52:58 -07:00
Pavol Vaskovic
d178b6e0cd Improved coverage with more tests: parse_args
Coverage at 66% according to coveragy.py
2017-06-01 22:19:33 +02:00
Pavol Vaskovic
49ddd96c83 Added documentation and test coverage.
compare_perf_test.py is now covered with unit tests and public methods are documented in the implementation.

Minor refactoring  to better conform to Python conventions:
* classes declared in new style
* proper private method prefix of single underscore
* replacing map with list comprehension where it was clearer

Unit test are executed as part of validation-test.

.gitignore was modified to ignore .coverage and htmlcov artifacts generated by the coverage.py package
2017-06-01 20:05:40 +02:00
Michael Ilseman
6ee5745a3a [benchmark] Add Russian StringWalk variant (off by default) 2017-05-31 11:34:42 -07:00
Dave Abrahams
7f9380d098 Remove bogus options from benchmark build
This option is misleading, and never should have been there in the first place.  Fortunately it was having no effect because the standard library is already compiled to SIL and the internal checks have been eliminated from any benchmark build.
2017-05-30 11:18:41 -07:00
Dave Abrahams
010412ced2 [stdlib] Add Substring comparison benchmarks
This exercise reveals lots of unimplemented cases, which, when implemented, will
cause us to uncomment the disabled code.
2017-05-24 20:56:26 -07:00
Joe Shajrawi
a408803345 Revert "[stdlib] Add Substring comparison benchmarks" 2017-05-24 20:20:55 -07:00
Dave Abrahams
dc184b5e96 [stdlib] Update main.swift for added benchmarks 2017-05-24 19:01:32 -07:00
Dave Abrahams
01e0f0aceb [stdlib] Add Substring comparison benchmarks
This reveals lots of unimplemented cases, which will cause us to uncomment the
disabled code.
2017-05-24 16:58:29 -07:00
Pavol Vaskovic
b93a3cab65 Update docs to reflect replacement of jinja by gyb 2017-05-20 20:48:30 +02:00
swift-ci
f49808eb0b Merge pull request #9728 from palimondo/disbable-hash-quadratic-benchmark 2017-05-18 07:41:14 -07:00
Pavol Vaskovic
05b977d54b Disable HashQuadratic benchmark
Moving HashQuadratic performance test from “precommit” suite to “other”, so that it does not slow down the testing until we have a fix for SR-3268.
2017-05-18 07:29:37 +02:00
Max Moiseev
d818008a25 [benchmark] Unbreaking the complex expression in ByteSwap
Fixes: <rdar://problem/31543153>
2017-05-17 15:56:18 -07:00
Pavol Vaskovic
a452b0b12e Fixed filename in the header comment. 2017-05-15 20:04:00 +02:00
Michael Ilseman
fb5734c24f Merge pull request #9575 from milseman/unihan_fasterhan
[stdlib] String: Walk Chinese/Japanese faster: 2x/4x forwards/backwards
2017-05-14 13:50:15 -07:00
Ben Cohen
ea2f64cad2 [stdlib] Add Sequence.Element, change ExpressibleByArrayLiteral.Element to ArrayLiteralElement (#8990)
* Give Sequence a top-level Element, constrain Iterator to match

* Remove many instances of Iterator.

* Fixed various hard-coded tests

* XFAIL a few tests that need further investigation

* Change assoc type for arrayLiteralConvertible

* Mop up remaining "better expressed as a where clause" warnings

* Fix UnicodeDecoders prototype test

* Fix UIntBuffer

* Fix hard-coded Element identifier in CSDiag

* Fix up more tests

* Account for flatMap changes
2017-05-14 06:33:25 -07:00
Dave Abrahams
5a0db2f389 [stdlib] Add initial Substring benchmarks 2017-05-12 19:59:41 -07:00
Michael Ilseman
075c6e6a04 [benchmark] Add more (off-by-default) StringWalk benchmarks.
Gyb up StringWalk, to avoid the code explosion. Add in benchmarks for
walking Chinese, Japanese, and Korean text.
2017-05-12 17:34:09 -07:00
eeckstein
087b84e800 Merge pull request #9330 from palimondo/check-results-error-msg
[benchmark] CheckResults with auto-generated error message
2017-05-12 08:47:40 -07:00
Slava Pestov
02dbc0f34f Merge pull request #9193 from palimondo/SR-4572
[benchmark] SR-4572 Remove jinja2 dependency from test harness generation
2017-05-11 15:16:11 -07:00
practicalswift
659b415462 [gardening] Fix typo. 2017-05-11 16:04:36 +02:00
practicalswift
8c40c65c80 [gardening] Fix typos. 2017-05-09 21:50:04 +02:00