Commit Graph

1638 Commits

Author SHA1 Message Date
practicalswift
764ce41c9a [gardening] Fix a vs an typos. 2017-05-09 20:47:27 +02:00
practicalswift
49ed8579c4 [gardening] Use American English. 2017-05-09 20:44:30 +02:00
Dave Abrahams
e0c08c7617 Merge branch 'master' into check-results-error-msg 2017-05-05 15:15:12 -07:00
swift-ci
85831df503 Merge pull request #9298 from apple/only-benchmark-interpolation-when-you-mean-to 2017-05-05 14:31:37 -07:00
Dave Abrahams
e32fb8badb Remove interpolated strings from benchmark CheckResults
This call was in many cases skewing the benchmark results.

Note: Intentionally staging this in without removing the old overload initially.
2017-05-05 10:25:02 -07:00
Mishal Shah
a05b35c332 Merge pull request #9026 from palimondo/SR-4598
[benchmark] Support multiple regular expression filters in Benchmark_Driver
2017-05-05 09:46:02 -07:00
Pavol Vaskovic
dcd98cc04f CheckResults with auto-generated error message
In order to minimize impact of results checking on test performance, this removes the @autoclosure for error message.

Added new version of `CheckResults` that takes only `resultsMatch: Bool` - rest of the parameters are defaulted to `StaticString`s for method and file name, plus line number. Old method was deprecated, but left in place as tool for debugging failing checks. All tests were move to use the new method.
2017-05-05 14:08:13 +02:00
Pavol Vaskovic
2f8f11801a Filters are now regullar expression patterns 2017-05-05 08:48:41 +02:00
practicalswift
a9c03845cc [gardening] Use consistent headers 2017-05-04 15:24:35 +02:00
practicalswift
cc6a160d91 [gardening] Remove unused Python property cv 2017-05-04 15:21:45 +02:00
Erik Eckstein
3c4fd12a3f benchmarks: Prevent the optimizer to completely optimize away Character results in the CharacterLiteralsSmall benchmark. 2017-05-03 09:05:45 -07:00
Erik Eckstein
fc4b5fa25d benchmarks: add a blackHole function, which can be used to prevent the optimizer to remove a result of a computation.
It’s similar to the _blackHole function in the stdlib unit tests.
2017-05-03 09:05:44 -07:00
Pavol Vaskovic
68fde69535 Applied DRYer. Made python lint happy. 2017-05-02 21:49:59 +02:00
Pavol Vaskovic
bdffb35d17 SR-4572 Remove jinja2 dependency from benchmark test harness generation 2017-05-02 16:20:30 +02:00
Ted Kremenek
749e05ba13 Merge pull request #8861 from tkremenek/ver-4-bump
Bump Swift version to 4.0
2017-04-28 14:03:55 -07:00
swift-ci
96516f851d Merge pull request #9088 from eeckstein/fix-typo 2017-04-28 07:28:59 -07:00
Ted Kremenek
ef98f2bd2a Merge branch 'master' into ver-4-bump 2017-04-27 23:19:14 -07:00
Erik Eckstein
62be182cac benchmarks: add a comment and use more meaningful numbers 2017-04-27 16:59:42 -07:00
swift-ci
ba67a51c54 Merge pull request #8978 from palimondo/SR-4659 2017-04-27 14:53:26 -07:00
Pavol Vaskovic
d77324782a Fix SR-4659 Benchmark logs should be tied to tested tree version 2017-04-27 23:09:14 +02:00
Maxim Moiseev
41b00c57ab Merge pull request #9044 from moiseev/substring-in
[stdlib] Substring
2017-04-26 19:25:59 -07:00
Max Moiseev
3b607436be Making benchmarks compile 2017-04-26 16:49:34 -07:00
Erik Eckstein
994c04f943 benchmarks: add another benchmark for appending multiple array elements.
In contrast to the other similar benchmarks, this benchmark starts with a new array each iteration.
2017-04-26 15:06:05 -07:00
Erik Eckstein
7914e09aa8 benchmarks: Add a benchmark for COW existentials 2017-04-26 14:54:16 -07:00
swift-ci
0ce898be79 Merge pull request #9036 from aschwaighofer/benchmark_more_autoreleasepools 2017-04-26 13:52:22 -07:00
Arnold Schwaighofer
ca40134c03 [benchmark] Sprinkle more autoreleasepools
SR-4666
2017-04-26 13:21:43 -07:00
Pavol Vaskovic
6d9de4ca70 Multiple filters are specified by repeated -f
Updated usage documentation to reflect this.
2017-04-26 21:12:59 +02:00
Pavol Vaskovic
c719818024 Fix SR-4601 Report Added and Removed Benchmarks in Performance Comparison (#8991)
* Refactor compare_perf_tests.py

* Fix SR-4601 Report Added and Removed Benchmarks in Performance Comparison

Improved HTML styling.

* Added back support for reading concatenated Benchmark_Driver output

PerformanceTestResults can be merged, computing new MIN, MAX, and running MEAN and SD.

* Handle output from Benchmark_O again

Treat MAX_RSS as optional column
2017-04-26 12:09:18 -07:00
swift-ci
1be1f07263 Merge pull request #9031 from aschwaighofer/benchmark_objectivec_bridge_autoreleasepool 2017-04-26 10:53:06 -07:00
Arnold Schwaighofer
a8e3fc9af3 [benchmark] Lower the max memory size of ObejctiveC bridging tests by moving the
autoreleasepool down such that we have less allocations in the pool until it is
empty

rdar://31788785
SR-4666
2017-04-26 10:01:55 -07:00
Arnold Schwaighofer
4338e525bf Merge pull request #9027 from aschwaighofer/benchmark_dictionarybridge_autoreleasepool
[benchmark] Add autorelease pool to fix memory hogging
2017-04-26 09:56:25 -07:00
Arnold Schwaighofer
7ed2122947 [benchmark] Add autorelease pool to fix memory hogging
rdar://31788785
2017-04-26 07:37:07 -07:00
Michael Ilseman
111d6200eb Merge pull request #8979 from milseman/parse_my_tweets_faster
[stdlib] Parse my tweets faster! 2x forwards, 3x reverse
2017-04-26 07:36:18 -07:00
Pavol Vaskovic
481f542100 Support multiple prefix filters 2017-04-26 16:14:59 +02:00
Ted Kremenek
758d5d465a Resolve merge conflict. 2017-04-25 16:21:56 -07:00
swift-ci
9b0fe1a47a Merge pull request #8975 from palimondo/SR-4598 2017-04-25 14:07:19 -07:00
Pavol Vaskovic
cecfe439c6 Display OPT choices in usage help
Fixed pyton-lint warnings
2017-04-25 03:50:40 +02:00
Pavol Vaskovic
5c29fa65b1 Renamed args.benchmarks - plural 2017-04-25 03:25:29 +02:00
swift-ci
a67c6d00f5 Merge pull request #8923 from moiseev/reverse-improvements 2017-04-24 17:06:07 -07:00
Michael Ilseman
517ac2a66f [benchmark] Add in some (default off) mixed-string micro benchmarks
Adds in a benchmark to see how quickly we can traverse mixed latin and
emoji strings. Default disabled and part of the stringTests suite.
2017-04-24 14:57:42 -07:00
Pavol Vaskovic
094094a04c Addressed python style issue raised during review 2017-04-24 23:53:40 +02:00
Pavol Vaskovic
3a5051e392 Fix SR-4598 Add option to run subset of benchmarks matching a prefix 2017-04-24 23:40:38 +02:00
Max Moiseev
9aa1e61851 Sorting performance by delta instead of ratio
...to avoid problems with rounding.
2017-04-24 14:34:38 -07:00
Maxim Moiseev
1c1b2b966d [stdlib] String : RangeReplaceableCollection & BidirectionalCollection (#8921)
* [stdlib] String : RangeReplaceableCollection & BidirectionalCollection

* Add source compatibility hack for Swift.max

* Add source compatibility hack for Swift.min

* Remove redundant conformance in benchmarks

* Fix stupid typo I thought I'd already pushed

* XFAIL testing now-redundant conformance

* XFAIL an IDE test for now
2017-04-23 20:04:54 -07:00
Andrew Trick
f5410be16b compare_perf_tests.py: fix column header formatting.
Column names must not contain spaces for tools that auto-format the table.
The extra "(%)" was completely redundant since every value in the column
reads as a percentage.
2017-04-22 10:16:17 -07:00
Ted Kremenek
3fb6e71411 Force the benchmark suite to build with -swift-version 3.
Some of the benchmarks use Swift 3 APIs.  Let’s keep them building that
way to not perturb benchmark numbers.

We should consider adding benchmarks that specifically enable -swift-version 4.
2017-04-21 17:25:45 -07:00
Max Moiseev
c4dc74b9b6 Fixing the crash in compare_perf_tests
Comparisons should only be performed on the intersection of test lists,
otherwise it would crash should the new benchmark be introduced.
2017-04-21 14:40:38 -07:00
Max Moiseev
d878b45e29 Reverse the order of improvements in the output of compare_perf_tests
Both regressions and improvements are sorted by the delta, which in case
of improvements produces the reversed order due to negative values of
delta.
This change makes the improvements ordered 'naturally':
most-improved-first.
2017-04-21 13:59:02 -07:00
Arnold Schwaighofer
a4c1bc9e5e Make python lint happy 2017-04-20 20:16:16 -07:00
Arnold Schwaighofer
c9aa05c3c6 [benchmark] Move existential tests out of precommit. They are not very stable. 2017-04-20 13:11:20 -07:00