Commit Graph

1638 Commits

Author SHA1 Message Date
Pavol Vaskovic
3ad6ee2c1c [benchmark] MapReduce Legacy Factor 2019-01-22 09:26:05 +01:00
Pavol Vaskovic
aca01c5268 [benchmark] LinkedList Legacy Factor 2019-01-22 09:26:05 +01:00
Pavol Vaskovic
cccb32bcb9 [benchmark] LazilyFiltered Legacy Factor
…and refactored setUp/tearDown
2019-01-22 09:26:05 +01:00
Pavol Vaskovic
f10ef5eb1a [benchmark] HashTest Legacy Factor 2019-01-22 09:26:05 +01:00
Pavol Vaskovic
9cf7f7e2a2 [benchmark] Hanoi Legacy Factor 2019-01-22 09:26:05 +01:00
Patrick Balestra
3c6b3ab4cc [benchmark] Fix linter errors in create_benchmark.py 2019-01-21 22:40:23 +01:00
Patrick Balestra
0b3fa54249 [benchmark] Split template into separate line and fix linter errors 2019-01-21 22:40:23 +01:00
Pavol Vaskovic
a72147662f [benchmark] ErrorHandling Legacy Factor 2019-01-21 21:33:14 +01:00
Patrick Balestra
daf7485e67 [benchmark] Update documentation to reflect usage of automated script to generate benchmarks 2019-01-20 22:22:20 +01:00
Patrick Balestra
1ca47e7870 [benchmark] Add script to automate creation of new single-source benchmarks
Adds a `create_benchmark` script that automates the following three tasks:
1. Add a new Swift file (YourTestNameHere.swift), built according to the template below, to the {{single-source}}directory.
2. Add the filename of the new Swift file to CMakeLists.txt
3. Edit main.swift. Import and register your new Swift module.

The process of adding new benchmarks is now automated and a lot less error-prone.
2019-01-20 22:22:08 +01:00
Patrick Balestra
fecff67e3e [benchmark] Sort registerBenchmark statements alphabetically 2019-01-20 22:00:26 +01:00
Gwynne Raskind
542d02a60e Merge pull request #21911 from gwynne/replace-deprecated-assertEquals
Replace deprecated Python unit test method usage
2019-01-18 10:06:13 -06:00
Gwynne Raskind
faf8a5edb6 Fix indentation for python_lint 2019-01-17 02:00:40 -06:00
Gwynne Raskind
09b4159cb2 Global replace of "assertEquals" with "assertEqual" in compliance with deprecation of assertEquals name in Python 2.7 2019-01-16 04:06:38 -06:00
Michael Gottesman
9911d0a1da [benchmark] Add autoreleasepools to the Codable tests to prevent "leaks".
This was caught by the runtime leak checker.

rdar://47241432
2019-01-15 22:33:41 -08:00
Pavol Vaskovic
626878e0d4 [benchmark] Data.[init,append].Sequence.[809B,64kB] 2019-01-15 22:04:46 +01:00
Pavol Vaskovic
eadb2430e1 [benchmark] DictionarySubscriptDefault LF 2019-01-11 22:43:21 +01:00
Pavol Vaskovic
acb2c18a6c [benchmark] DictionarySwap Legacy Factor 2019-01-11 22:43:15 +01:00
Pavol Vaskovic
85c8bff82a [benchmark] DictionaryRemove Legacy Factor 2019-01-11 13:19:52 +01:00
Pavol Vaskovic
8d0544dc39 [benchmark] DictionaryCompatMapValues legacyFactor 2019-01-11 13:18:51 +01:00
Pavol Vaskovic
64d610b40b [benchmark] DictionaryLiteral Legacy Factor 2019-01-11 13:18:50 +01:00
Pavol Vaskovic
b3fea042ff [benchmark] Dictionary2(OfObjects) Legacy Factor 2019-01-11 13:18:48 +01:00
Pavol Vaskovic
9d033cd276 [benchmark] Dictionary(OfObjects) Legacy Factor 2019-01-11 13:18:46 +01:00
Pavol Vaskovic
6caff6a0d7 [benchmark] DeadArray Legacy Factor 2019-01-11 13:18:43 +01:00
Itai Ferber
1d10a6a155 Merge pull request #21766 from itaiferber/data-additional-benchmarks
Additional Data benchmarks
2019-01-10 13:59:25 -08:00
Itai Ferber
04eaf07f1d Re-add Data.append.Sequence.ExactCount 2019-01-10 12:38:21 -08:00
Pavol Vaskovic
e3a843374d Reduce Data.hash.Medium multiplier
Co-Authored-By: itaiferber <itai@itaiferber.net>
2019-01-10 12:19:39 -08:00
Itai Ferber
e57ba83f48 Reduce multipliers on Data.init.Sequence 2019-01-10 12:17:15 -08:00
Itai Ferber
3952b7359c Update benchmark names 2019-01-10 12:13:21 -08:00
Pavol Vaskovic
5cc751cad0 Merge pull request #21717 from palimondo/cells-interlinked
[benchmark] BenchmarkDoctor: Lower runtime bound + Set.Empty fixes
2019-01-10 21:11:26 +01:00
Itai Ferber
beb326e808 Data hashing benchmarks 2019-01-10 10:23:33 -08:00
Itai Ferber
a457b0d223 Data.init<S>/append<S> worst-case benchmarks 2019-01-10 10:23:11 -08:00
Pavol Vaskovic
b4f1f4747d [benchmark] Remove 0s from Empty names. 2019-01-10 18:04:10 +01:00
Pavol Vaskovic
a30b7749dd [benchmark] SuperChars: Further lower the workload 2019-01-10 12:34:46 +01:00
Michael Ilseman
779252ae13 [benchmark] Rework SuperChars
Rework SuperChars into something that:

* Doesn't crash in asserts configurations (i.e. has valid Characters)
* Actually runs (i.e. use blackHole on comparison result)
* Runs in around 1ms (it's cubic complexity in payload size)
2019-01-09 16:58:57 -08:00
Pavol Vaskovic
2c271493d5 [benchmark] Limit of Accuracy in Setup Overhead
Clarified limit of accuracy in setup overhead detection.
2019-01-09 18:01:06 +01:00
Pavol Vaskovic
b4d8172228 [benchmark] Adjust loop multipliers Set.*
Increase the multipliers to get reliably measurable runtimes.
2019-01-09 12:22:48 +01:00
Lance Parker
15aaa1e777 [stdlib]String normalization functions (#21026)
* fast/foreignNormalize functions
2019-01-08 13:55:29 -08:00
Pavol Vaskovic
c011bd70c3 [benchmark] Add swapped Set.*.Empty variants
Let’s also test Empty sets as the right-hand side parameter.
2019-01-08 21:54:59 +01:00
Pavol Vaskovic
84580f88fe [benchmark] Adjust new tests to Naming Convention
Most of these recently added benchmarks have too low loop multiplier that results in near zero or zero measured runtime.

Since fixing this will change the runtimes, it is also an opportunity to properly apply the new naming convention.
2019-01-08 21:53:18 +01:00
Pavol Vaskovic
2096151ee9 [benchmark] BenchmarkDoctor: Lower runtime limit
Warn about runtimes under 20 μs and flag 0 μs runtimes as errors.
2019-01-08 19:16:40 +01:00
Pavol Vaskovic
8a8a3ad6df [benchmark] Limit setup overhead detection (>20)
For really small runtimes < 20 μs this method of setup overhead detection doesn’t work. Even 1μs change in 20μs runtime is 5%. Just return no overhead.
2019-01-08 19:15:29 +01:00
Pavol Vaskovic
8fcf884076 Merge pull request #21684 from palimondo/cells-interlinked
[benchmark] run_smoke_bench with BenchmarkDriver
2019-01-08 19:09:11 +01:00
Pavol Vaskovic
bd16513997 Merge pull request #21516 from palimondo/a-tall-white-fountain-played
[benchmark] Janitor Duty: Datalore Legacy
2019-01-08 06:31:41 +01:00
Pavol Vaskovic
7db46b3b14 [benchmark] Inlined loop miltipliers
Also added 2 forgotten legacy factors.
2019-01-08 02:06:10 +01:00
Pavol Vaskovic
d854f0f898 [benchmark] test_performance with BenchmarkDriver
Refactored `test_perfomance` function to use existing  BenchmarkDriver and TestComparator.

This replaces hand-rolled parser and comparison logic with library functions which already have full unit test coverage.
2019-01-08 00:22:00 +01:00
Pavol Vaskovic
cd4886aa2b [Gardening] Move imports and DriverArgs to top 2019-01-07 20:59:47 +01:00
Pavol Vaskovic
4a716445df [benchmark] BernchmarkDriver run in batch mode
Finished support for running all active tests in one batch. Returns a dictionary of PerformanceTestResults.

Known test names are passed to the harness in a compressed form as test numbers.
2019-01-07 20:59:39 +01:00
Pavol Vaskovic
df3389259b [benchmark] BenchmarkDriver: store test_numbers 2019-01-07 20:57:47 +01:00
Pavol Vaskovic
1f58ad6662 [Gardening] Better names: _tests_by_name_or_number 2019-01-07 20:57:47 +01:00