Commit Graph

1638 Commits

Author SHA1 Message Date
Pavol Vaskovic
cdcb631469 [benchmark] Refactor run runBenchmarks logVerbose
Extracted nested func logVerbose as instance method on SampleRunner.

Internalized the free functions `runBech` and `runBenchmarks` into SampleRunner as methods `run` and `runBenchmarks`.
2018-08-31 17:17:48 +02:00
Pavol Vaskovic
265f537d10 [benchmark] Extract yield & add resetMeasurements 2018-08-31 17:17:48 +02:00
Pavol Vaskovic
be39c02001 [benchmark] Refactor numIters computation
The spaghetti if-else code was untangled into nested function that computes `iterationsPerSampleTime` and a single constant `numIters` expression that takes care of the overflow capping as well as the choice between fixed and computed `numIters` value.

The `numIters` is now computed and logged only once per benchmark measurement instead of on every sample.

The sampling loop is now just a single line. Hurrah!

Modified test to verify that the `LogParser` maintains `num-iters` derived from the `Measuring with scale` message across samples.
2018-08-31 17:17:48 +02:00
Pavol Vaskovic
46ee2a4bd8 [benchmark] Refactor sampling loop with addSample
Extracted sample saving to inner func `addSample`.
Used it to save the `oneIter` sample from `numIters` calibration when it comes out as 1 and continue the for loop to next sample.

This simplified following code that can now always measure the sample with `numIters` and save it.
2018-08-31 07:32:23 +02:00
Pavol Vaskovic
3c55e30382 [benchmark] Gardening: Documentation of numIters
Clarified the need for capping `numIters` according to the discussion at https://github.com/apple/swift/pull/17268#issuecomment-404831035

The sampling loop is a hairy piece of code, because it’s trying to reuse the calibration measurement as a regular sample, in case the computed `numIters` turns out to be 1. But it conflicts with the case when `fixedNumIters` is 1, necessitating a separate measurement in the else branch… That was a quick fix back then, but its hard to make it clean. More thinking is required…
2018-08-31 07:32:22 +02:00
Pavol Vaskovic
6e27af7142 [benchmark] Gardening: Sensibly rename variables
To make sense of this spaghetti code, let’s first use reasonable variable names:
* scale -> numIters
* elapsed_time -> time
2018-08-31 07:32:21 +02:00
Pavol Vaskovic
46bef893d0 [benchmark] Gardening: DRYer verbose log 2018-08-31 07:32:20 +02:00
Pavol Vaskovic
9c4876eabd [benchmark] Refactor to currency type Int
Removed unnecessary use of UInt64, where appropriate, following the advice from Swift Language Guide:

> Use the `Int` type for all general-purpose integer constants and variables in your code, even if they’re known to be nonnegative. Using the default integer type in everyday situations means that integer constants and variables are immediately interoperable in your code and will match the inferred type for integer literal values.
https://docs.swift.org/swift-book/LanguageGuide/TheBasics.html#ID324
2018-08-31 07:32:19 +02:00
Pavol Vaskovic
f017d98050 [benchmark] Refactor to report samples in μs
Moved the adjustment of `lastSampleTime` to account for the `scale` (`numIters`) and conversion to microseconds into SampleRunner’s `measure` method.
2018-08-31 07:32:18 +02:00
Pavol Vaskovic
a03aede90d [benchmark] Gardening: scale was always Int
Since the `scale` (or `numIters`) is passed to the `test.runFunction` as `Int`, the whole type-casting dance here was just silly!
2018-08-31 07:32:17 +02:00
Pavol Vaskovic
bf4a343124 [benchmark] Gardening: numSamples UInt vs Int
Type check command line argument to be non-negative, but store value in currency type `Int`.
2018-08-31 07:32:16 +02:00
Pavol Vaskovic
77dff0a1d7 [benchmark] Gardening: afterRunSleep is UInt32 2018-08-31 07:32:14 +02:00
Pavol Vaskovic
aa4b84934a [benchmark] Move stats computation to BenchResults 2018-08-31 07:32:12 +02:00
Pavol Vaskovic
0db20feda2 [benchmark] Fix index computation for quantiles
Turns out that both the old code in `DriverUtils` that computed median, as well as newer quartiles in `PerformanceTestSamples` had off-by-1 error.

It trully is the 3rd of the 2 hard things in computer science!
2018-08-31 07:32:10 +02:00
Pavol Vaskovic
5cd9f53840 [benchmark] Refactor min max median computation
We can spare 2 array passes (for min and max), if we just sort first.
2018-08-31 07:32:09 +02:00
Pavol Vaskovic
963995fa2b [benchmark] Refactor mean and stdev computation 2018-08-31 07:32:08 +02:00
Pavol Vaskovic
974994c13a [benchmark] Gardening: Timer Parasite Control
Improve conformance to Swift Naming Guidelines
https://swift.org/documentation/api-design-guidelines/#parameter-names

Removed the gross undescores and ticks from parameter names. Ticks are ectoparasites feeding on the blood. We are just measuring time and there is also no [mysterious ticking noise](http://bit.ly/TickNoise) here either…
2018-08-31 07:32:06 +02:00
Pavol Vaskovic
c1a694de30 [benchmark] Gardening: Extract constant oneSecond 2018-08-31 07:32:04 +02:00
Pavol Vaskovic
b373132548 [benchmark] Gardening: Code format class Timer 2018-08-31 07:32:03 +02:00
Pavol Vaskovic
e6cff27bab [benchmark] Gardening: Indentation of .listTests 2018-08-31 07:32:02 +02:00
swift-ci
33d95a3d22 Merge pull request #18984 from graydon/follow-my-simple-instruction 2018-08-30 16:20:29 -07:00
eeckstein
10468c6e05 Merge pull request #19011 from palimondo/fluctuation-of-the-pupil
[benchmark] Yield when time slice expires
2018-08-29 10:44:33 -07:00
Graydon Hoare
e7903f95c2 [benchmark] Compensate for missing header in some SDKs. 2018-08-28 14:00:19 -07:00
Pavol Vaskovic
b482a049c3 [benchmark] Gentleman yields. Don’t be a CPU hog!
It pays off of to be a nice process and yield the processor at regular intervals, to prevent having measured samples corrupted by preemptive multitasking.

When the scheduled time slice (10ms on Mac OS) is probably about to expire during the next measurement, we voluntarily yield  and resume the measurement within a fresh scheduler quantum.

This cooperative approach to multitasking improves the sample quality and robustness of the measurement process.
2018-08-28 05:37:43 +02:00
Pavol Vaskovic
7969be1c35 [benchmark] Explicitly enable static dispatch
Marking `Timer` and `SampleRunner` classes as `final` to make sure their methods use static dispatch.
2018-08-28 05:37:43 +02:00
Pavol Vaskovic
c98006c10b [benchmark] Gardening: Fixed indentation 2018-08-28 05:37:43 +02:00
Erik Eckstein
1f32935fc4 benchmarks: fix regexp for parsing code size results
Accept a '.' in the benchmark name which is used for .o and .dylib files
2018-08-27 17:07:40 -07:00
Graydon Hoare
6d32861df7 [benchmark] Fix module-inclusion bug caused by my forgetting about case-insensitive filesystems. 2018-08-27 15:18:55 -07:00
Chéyo Jiménez
a527e53e17 Renamed DictionaryLiteral to KeyValuePairs (#16577)
* renamed DictionaryLiteral to KeyValuePairs per SE-0214

* renamed DictionaryLiteral type tests to KeyValuePairs

* [SE-0214] Move changelog entry (Swift 4.2 => 5.0)

* [SE-0214] Update comment in AST/Expr.h

* [SE-0214] Use generic typealias

See also <https://github.com/apple/swift/pull/17711>

* [SE-0214] Update source-stability.swift.expected
2018-08-27 10:51:12 -07:00
Graydon Hoare
94725dff2b [benchmark] Add preliminary helper for measuring instructions executed. 2018-08-25 01:29:37 -07:00
eeckstein
e66170211a Merge pull request #18924 from palimondo/fluctuation-of-the-pupil
BenchmarkDriver Strangler replaces Benchmark_Driver run
2018-08-24 16:35:34 -07:00
Karoy Lorentey
11b290c452 [benchmark] SetTests: Move legacy benchmarks to end of list. 2018-08-24 22:35:41 +01:00
Karoy Lorentey
09946d074d [benchmark] SetTest: Add tests for more operations and more cases
- Isolate legacy tests. Add new tests for the same operations, with updated iteration counts and names.
- Rename new tests to follow a consistent naming scheme.
- Add tests for Set.subtracting.
- Add tests on integer sets with 50% and 100% overlap. (isSubset, intersection, union, symmetricDifference, subtracting)
2018-08-24 22:34:05 +01:00
Karoy Lorentey
91dc5c6a73 [benchmark] SetTests: Explain input sets 2018-08-24 17:54:35 +01:00
Karoy Lorentey
a154e46fe1 [benchmark] SetTests: Re-add original tests
The code isn’t the same, but comparisons with previous releases will still be useful (as long as timings match with the original code).
2018-08-24 11:59:54 +01:00
Karoy Lorentey
6426e7e14c [benchmark] SetTests: Better parametrization, separate setup 2018-08-24 11:58:33 +01:00
Karoy Lorentey
ebb5adffba [benchmark] Fix benchmarks for Set operations
- Don’t use a random number generator
- Ensure there is a 25% overlap between sets. Original benchmarks tested non-overlapping sets only.
- Change names; results aren’t directly comparable.
2018-08-23 19:53:31 +01:00
Pavol Vaskovic
13c499339b [benchmark] Class descriptions in module doctring 2018-08-23 19:59:15 +02:00
Pavol Vaskovic
49e8e692fb [benchmark] Strangle run and run_benchmarks
Moved all `run` command related functionality to `BenchmarkDriver`.
2018-08-23 18:01:46 +02:00
Pavol Vaskovic
6bddcbe9e4 [benchmark] Refactor run_benchmarks log format 2018-08-23 18:01:46 +02:00
Pavol Vaskovic
7ae5d7754c [benchmark] Report totals as a sentence
Clean up after removing bogus agregate statistics from last line of the log. It makes more sense to report the total number of executed benchmarks as a sentence that trying to fit into the format of preceding table.

Added test assertion that `run_benchmarks` return csv formatted log, as it is used to write the log into file in `log_results`.
2018-08-23 18:01:46 +02:00
Pavol Vaskovic
ef1461ca46 [benchmark] Strangle log_results
Moved `log_results` to BenchmarkDriver.
2018-08-23 18:01:46 +02:00
Pavol Vaskovic
a10b6070dd [benchmark] Refactor log_results
Added tests for `log_results` and the *space-justified-columns* format emited to stdout while logging to file.
2018-08-23 18:01:46 +02:00
Pavol Vaskovic
1d3fa87fdd [benchmark] Strangle log_results -> log_file
Moved the `log_file` path construction to the `BenchmarkDriver`.
Retired `get_*_git_*` functions.
2018-08-23 18:01:41 +02:00
Pavol Vaskovic
049ffb34b0 [benchmark] Fix parsing formatted text
The test number column in the space justified column format emmited by the Benchmark_Driver to stdout  while logging to file is right aligned, so it must handle leading whitespace.
2018-08-23 12:31:00 +02:00
Pavol Vaskovic
0d64386b53 [benchmark] Documentation improvements
Improving complience with
PEP 257 -- Docstring Conventions
https://www.python.org/dev/peps/pep-0257/
2018-08-23 11:45:43 +02:00
Joe Shajrawi
5e7ea888fb [Exclusivity] Teach MemAccessUtils about Projection Paths
Consider a class ‘C’ with distinct fields ‘A’ and ‘B’

And consider we are accessing C.A and C.B inside a loop

LICM well not hoist the exclusivity checking outside of the loop because isDistinctFrom(C.A, C.B) returns false.

This is because the helper function bails if isUniquelyIdentified returns false (which is the case in class kinds)

Same with all other potential access enforcement optimizations.

This PR resolves that
2018-08-22 18:11:55 -07:00
swift-ci
643154fef2 Merge pull request #18719 from palimondo/fluctuation-of-the-pupil 2018-08-22 12:48:56 -07:00
Erik Eckstein
13aae18d55 benchmarks: fix the cmake code to detect if SWIFT_EXEC supports the -align-module-to-page-size option.
This check should only be done for pre-built compilers.
2018-08-22 09:58:20 -07:00
Pavol Vaskovic
f38e6df914 [benchmark] Doctor verifies constant memory use
This needs to be finished with function approximating normal range based on the memory used.
2018-08-20 16:52:07 +02:00