averello/swift-mirror
1
0
Fork 0
mirror of https://github.com/apple/swift.git synced 2025-12-14 20:36:38 +01:00
Code Issues Packages Projects Releases Wiki Activity
87,123 Commits 622 Branches 2,942 Tags
22d37eccd818b51fbbefda5b6f13d2bc8f44f757
Commit Graph

231 Commits

Author SHA1 Message Date
Pavol Vaskovic
007d398f4a [Gardening] ReportFormatter: tying up loose ends 2019-05-24 00:18:44 +02:00
Pavol Vaskovic
b3f7996ea7 [benchmark] ReportFormatter: better inline headers 
Improve inline headers in `single_table` mode to also print labels for the numeric columns.

Sections in the `single_table` are visually distinguished by a separator row preceding the the inline headers.

Separated header label styles for git and markdown modes with UPPERCASE and **Bold**  formatting respectively.

Inlined section template definitions.
 2019-05-23 23:24:51 +02:00
Pavol Vaskovic
73b31006ee [benchmark] Fix help printing for run_smoke_bench 2019-05-23 21:40:44 +02:00
Pavol Vaskovic
9750581bf5 [benchmark] ReportFormatter: right-align num cols 2019-05-23 19:32:34 +02:00
Pavol Vaskovic
af7ef03aaf [benchmark] ReportFormatter: refactor header logic 
Confine the logic for printing headers to the header function.
 2019-05-23 17:28:21 +02:00
Pavol Vaskovic
a998e18e18 [benchmark] ReportFormatter: faster templating 
It is slightly faster to simply concatenate strings that don’t require special formatting.
 2019-05-23 12:29:19 +02:00
Pavol Vaskovic
49d25bfc51 [benchmark] ReportFomatter: de-tuple 
Remove unnecessary list-to-tuple conversions.
 2019-05-23 12:20:19 +02:00
Pavol Vaskovic
081e1c94a5 [benchmark] Add unit test for single table report 2019-05-22 14:54:00 +02:00
Michael Gottesman
c86c1763c6 [benchmarks] Add support to the build-script swiftpm benchmarks for building the benchmarks in -Osize. 2019-04-11 10:10:38 -07:00
Michael Gottesman
53ff97428a [benchmarks] Change the build_script_helper to use subdirectories for each build and install final binaries in a toplevel ./bin build directory. 
This will let me:

1. Add -Osize support easily.
2. Put all of the binaries in the same directory so that Benchmark_Driver can
   work with them via the -tools argument.
 2019-04-10 22:18:50 -07:00
Michael Gottesman
115f7a43e0 Move build_script_helper from ./benchmarks/utils => ./benchmarks/scripts. 2019-04-10 22:18:50 -07:00
Pavol Vaskovic
691007b029 [benchmark] LogParser: Accept -?! in bench. names 
Extend parser to support benchmark names that include `-?!` in names, to fully support the new Naming Convention from PR #20334.
 2019-02-19 23:31:58 +01:00
Pavol Vaskovic
84e7d4dfb8 [benchmark] Adjust Driver’s console output format 
…to handle longer benchmark names, assuming maximum length of 40 characters.
 2019-02-19 23:28:51 +01:00
Pavol Vaskovic
3f179f39e0 Increase # of independent samples for changes. 
Multimodal benchmarks with significant delta between the modes can report false performance changes when we gather too few independent samples. This increases the minimal number of independent samples from 5 to 10.
Fix for https://bugs.swift.org/browse/SR-9907
 2019-02-12 11:42:51 +01:00
Pavol Vaskovic
85ba83191e [benchmark] Remove unused function get_results 
Remove the `get_results` function, which is no longer used after the refactoring that rebased the benchmark measurements on `BenchmarDriver` class in #21684.
 2019-02-04 10:11:57 +01:00
Patrick Balestra
3c6b3ab4cc [benchmark] Fix linter errors in create_benchmark.py 2019-01-21 22:40:23 +01:00
Patrick Balestra
0b3fa54249 [benchmark] Split template into separate line and fix linter errors 2019-01-21 22:40:23 +01:00
Patrick Balestra
1ca47e7870 [benchmark] Add script to automate creation of new single-source benchmarks 
Adds a `create_benchmark` script that automates the following three tasks:
1. Add a new Swift file (YourTestNameHere.swift), built according to the template below, to the {{single-source}}directory.
2. Add the filename of the new Swift file to CMakeLists.txt
3. Edit main.swift. Import and register your new Swift module.

The process of adding new benchmarks is now automated and a lot less error-prone.
 2019-01-20 22:22:08 +01:00
Gwynne Raskind
faf8a5edb6 Fix indentation for python_lint 2019-01-17 02:00:40 -06:00
Gwynne Raskind
09b4159cb2 Global replace of "assertEquals" with "assertEqual" in compliance with deprecation of assertEquals name in Python 2.7 2019-01-16 04:06:38 -06:00
Pavol Vaskovic
2c271493d5 [benchmark] Limit of Accuracy in Setup Overhead 
Clarified limit of accuracy in setup overhead detection.
 2019-01-09 18:01:06 +01:00
Pavol Vaskovic
2096151ee9 [benchmark] BenchmarkDoctor: Lower runtime limit 
Warn about runtimes under 20 μs and flag 0 μs runtimes as errors.
 2019-01-08 19:16:40 +01:00
Pavol Vaskovic
8a8a3ad6df [benchmark] Limit setup overhead detection (>20) 
For really small runtimes < 20 μs this method of setup overhead detection doesn’t work. Even 1μs change in 20μs runtime is 5%. Just return no overhead.
 2019-01-08 19:15:29 +01:00
Pavol Vaskovic
d854f0f898 [benchmark] test_performance with BenchmarkDriver 
Refactored `test_perfomance` function to use existing  BenchmarkDriver and TestComparator.

This replaces hand-rolled parser and comparison logic with library functions which already have full unit test coverage.
 2019-01-08 00:22:00 +01:00
Pavol Vaskovic
cd4886aa2b [Gardening] Move imports and DriverArgs to top 2019-01-07 20:59:47 +01:00
Pavol Vaskovic
4a716445df [benchmark] BernchmarkDriver run in batch mode 
Finished support for running all active tests in one batch. Returns a dictionary of PerformanceTestResults.

Known test names are passed to the harness in a compressed form as test numbers.
 2019-01-07 20:59:39 +01:00
Pavol Vaskovic
df3389259b [benchmark] BenchmarkDriver: store test_numbers 2019-01-07 20:57:47 +01:00
Pavol Vaskovic
1f58ad6662 [Gardening] Better names: _tests_by_name_or_number 2019-01-07 20:57:47 +01:00
Pavol Vaskovic
3023ab5545 [benchmark] BenchmarkDriver sample_time support 
Added support for Benchmark_X’s `--sample-time` parameter .
 2019-01-07 20:57:42 +01:00
Pavol Vaskovic
b831f93dd4 [benchmark] BenchmarkDoctor: Optional markdown arg 
Don't require the presence of `markdown` argument for initialization.
(It doesn't exist when BenchmarkDoctor is used from `run_smoke_bench`.)
 2018-12-21 21:26:24 +01:00
Pavol Vaskovic
46f94d7709 [benchmark] BenchmarkDriver check --markdown 
Added `--markdown` flag for the `check` command to output the `BenchmarkDoctor`’s report in the Markdown format (as used by swift-ci on GitHub).
 2018-12-21 01:22:38 +01:00
Andrew Trick
5154886491 Merge pull request #20334 from palimondo/within-cells-interlinked 
[benchmark] Naming Convention
 2018-12-13 08:20:23 -08:00
Pavol Vaskovic
ec836bd04b Merge pull request #20861 from palimondo/a-tall-white-fountain-played 
[benchmark] Janitor Duty, Legacy Factor: A-C
 2018-12-07 19:49:06 +01:00
Pavol Vaskovic
9d6f7ad160 [benchmark] Driver & Doctor: Lower the sample cap 
Lowered the default sample cap from 2k to 200. (This doesn’t effect manually specified `--num-samples` argument in the driver.)

Swift benchmarks have pretty constant performance profile over time. It’s more beneficial to get multiple independent measurements faster, than more samples from the same run.
 2018-12-07 15:06:43 +01:00
Erik Eckstein
5797e3d8c7 benchmarks: Remove the obsolete bench_code_size.py and run_smoke_bench.py scripts. 
Those scripts are replaced by run_smoke_bench.

This is a follow-up commit to 040aa06fec
 2018-12-05 16:02:46 -08:00
Pavol Vaskovic
d6392cc014 Merge pull request #20807 from palimondo/and-dreadfully-distinct 
[benchmark] Added Benchmark Check Report
 2018-11-29 05:35:18 +01:00
Pavol Vaskovic
cd47f32350 [benchmark] Don’t emit report for no added tests 2018-11-28 23:57:23 +01:00
Pavol Vaskovic
94e86c043e [benchmark] Call test_opt_levels directly 
Simplify the code by removing indirection:
After removal of -check-added option, only single function remains.
 2018-11-28 22:47:02 +01:00
Pavol Vaskovic
98d6f22a7d [benchmark] run_smoke_bench -skip-check-added 
Removed the option to `-check-added`, as it’s now run by default. Replaced with option to skip checking added benchmarks: `-skip-check-added`.
 2018-11-28 21:47:57 +01:00
Pavol Vaskovic
cbd882119c [benchmark] Added Benchmark Check Report 
Produce Markdown formatted report, analyzing the quality of newly added benchmarks.
 2018-11-27 23:01:39 +01:00
Pavol Vaskovic
92cf40dcd3 [benchmark] MarkdownReportHandler 
`logging.Handler` that creates nicely formatted report from `BecnhmarkDoctor`’s `check` in Markdown table for display on GitHub.
 2018-11-27 22:55:02 +01:00
Pavol Vaskovic
9a04207735 [benchmark] Doctor: emit mem_page details info 
Promoting previously DEBUG message to INFO.
 2018-11-27 22:49:07 +01:00
Pavol Vaskovic
bc0064d285 [benchmark] Simpler naming convention regex 2018-11-19 10:03:33 +01:00
Pavol Vaskovic
4e5f51d2f6 [benchmark] run_smoke tests with new naming 
Allow for running of test matching the naming convention proposed in #20334.
 2018-11-17 22:35:58 +01:00
Pavol Vaskovic
af719248e6 [benchmark] run_smoke_bench deterministic hashing 
Tests that used hashing were being unnecessarily tested multiple times, because this environment variable was missing.
 2018-11-17 21:38:46 +01:00
Graydon Hoare
6b972f5165 Merge pull request #20347 from graydon/flake8-fixes 
Flake8 fixes
 2018-11-06 17:30:01 -08:00
Graydon Hoare
71da5ec519 Fix flake8 warning W605 invalid escape sequence. 2018-11-06 11:38:39 -08:00
eeckstein
f64f02bfde Merge pull request #20212 from palimondo/fluctuation-of-the-pupil 
[benchmark] Legacy Factor
 2018-11-06 11:31:08 -08:00
Graydon Hoare
4aa01e908a Fix flake8 warning E741 ambiguous variable name 'l'. 2018-11-05 21:01:33 -08:00
Pavol Vaskovic
b4f901bae4 [benchmark] Naming Convention 
New benchmark naming convention for better readability and improved naming system that accounts for performance coverage growth going forward.
 2018-11-05 22:44:50 +01:00