Commit Graph

48 Commits

Author SHA1 Message Date
Erik Eckstein eaff604f41 tests: don't fail if instruction counts are not available for scale tests
Instead of explicitly limiting those tests to specific platforms, just ignore "No data" results.

rdar://176820820
2026-05-12 21:03:36 +02:00
Erik Eckstein 15711e20b1 utils: add option to do a simple 3-point test with utils/scale-test and check the superlinear factor
This is useful for general compile time regression tests which check overall user time of the whole compilation pipeline.
2026-05-07 09:52:09 +02:00
Jamie 7757cf62d2 [SILOptimizer]: add a (xfail) scale-test for poor DI scaling (#88064) 2026-03-25 05:44:49 -07:00
Slava Pestov 1277763350 scale-test: Use the same seed for the RNG every time 2025-10-23 13:15:27 -04:00
Slava Pestov 5ec225eae6 scale-test: Pass -fine-grained-timers
Also, if all the stats are zero, raise an error instead of
silently succeeding.
2024-09-23 18:04:25 -04:00
Josh Soref a296687335 spelling: between
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
2022-09-15 19:43:36 -04:00
YOCKOW c1e154a9cb [Gardening] Remove trailing whitespaces in Python scripts. (W291)
That has been marked as 'FIXME' for three years.
This commit fixes it.
2022-08-25 16:08:36 +09:00
YOCKOW d10381557b [NFC] Python Lint: Fix E275(missing whitespace after keyword) issues. 2022-08-21 16:07:07 +09:00
Daniel Duan 3dfc40898c [NFC] Remove Python 2 imports from __future__ (#42086)
The `__future__` we relied on is now,  where the 3 specific things are
all included [since Python 3.0](https://docs.python.org/3/library/__future__.html):

* absolute_import
* print_function
* unicode_literals
* division

These import statements are no-ops and are no longer necessary.
2022-04-13 14:01:30 -07:00
Alex Hoppen 13dd22f0cc [scale-test] Don't consider timers for the scale tests
Timers are just too noisy to give reliable results. Previously, the
timers were not considered due to a bug in process-stats-dir that would
(almost) always output 0. This bug has been fixed in 92073c671e.
2021-01-22 14:50:07 +01:00
Saleem Abdulrasool deff87d4e8 validation-test: make scale-test explicitly handle encoding
During the Python 2 to Python 3 conversion, the difference in encoding
became apparent.  Explicitly handle the encoding by opening the file
with an explicit encoding.  This prevents falling back to the `C` locale
which will use ASCII for UTF-8 which fails.
2020-07-23 08:59:28 -07:00
tbkka 868425be8a More Python3 lint fixes (#32967)
* More Python3 lint fixes

Some of the issues addressed include:
* Don't use `l` as a variable name (confusable with `1` or `I`)
* `print` statement does not exist in Py3, use `print` function instead
* Implicit tuple deconstruction in function args is no longer supported,
  use explicit splat `*` at the call site instead
* `xrange` does not exist in Py3, use `range` instead

* Better name per review feedback
2020-07-19 21:44:13 -07:00
Saleem Abdulrasool 586f7ea4df validation: make scale-test Python 3 friendly
`zip` will return a generator in Python 3 rather than the zipped list.
This results in the Nelder-Mead Simplex to fail as it does not actually
perform the optimization of the data.  Explicitly convert the data to a
form which can be consumed.
2020-07-14 16:49:51 +00:00
Saleem Abdulrasool b6caab8cb7 test: use the o octal specifier for python 3 compatibility
This adjusts the permissions to use the octal specifier explicitly.
2020-07-13 15:50:40 +00:00
Ross Bayer ac431fa3e1 [Build System: build-script] Remove the old which module in swift_build_support and replace uses with the which function provided in the build_swift shell module. 2020-01-19 03:49:13 -08:00
Alex Hoppen 932525d762 [gardening] Fix several python-lint warnings 2019-10-29 10:40:20 -07:00
Alex Hoppen 776e2c0030 Revert "Migrate building SwiftSyntax to swift_build_support" 2019-10-29 09:55:32 -07:00
Alex Hoppen 46501b881f [gardening] Fix several python-lint warnings 2019-10-25 15:58:07 -07:00
Andrew Trick c7b4e7c8b4 Add scale-test --save-temps option.
I'm not sure how anyone debugs these tests otherwise.
2019-05-13 16:54:55 -07:00
Pavol Vaskovic c1388abd1b [Gardening] Fix I201
I201 Missing newline before sections or imports.
2018-11-28 17:13:04 +01:00
Xi Ge 53121aff84 scale-test: teach the test driver to test parse only. 2018-09-12 14:15:28 -07:00
David Ungar 28b206008f FrontendInputs data structure redo.
- Outlaw duplicate input files, fix driver, fix tests, and add test.
- Reflect that no buffer is present without a (possibly pseudo) named file.
- Reflect fact that every input has a (possible pseudo) name.
- Break up CompilerInstance::setup.

Don't bail on dups.
2017-12-05 17:28:03 -08:00
Graydon Hoare 2c281c7a20 [scale-test] Use -stats-output-dir when available. 2017-09-12 22:36:12 -07:00
Graydon Hoare c774d7f21d [scale-test] Revive direct linear fit as first-pass, tighten thresholds. 2017-09-12 22:36:11 -07:00
Mark Lacey d302f8caca Add --invert-result option to scale-test.
This makes it possible to distinguish between failed data fitting and
things that fail for other reasons (e.g. the test doesn't compile
properly).
2017-08-30 19:41:52 -07:00
swift-ci f2a1ffeb64 Merge pull request #11700 from moiseev/scale-test-error 2017-08-30 18:44:27 -07:00
Max Moiseev f2f352b1cc [scale-test] Reporting an error if --step is wider than the begin/end range 2017-08-30 17:35:48 -07:00
Graydon Hoare 6053946eaf [scale-test] Special-case constants; don't bother fitting anything. 2017-08-27 01:54:38 -07:00
Graydon Hoare 910c8540b7 [scale-test] Handle ill-fitting scenarios a bit more robustly. 2017-08-26 20:33:36 -07:00
Graydon Hoare 8be9394ded [scale-test] Fix review comments. 2017-08-24 23:09:05 -07:00
Graydon Hoare b082e05440 [scale-test] Numerical nonlinear regression, avoids linear(log()) mis-fitting. 2017-08-23 22:32:41 -07:00
Huon Wilson 7f149360b8 [scale-test] Allow compiler invocations that don't succeed.
This allows invocations that fail with a specific exit code, e.g. 0 is the old
behaviour, for success, but also, for instance, 1 for measuring examples that
fail to typecheck.
2017-08-11 15:39:28 -07:00
Huon Wilson 3f712ad87c [scale-test] Diagnose exponential growth explicitly.
This, somewhat questionably, fits the polynomial model and the
exponential model, and then chooses the one with the best R^2. However,
no matter how statistically valid this is, it works reasonably in
practice.

Slow growing things sometimes get classified as 1.0^n or 1.1^n, but
these are either spurious or not relevant, and so a similar thresholding
to the polynomial fit is used.
2017-04-10 13:22:16 -07:00
Hugh Bellamy ad4b338062 Fix python lint failures now not excluded as we provide custom exclusions
Looks like flake8 enables other rules when you add something to the
exclusion list. We added W291
2017-03-27 12:31:56 +07:00
Hugh Bellamy a5e7514197 Prefer the print function to the print statment in python code 2017-02-20 11:12:21 +07:00
Hugh Bellamy 4f23d61da0 Import print_function wherever we use print() in python code 2017-02-20 11:11:27 +07:00
Hugh Bellamy a977b8cc5f Python lint scale-test 2017-02-09 18:15:06 +07:00
Slava Pestov b9570eebfe scale-test: Fix a typo 2017-01-08 18:12:47 -08:00
David Farler b7d17b25ba Rename -parse flag to -typecheck
A parse-only option is needed for parse performance tracking and the
current option also includes semantic analysis.
2016-11-28 10:50:55 -08:00
Graydon Hoare 2a89323077 [Compile perf] flake8 fixes for scale-test. 2016-11-07 15:57:51 -08:00
Graydon Hoare 3d8f8e050c [Compile perf] Use lit tmpdir in scale-test, rdar://29090287 2016-11-07 15:57:33 -08:00
Graydon Hoare ac7d397aee [Compile perf] Add various convenience options to scale-test 2016-11-03 11:23:27 -07:00
Graydon Hoare c6d38f5ac1 [Compile perf] Treat missing data as an error in scale-test. 2016-11-02 14:05:02 -07:00
Graydon Hoare b9db5cf6b2 [Compile perf] Replace numpy with helper function in scale-test 2016-11-02 14:05:01 -07:00
Graydon Hoare 587df1e04d [Compile perf] Add --debug to scale-test, to run tests under lldb 2016-11-02 09:40:59 -07:00
Graydon Hoare d9fda6e0cb [Compile perf] Add a multi-primary cumulative mode to utils/scale-test 2016-11-02 09:40:59 -07:00
Graydon Hoare 9fa3f49828 [scale-test] Add --dtrace and --select support 2016-10-13 11:45:50 -07:00
Graydon Hoare 0b3e451940 Add utils/scale-test 2016-10-11 12:04:33 -07:00