Commit Graph

231 Commits

Author SHA1 Message Date
David Ungar
3463451049 More fixes. 2018-11-29 13:23:33 -08:00
David Ungar
6823b937af Improvements thanks to Jordan’s feedback. 2018-11-28 10:40:36 -08:00
David Ungar
4c61354ff8 Added some comments around incremental compilation. 2018-11-27 21:18:42 -08:00
David Ungar
19caf599cb Review fixes 2018-10-29 10:27:53 -07:00
David Ungar
1f4dfce598 Add -enable-experimental-dependencies and push it through. 2018-10-28 00:39:36 -07:00
Graydon Hoare
0f26b9c12d [Driver] <rdar://43955209> Remove obsolete/fragile batch mode 'repartitioning' code. 2018-10-09 13:31:41 -07:00
Jordan Rose
e224e31720 Break almost all dependencies of Driver on Frontend
- Sink OutputFileMap{.h,.cpp} and ReferenceDependencyKeys.h to Basic
- Remove unnecessary includes of Frontend.h.
2018-08-27 20:47:58 -07:00
Graydon Hoare
d5a9f44e4a [BatchMode] <rdar://41271283> Limit memory pressure on large modules. 2018-07-30 17:26:31 -07:00
Bob Wilson
8e330ee344 NFC: Fix indentation around the newly renamed LLVM_DEBUG macro.
Jordan used a sed command to rename DEBUG to LLVM_DEBUG. That caused some
lines to wrap and messed up indentiation for multi-line arguments.
2018-07-21 00:56:18 -07:00
Jordan Rose
cefb0b62ba Replace old DEBUG macro with new LLVM_DEBUG
...using a sed command provided by Vedant:

$ find . -name \*.cpp -print -exec sed -i "" -E "s/ DEBUG\(/ LLVM_DEBUG(/g" {} \;
2018-07-20 14:37:26 -07:00
Robert Widmann
81e9a3f660 [NFC] Drop PerformJobsState as a friend class
Make the coupling between PerformJobsState and Compilation indirect.
2018-07-10 14:57:01 -07:00
Jordan Rose
7d8e40b0bb Merge pull request #16362 from dabelknap/frontend_responsefile
Wrap Command Line Arguments in a Response File if System Limits are Exceeded

https://bugs.swift.org/browse/SR-4517
2018-06-21 16:31:20 -07:00
Austin Belknap
d71373c95c Encapsulate the logic for returning a response file or the full arg vector. 2018-06-15 15:16:35 -07:00
Graydon Hoare
dcc4373dab [BatchMode] <rdar://40526328> Emit signalled message (SIGINT) for batch constituents cancelled due to errors elsewhere. 2018-06-14 11:51:31 -07:00
Graydon Hoare
8be2c53b24 [BatchMode] Add -driver-batch-count to allow overriding batch count inferred by -j 2018-06-14 11:51:31 -07:00
Robert Widmann
da13d89c24 Move DummyQueue logic into buildTaskQueue 2018-05-29 21:58:27 -07:00
Robert Widmann
d7493dd0fa [NFC] Expose the TaskQueue as a Compilation Parameter
Shuffle the responsibility for creating the TaskQueue out of the
Compilation's internal job state object and into the driver.  Expose
a builder convenience function that handles the argument parsing.
2018-05-29 21:58:27 -07:00
Benjamin Herzog
ac10fb3462 [Driver] Added process information to emitted messages
When providing the -parseable-output flag to the swift compiler, it will provide json formatted messages about tasks that run.
I added some optional usage information in form of user time, system time and maxrss to the output. This can be used by other tools using the compiler to get some insights about time and memory usage.
Since the output does not longer match processes run (in batch mode), I also added a real_pid field so the client could reason about jobs that belong together if needed.

rdar://39798231
2018-05-25 13:12:57 -07:00
Austin Belknap
da19625711 Merge branch 'master' into frontend_responsefile 2018-05-11 14:11:34 -07:00
Graydon Hoare
a0b4f3f481 Merge pull request #16492 from graydon/rdar-39981525-batch-mode-show-every-job-via-quasi-pids
[BatchMode] <rdar://39981525> Emit parseable output for each job, using quasi-PIDs.
2018-05-10 23:03:52 -07:00
Jordan Rose
606ee29b74 [Driver] -enable-batch-mode implies -continue-building-after-errors (#16521)
The logic in 46b8ad3 to avoid putting certain diagnostics into
serialized diagnostic files only makes sense if

1. every diagnostic emitted in file A.swift while processing a
   different file B.swift would be emitted if we processed A.swift on
   its own

2. we actually do process A.swift on its own in the same build,
   or have previously done an incremental build and produced the same
   diagnostic

But the latter isn't actually guaranteed: if one batch job exits with
a failure status, the driver will exit as well, assuming there's no
point in continuing. Fortunately, we do have a flag that overrides
this behavior, -continue-building-after-errors.

(As noted in the patch, -continue-building-after-errors isn't *exactly*
what we want. But it's conservatively correct.)
2018-05-10 21:44:12 -07:00
Graydon Hoare
39fdf5d4e5 [BatchMode] <rdar://39981525> Emit parseable output for each job, using quasi-PIDs. 2018-05-10 20:34:33 -07:00
Austin Belknap
59769014fc Wrap command line arguments in a response file if they exceed system limits. 2018-05-03 14:09:07 -07:00
Graydon Hoare
c1682fa465 [Driver] <rdar://39504759> Continue demultiplexing subprocess output after initial read. 2018-04-18 11:52:11 -07:00
Nathan Hawes
0a4204e99f [Driver] Add -driver-filelist-threshold to set the number of inputs beyond which filelists are used
When generating a compiler invocation in driver::createCompilerInvocation()
we end up using filelists if the number of inputs is > 128 (to work around
command line arg limits). We never actually write them out though, and so
fail when parsing the frontend arguments that reference them.

As this function is called frequently by SourceKit and command line limits
aren't a concern here, this patch makes the 128 threshold value configurable
via a new -driver-filelist-threshold option. This is set to its maximum value
in driver::createCompilerInvocation() to ensure filelists aren't used. This
new option makes the existing -driver-use-filelists (that forces filelists to
be used) redundant as it's now equivalent to -driver-filelist-threshold=0.

Resolves rdar://problem/38231888
2018-04-12 05:24:03 -07:00
Rintaro Ishizaki
2cec8dfc99 [Driver] Factor out build-record filename construction
* Retrive filename for read and write at the same time.
* Read LastBuildTime directly from the build record.
2018-04-03 20:39:55 +09:00
Rintaro Ishizaki
18ceef56e3 [Driver] Use separated build record file for '-emit-module' only mode
* In full compilation '-c' with '-emit-module', output duplicated build
  record file for full compilation *and* emit module ('~moduleonly') mode.
* In '-emit-module' only mode, use '~moduleonly' build record.
2018-04-02 21:04:13 +09:00
David Ungar
4fa3001bd1 Harden compiler against -num-threads and -enable-batch-mode. 2018-03-26 17:04:31 -07:00
David Ungar
807dd5864e Renamed swift::types to swift::file_types. 2018-03-22 08:40:06 -07:00
David Ungar
e3535ff3d8 Merge pull request #15371 from davidungar/PR-18-11-fileMap-redo
[Batch mode] Write out filemaps for supplementary outputs.
2018-03-21 17:05:36 -07:00
David Ungar
90773d298c Write out filemaps for supplementary outputs.
Add -driver-force-one-batch-repartition and enhance  batch_mode-overlong_argv test.
2018-03-20 09:42:13 -07:00
Pavel Yaskevich
fd3610b8c0 Revert "[Batch mode] Write supplementary output file map. rdar://problem/38157859" 2018-03-17 21:01:46 -07:00
David Ungar
a0cd1c57bc Merge pull request #15013 from davidungar/PR-18-11-fileMap
[Batch mode] Write supplementary output file map. rdar://problem/38157859
2018-03-16 17:38:53 -07:00
David Ungar
919c224d5a Handle index file path differently. 2018-03-15 22:00:16 -07:00
David Ungar
3a0acdaf0c Fix supplementary output filemap generation to account for CommandOutput. 2018-03-15 21:28:56 -07:00
David Ungar
9dcbaa38f7 Factor out writeOutputFileMap 2018-03-15 13:54:43 -07:00
David Ungar
e499aeda9b Redo force repartion flag and add a test. 2018-03-13 17:20:12 -07:00
David Ungar
3570af7476 Fixes 2018-03-12 15:40:08 -07:00
David Ungar
180bdd61be Implement -index-file-path 2018-03-11 11:48:21 -07:00
David Ungar
ba1a714dbe Formatting 2018-03-10 16:10:38 -08:00
David Ungar
c9db72c1cb Debug macro 2018-03-09 19:58:20 -08:00
David Ungar
65267c6568 WIP 2018-03-09 15:45:38 -08:00
David Ungar
adfda9bbc8 Add -driver-force-one-batch-repartition and enhance batch_mode-overlong_argv test. 2018-03-09 13:54:44 -08:00
David Ungar
798dd7b490 Write out filemaps for supplementary outputs. 2018-03-09 13:54:44 -08:00
Graydon Hoare
fa830fe5aa [BatchMode] Preserve invariant that shuffled batches are subsequences of inputs. 2018-03-09 10:13:04 -08:00
Sho Ikeda
cea6c03eb2 [gardening] Use !empty() over size() > 0 2018-03-08 09:21:09 +09:00
David Ungar
90d2740817 Merge pull request #14961 from davidungar/PR-18-10-HHH
[Batch mode] Restore -### functionality for batch-mode by extending OutputLevel.
2018-03-06 09:54:03 -08:00
Graydon Hoare
6c9029ead8 [BatchMode] Fix rdar://38123690 a little more robustly. 2018-03-05 13:43:30 -08:00
Graydon Hoare
eb315e74ed [BatchMode] Temporarily cap max batch size, fixing rdar://38123690 2018-03-05 12:28:48 -08:00
Graydon Hoare
899c1f5b87 [Driver] Handle failure-to-exec somewhat more politely, rdar://37865437. 2018-03-05 12:28:48 -08:00