Commit Graph

305 Commits

Author SHA1 Message Date
David Ungar
38a6559aa9 Take path arg for comparo 2019-11-18 20:23:10 -08:00
David Ungar
0a0d22abab fmt 2019-11-16 23:54:52 -08:00
David Ungar
edbbdb65c7 WIP 2019-11-16 23:41:31 -08:00
David Ungar
a6bf84e20b refactoring WIP unfmt 2019-11-16 23:17:37 -08:00
David Ungar
3433bf8cef comparision WIP 2019-11-16 21:39:19 -08:00
David Ungar
0b00afea7e pass compilation in to loadAllInfo in prep to do removed files 2019-11-16 08:52:11 -08:00
David Ungar
15ea29c3aa Handle new file 2019-11-15 22:22:06 -08:00
David Ungar
754c44fbc7 Test top-level additions 2019-11-15 13:21:45 -08:00
David Ungar
30429c0fa1 Ensure that files with missing supplementaries get recompiled. 2019-11-12 20:41:03 -08:00
David Ungar
033eaec1e8 Fix bug around scheduling when dep loads fail. 2019-11-12 20:41:03 -08:00
David Ungar
b309759028 Keep message same as before for the sake of tests. 2019-11-12 20:41:03 -08:00
David Ungar
69dcc3ba18 Fix bug for newly-added failing job with swiftdeps filename 2019-11-12 20:41:03 -08:00
David Ungar
2cbd5e7a16 Ensure that a dependency load failure schedules all jobs. 2019-11-12 20:41:03 -08:00
David Ungar
742c3985bd Source-range-based dependencies
Frontend outputs source-as-compiled, and source-ranges file with function body ranges and ranges that were unparsed in secondaries.
Driver computes diffs for each source file. If diffs are in function bodies, only recompiles that one file. Else if diffs are in what another file did not parse, then the other file need not be rebuilt.
2019-11-12 20:41:02 -08:00
Jordan Rose
632399e0f8 [Driver] Include more info in "crash because TMPDIR is borked" errors (#27412)
Two places in Driver are creating temporary files at a point in the
process where failure is not expected. We should do something better
about this, but meanwhile harmonize their failures and include a
little more info.

Filed https://bugs.swift.org/browse/SR-11541 to improve this.
2019-09-30 18:18:27 -07:00
Jason Mittertreiner
8998235ed6 Enable Child Memory Usage Tracking on Windows (#23686)
Windows requires a handle to get memory usage, so do a slight refactor
to collect the child's memory usage as it exits instead of as the parent
is cleaning up.
2019-03-30 18:54:30 -07:00
David Ungar
2f65ac3735 Cosmetic cleanups 2019-01-30 17:16:49 -08:00
David Ungar
1fa76b97da Rm debugging code 2019-01-28 15:59:55 -08:00
David Ungar
14ba2a1b60 Format 2019-01-28 15:56:18 -08:00
David Ungar
ae46aa918a Various fixes + debug unfmt 2019-01-27 19:12:59 -08:00
David Ungar
e912642304 1st cut dependency tracing 2019-01-26 18:50:41 -08:00
David Ungar
9d84bc2af1 WIP tracer 2019-01-26 09:36:40 -08:00
David Ungar
323c42c532 Fomat 2019-01-24 18:46:49 -08:00
David Ungar
b520b242dc Collapsed lots of changes. 2019-01-24 18:46:49 -08:00
David Ungar
3463451049 More fixes. 2018-11-29 13:23:33 -08:00
David Ungar
6823b937af Improvements thanks to Jordan’s feedback. 2018-11-28 10:40:36 -08:00
David Ungar
4c61354ff8 Added some comments around incremental compilation. 2018-11-27 21:18:42 -08:00
David Ungar
19caf599cb Review fixes 2018-10-29 10:27:53 -07:00
David Ungar
1f4dfce598 Add -enable-experimental-dependencies and push it through. 2018-10-28 00:39:36 -07:00
Graydon Hoare
0f26b9c12d [Driver] <rdar://43955209> Remove obsolete/fragile batch mode 'repartitioning' code. 2018-10-09 13:31:41 -07:00
Jordan Rose
e224e31720 Break almost all dependencies of Driver on Frontend
- Sink OutputFileMap{.h,.cpp} and ReferenceDependencyKeys.h to Basic
- Remove unnecessary includes of Frontend.h.
2018-08-27 20:47:58 -07:00
Graydon Hoare
d5a9f44e4a [BatchMode] <rdar://41271283> Limit memory pressure on large modules. 2018-07-30 17:26:31 -07:00
Bob Wilson
8e330ee344 NFC: Fix indentation around the newly renamed LLVM_DEBUG macro.
Jordan used a sed command to rename DEBUG to LLVM_DEBUG. That caused some
lines to wrap and messed up indentiation for multi-line arguments.
2018-07-21 00:56:18 -07:00
Jordan Rose
cefb0b62ba Replace old DEBUG macro with new LLVM_DEBUG
...using a sed command provided by Vedant:

$ find . -name \*.cpp -print -exec sed -i "" -E "s/ DEBUG\(/ LLVM_DEBUG(/g" {} \;
2018-07-20 14:37:26 -07:00
Robert Widmann
81e9a3f660 [NFC] Drop PerformJobsState as a friend class
Make the coupling between PerformJobsState and Compilation indirect.
2018-07-10 14:57:01 -07:00
Jordan Rose
7d8e40b0bb Merge pull request #16362 from dabelknap/frontend_responsefile
Wrap Command Line Arguments in a Response File if System Limits are Exceeded

https://bugs.swift.org/browse/SR-4517
2018-06-21 16:31:20 -07:00
Austin Belknap
d71373c95c Encapsulate the logic for returning a response file or the full arg vector. 2018-06-15 15:16:35 -07:00
Graydon Hoare
dcc4373dab [BatchMode] <rdar://40526328> Emit signalled message (SIGINT) for batch constituents cancelled due to errors elsewhere. 2018-06-14 11:51:31 -07:00
Graydon Hoare
8be2c53b24 [BatchMode] Add -driver-batch-count to allow overriding batch count inferred by -j 2018-06-14 11:51:31 -07:00
Robert Widmann
da13d89c24 Move DummyQueue logic into buildTaskQueue 2018-05-29 21:58:27 -07:00
Robert Widmann
d7493dd0fa [NFC] Expose the TaskQueue as a Compilation Parameter
Shuffle the responsibility for creating the TaskQueue out of the
Compilation's internal job state object and into the driver.  Expose
a builder convenience function that handles the argument parsing.
2018-05-29 21:58:27 -07:00
Benjamin Herzog
ac10fb3462 [Driver] Added process information to emitted messages
When providing the -parseable-output flag to the swift compiler, it will provide json formatted messages about tasks that run.
I added some optional usage information in form of user time, system time and maxrss to the output. This can be used by other tools using the compiler to get some insights about time and memory usage.
Since the output does not longer match processes run (in batch mode), I also added a real_pid field so the client could reason about jobs that belong together if needed.

rdar://39798231
2018-05-25 13:12:57 -07:00
Austin Belknap
da19625711 Merge branch 'master' into frontend_responsefile 2018-05-11 14:11:34 -07:00
Graydon Hoare
a0b4f3f481 Merge pull request #16492 from graydon/rdar-39981525-batch-mode-show-every-job-via-quasi-pids
[BatchMode] <rdar://39981525> Emit parseable output for each job, using quasi-PIDs.
2018-05-10 23:03:52 -07:00
Jordan Rose
606ee29b74 [Driver] -enable-batch-mode implies -continue-building-after-errors (#16521)
The logic in 46b8ad3 to avoid putting certain diagnostics into
serialized diagnostic files only makes sense if

1. every diagnostic emitted in file A.swift while processing a
   different file B.swift would be emitted if we processed A.swift on
   its own

2. we actually do process A.swift on its own in the same build,
   or have previously done an incremental build and produced the same
   diagnostic

But the latter isn't actually guaranteed: if one batch job exits with
a failure status, the driver will exit as well, assuming there's no
point in continuing. Fortunately, we do have a flag that overrides
this behavior, -continue-building-after-errors.

(As noted in the patch, -continue-building-after-errors isn't *exactly*
what we want. But it's conservatively correct.)
2018-05-10 21:44:12 -07:00
Graydon Hoare
39fdf5d4e5 [BatchMode] <rdar://39981525> Emit parseable output for each job, using quasi-PIDs. 2018-05-10 20:34:33 -07:00
Austin Belknap
59769014fc Wrap command line arguments in a response file if they exceed system limits. 2018-05-03 14:09:07 -07:00
Graydon Hoare
c1682fa465 [Driver] <rdar://39504759> Continue demultiplexing subprocess output after initial read. 2018-04-18 11:52:11 -07:00
Nathan Hawes
0a4204e99f [Driver] Add -driver-filelist-threshold to set the number of inputs beyond which filelists are used
When generating a compiler invocation in driver::createCompilerInvocation()
we end up using filelists if the number of inputs is > 128 (to work around
command line arg limits). We never actually write them out though, and so
fail when parsing the frontend arguments that reference them.

As this function is called frequently by SourceKit and command line limits
aren't a concern here, this patch makes the 128 threshold value configurable
via a new -driver-filelist-threshold option. This is set to its maximum value
in driver::createCompilerInvocation() to ensure filelists aren't used. This
new option makes the existing -driver-use-filelists (that forces filelists to
be used) redundant as it's now equivalent to -driver-filelist-threshold=0.

Resolves rdar://problem/38231888
2018-04-12 05:24:03 -07:00
Rintaro Ishizaki
2cec8dfc99 [Driver] Factor out build-record filename construction
* Retrive filename for read and write at the same time.
* Read LastBuildTime directly from the build record.
2018-04-03 20:39:55 +09:00