This was just a wrapper around SmallVector that optionally owned the Job pointers
in it. Now that all Jobs are owned by the Compilation, we don't have to worry
about this any more.
No functionality change.
Swift SVN r29668
Previously, we would process all of a job's dependencies separately before
even scheduling it, and we wouldn't interleave dependencies from different
jobs. This meant (a) more overhead than necessary, and more importantly
(b) -embed-bitcode builds weren't being parallelized.
rdar://problem/21129029
Swift SVN r29665
...not if it's newer than its output .o file. This handles cases where the
object file is generated too quickly (rdar://problem/19404140) or when you
revert to a previous version of the file, mtime intact (rdar://problem/19720146).
There's a lot of test churn here; the only real new test is the backwards
mtime update in one-way.swift.
Swift SVN r29584
No test case; this is apparently hitting Enrico but not reproducing in
any obvious way for me. Nevertheless, it /could/ be an issue, so let's be
conservative.
rdar://problem/20402875
Swift SVN r26882
This is instead of verifying that it's 0 after running a round.
The previous way would cause it to assert if it the blocking task
was at a higher level of the stack than the current level, and
thus in a different TaskQueue. This way we just verify that
no new tasks are left over.
Swift SVN r26501
If a build fails in the middle, we try to determine which other files need
to be rebuilt. However, we may not be able to do that as precisely if the
dependency graph itself is incomplete. In this case, just be conservative
and assume we need to rebuild everything. We may want to revisit this in
the future with a more-aggressive-but-still-safe bound.
This was manifesting itself as an assertion failure, trying to pull
information from the graph that wasn't there.
rdar://problem/19640006
Swift SVN r24823
If certain command-line arguments change, the results of the last
compilation aren't reusable, i.e. we can't do an incremental build.
Do a full rebuild when we detect that this happens.
(Which command-line options? Conservatively assume all of them, /except/
those with the new DoesNotAffectIncrementalBuild flag in Options.td.)
Swift SVN r24385
After we've added all files that are explicitly out of date, check the set
of external dependencies in the graph and see if any of them have been
modified more recently than the oldest object file (or, if an object file
is missing, the corresponding source file; see previous commit). If so,
mark that external dependency as dirty and schedule anything touched by that.
In practice, due to the way external dependencies are collected, this will
almost always lead to a full rebuild. However, the way this is structured
is semantically correct even if that were not the case: an external
dependency is a cascading dependency like any other.
One particular point of information: normal cascading dependencies can be
discovered retroactively, i.e. after a particular source file has already
been compiled. Can that happen for external dependencies? In theory, yes,
due to the leakiness of imports within a module. (If a.swift loads a module
with an extension on String, that extension will be visible to b.swift in
the same module, even though it shouldn't be.) But that's true even if the
external dependency /hasn't/ changed. Given that it's something we consider
a flaw (if low-priority: rdar://problem/16154294), and that it would be
harmless in most actual circumstances, I don't think we should actually
force a full rebuild if one file's imports change.
This completes rdar://problem/19270920
Swift SVN r24337
r23968 wrote out a record of which source files were included in a build,
and whether they were succesfully compiled or not...and if not, whether
they were out of date because of a cascading or non-cascading dependency.
This commit uses that information to decide what files might need to be
rebuilt even if a particular input doesn't change and doesn't appear to
have any changed dependencies. The two interesting cases are:
- A file was going to be built last time, but the build was halted
because of an error. Build it this time.
- One of the files was removed and thus we've lost a source of dependency
information; rebuild everything!
rdar://problem/19270980
Swift SVN r24018
"private" is a very overloaded term already. "Cascading" instead of
"non-private" is a bit more clear about what will happen with this sort
of lookup.
No functionality change. There are some double negatives I plan to clean
up in the next commit, but this one was supposed to be very mechanical.
Swift SVN r23969
This is important because we might get part-way through the full
compilation, overwriting swiftdeps files as we go, and then encounter an
error. We don't want to lose information about any decls that have been
removed since the previous compile, so we propagate forward the information
we already have by saving it to a "build record" file.
More simply, this is necessary to track when a file is removed from a target.
The next commit will handle reading in this file at the start of a build.
Swift SVN r23968
Previously, the driver waited for the first set of known-dirty jobs to
finish before doing /any/ dependency analysis. This was correct, but
could take a lot longer (consider waiting for one touched file to compile
and then finding out that it affects three others, when all four could
have been built in parallel). This only affects the incremental build.
Swift SVN r23967
Specifically, we care about the case where a job is run because of a private
dependency, and then a non-private dependency turns out to be dirty. In
this case, we still need to make sure to build all downstream files.
With this the driver support for private dependencies should be complete
and correct.
Swift SVN r23853
- Add flags to dependency entries in DependencyGraph.
- Don't traverse past private dependencies in markTransitive.
- Only mark dependent jobs after a build if the build was triggered
(a) explicitly (because the file is out of date), or
(b) because of a non-private dependency.
This still isn't fully correct because of new non-private dependencies
discovered /after/ building an individual file, but it's on the way there.
Solving that problem will require tracking which dependencies have already
been marked dirty (next commit).
Swift SVN r23852
- Give loadWithPath an enum result that includes "NeedsRebuilding".
This will be returned when a new dependency is discovered that
retroactively affects the graph.
- Don't clear the "provides" set for a node when it gets reloaded;
just append to it. This lets us avoid calling markTransitive twice.
- Use proper types for "depends" and "provides" entries instead of std::pair.
- Use swift::OptionSet instead of a manual bitmask.
- Use separate "depends" and "provides" callbacks when parsing dependency
files.
No expected functionality change.
Swift SVN r23851
This teaches the driver's Compilation to not run jobs where the base input
is older than the main output (r23221) when we're tracking dependencies.
After a compile command finishes, anything that depended on the file that
just got compiled will get scheduled.
This has the nice side effect of trying to rebuild changed files first.
The tests here aren't really testing the dependency graph yet, because the
files don't include any dependencies. I'll be adding several more test
scenarios in the next few commits.
Part of rdar://problem/15353101
Swift SVN r23273
...and rename Command to Job (previously the name of the base class).
We never generated job lists directly contained in other job lists, so
let's not even worry about this case. We may some day need to break Job
out into separate subclasses (Clang has Command and FallbackCommand in
addition to JobList), but we should be able to keep the list separate.
No intended functionality change.
Swift SVN r23144
Instead, if we can't schedule a command, record why it was blocked. When the
blocking command completes, we then try to reschedule everything that was
blocked on it.
This is also more robust for cross-job-list dependencies---things like the
link job depending on the merge-module job and both depending on compile jobs.
Swift SVN r23143
When "-parseable-output" is passed to the driver, it will now emit output in a
parseable format. (This format is described in docs/DriverParseableOutput.rst,
which was added in a previous commit.)
This is achieved by adding four functions (one for each kind of message). These
are in a new swift::driver::parseable_output namespace, and given the right
parameters, will output the appropriate message in JSON to the given
llvm::raw_ostream. These functions are then called by
Compilation::performJobsInList:
- "began" messages are emitted by the taskBegan callback
- "finished" messages are emitted by the taskFinished callback
- "signalled" messages are emitted by the taskSignalled callback
- "skipped" messages are emitted by the handleCommandWhichDoesNotNeedToExecute
lambda
(Note that "skipped" messages will not be emitted in practice, since the driver
does not yet support partial compilation.)
This fixes <rdar://problem/15958329>.
Swift SVN r20873
This level is selected by -parseable-output. This flag is only accepted by
swiftc, since it does not make sense for any of the interactive modes.
(Currently, this level prints out the same information as Verbose, with a
"Command: " string prepended.)
Additionally, in Compilation::performJobs, set RequiresBufferedOutput to true if
parseable output was requested, since parseable output will require buffered
output.
Part of <rdar://problem/15958329>.
Swift SVN r20872
Previously, Verbose output was not produced for this case; it is now, and this
is achieved by making performSingleCommand() a private method on Compilation
instead of a static function.
Swift SVN r20850
If a temporary file is mentioned in the output map, it is also preserved.
We can make a nicer -save-temps later, but for now this will at least stop
leaving random files in /var/tmp.
<rdar://problem/16874893>
Swift SVN r17850
Added a TaskSignalledCallback to TaskQueue, which will be called instead of
TaskFinishedCallback if the task exited abnormally.
In Unix/TaskQueue.inc, check WIFSIGNALED if the task did not pass WIFEXITED,
and call the TaskSignalledCallback if necessary. In Default/TaskQueue.inc, check
for a return code of -2; if present, call the TaskSignalledCallback instead of
the TaskFinishedCallback.
Updated Compilation to pass a TaskSignalledCallback.
Added diagnostics to indicate when a command signalled as well as when a command
failed with either poor diagnostics or a non-1 exit code. (These match Clang’s
diagnostics.) Added tests to ensure these diagnostics are emitted when the
frontend crashes or fails an assertion (if assertions are available).
This fixes <rdar://problem/16012199>.
Swift SVN r13654
Added an OutputLevel enum, which is used by Compilation to determine what kind
of output it should emit when performing Jobs.
In Driver::buildCompilation(), set the default OutputLevel for the Compilation
to Normal, but set it to Verbose if -v was passed. This means that, by default,
the driver will only print subtask output; this matches Clang's behavior.
Swift SVN r13255
Added DiagnosticsDriver.def and DiagnosticsDriver.h for driver-only diagnostics.
(Diagnostics which are shared with the frontend remain in
DiagnosticsFrontend.{def,h}.)
Added a DiagnosticEngine& to Compilation, so that it can emit diagnostics for
events which occur while performing Jobs.
Replaced all of the locations where we were manually printing error messages to
emitting real diagnostics, adding diagnostics if necessary.
Updated Driver::buildCompilation() so that it fails early if any errors were
encountered.
Updated test/Driver/actions.swift to pass a -module-name for multi-input tests.
Swift SVN r13175
Added swift::ExecuteInPlace(), which on Unix acts as a wrapper for execv()
and execve(). On other platforms, swift::ExecuteInPlace() is a wrapper for
llvm::sys::ExecuteAndWait(), so callers must be prepared for ExecuteInPlace()
to return in a non-error situation.
Added support in Compilation::performJobs() to detect that the Compilation has
exactly one Command to run. If that's the case, and buffered output isn't
required, execute that Command using swift::ExecuteInPlace() (instead of
creating a TaskQueue, which may unconditionally buffer output).
This change will allow the driver to invoke the frontend's REPL and immediate
modes without buffering output (and, on Unix, without a separate process being
spawned).
Also updated test/Driver/basic.swift to emit output with these changes in place.
Swift SVN r12544
Across recursive calls to performJobsInList(), keep track of ScheduledCommands
and FinishedCommands, so we can do the following:
- Don’t schedule a Command which is already scheduled (since it may be
scheduled as a peer before we try to schedule it as an input).
- Don’t schedule a Command whose inputs have not all finished.
Swift SVN r12444
This instructs the driver to skip task execution by using a DummyTaskQueue in Compilation::performJobsInList().
This option should generally only be used for testing, as DummyTaskQueue may provide fake buffered output.
Swift SVN r12350
In Compilation, perform Jobs by scheduling Commands which need to be run (currently
all Commands, but this can be refined later) on TaskQueues by doing the following:
1. Perform all inputs to all Jobs in the current JobList. (Treat a JobList as its
own inputs for scheduling purposes.)
2. After each Command's inputs run, check to see if that Command needs to run.
If so, schedule it for execution. (Currently, all Commands always need to run.)
3. Set up a TaskFinishedCallback which will check to see if any other Commands need
to run as a result of executing a particular Command. (Currently, all Commands
depend on all other Commands.) If a Command needs to run and it is not already
scheduled for execution, schedule it for execution.
4. Ask the TaskQueue to execute. (This will call the TaskFinishedCallback as
necessary).
Swift SVN r12060
Actual execution of Jobs is still performed under-the-covers by JobList::run(), but this abstraction will allow the Compilation to execute Jobs in parallel.
Swift SVN r11334
The driver does not yet support parallel execution, but this sets the Compilation up with the necessary information to execute commands in parallel.
Swift SVN r11333
- Added a couple of new targets:
- libswiftDriver, which contains most of the driver implementation
- swift_driver, which produces the actual executable
- Added centralized version information into libswiftBasic.
- Added a new "Driver Design & Internals" document, which currently describes
the high-level design of the Swift driver.
- Implemented an early version of the functionality of the driver, including
versions of the Parse, Pipeline, Bind, Translate, and Execute driver stages.
Parse, Pipeline, and Bind are largely implemented; Translate and Execute are
early placeholders. (Translate produces "swift_driver --version" and "ld -v"
commands, while Execute performs all subtasks sequentially, rather than in
parallel.)
This is just the starting point for the Swift driver. Tests for the existing
behavior are forthcoming.
Swift SVN r10933
