Generate frontend commands with -filelist in them. This isn't actually
implemented yet, but we can start testing at this point.
Part 1 of https://bugs.swift.org/browse/SR-280.
Previously jobs had to grovel this information out of the raw argument
list, which dropped the types we had inferred on input files. This
makes things more consistent across the compiler, though arguably we
should be able to designate "primary" and "non-primary" inputs on a
per-action basis rather than resorting to "global" state.
Use this new information to stop passing object file inputs to the
Swift frontend, fixing rdar://problem/23213785.
The list wouldn't have to live on the Compilation, but I'm going to use
it to fix SR-280 / rdar://problem/23878192 as well.
Implementing this for the general case either requires duplicating the
existing environment, or modifying TaskQueue to support "extra environment"
settings. Since we don't actually have any use cases for this yet, I'm
leaving it unimplemented for now.
Rest of rdar://problem/23588774
For now, just error out at the end of the build if something was modified,
forcing a rebuild. The incremental logic should get that rebuild right.
We could automatically restart the rebuild, but that could lead to infinite
compilation if the user continues to edit an important file.
DerivedArgList has a pointer to the InputArgList it came from, so we can't
just std::move it around. Put most of the driver back the way it was, with
small changes to clarify ownership.
Swift SVN r31811
The "Tool" abstraction wasn't buying us enough to deserve the added
complexity. Now a ToolChain turns Actions into Jobs, and every helper
tool is searched for relative to Swift first. Much simpler.
Swift SVN r31563
This was just a wrapper around SmallVector that optionally owned the Job pointers
in it. Now that all Jobs are owned by the Compilation, we don't have to worry
about this any more.
No functionality change.
Swift SVN r29668
Previously, we would process all of a job's dependencies separately before
even scheduling it, and we wouldn't interleave dependencies from different
jobs. This meant (a) more overhead than necessary, and more importantly
(b) -embed-bitcode builds weren't being parallelized.
rdar://problem/21129029
Swift SVN r29665
...not if it's newer than its output .o file. This handles cases where the
object file is generated too quickly (rdar://problem/19404140) or when you
revert to a previous version of the file, mtime intact (rdar://problem/19720146).
There's a lot of test churn here; the only real new test is the backwards
mtime update in one-way.swift.
Swift SVN r29584
No test case; this is apparently hitting Enrico but not reproducing in
any obvious way for me. Nevertheless, it /could/ be an issue, so let's be
conservative.
rdar://problem/20402875
Swift SVN r26882
This is instead of verifying that it's 0 after running a round.
The previous way would cause it to assert if it the blocking task
was at a higher level of the stack than the current level, and
thus in a different TaskQueue. This way we just verify that
no new tasks are left over.
Swift SVN r26501
If a build fails in the middle, we try to determine which other files need
to be rebuilt. However, we may not be able to do that as precisely if the
dependency graph itself is incomplete. In this case, just be conservative
and assume we need to rebuild everything. We may want to revisit this in
the future with a more-aggressive-but-still-safe bound.
This was manifesting itself as an assertion failure, trying to pull
information from the graph that wasn't there.
rdar://problem/19640006
Swift SVN r24823
If certain command-line arguments change, the results of the last
compilation aren't reusable, i.e. we can't do an incremental build.
Do a full rebuild when we detect that this happens.
(Which command-line options? Conservatively assume all of them, /except/
those with the new DoesNotAffectIncrementalBuild flag in Options.td.)
Swift SVN r24385
After we've added all files that are explicitly out of date, check the set
of external dependencies in the graph and see if any of them have been
modified more recently than the oldest object file (or, if an object file
is missing, the corresponding source file; see previous commit). If so,
mark that external dependency as dirty and schedule anything touched by that.
In practice, due to the way external dependencies are collected, this will
almost always lead to a full rebuild. However, the way this is structured
is semantically correct even if that were not the case: an external
dependency is a cascading dependency like any other.
One particular point of information: normal cascading dependencies can be
discovered retroactively, i.e. after a particular source file has already
been compiled. Can that happen for external dependencies? In theory, yes,
due to the leakiness of imports within a module. (If a.swift loads a module
with an extension on String, that extension will be visible to b.swift in
the same module, even though it shouldn't be.) But that's true even if the
external dependency /hasn't/ changed. Given that it's something we consider
a flaw (if low-priority: rdar://problem/16154294), and that it would be
harmless in most actual circumstances, I don't think we should actually
force a full rebuild if one file's imports change.
This completes rdar://problem/19270920
Swift SVN r24337
r23968 wrote out a record of which source files were included in a build,
and whether they were succesfully compiled or not...and if not, whether
they were out of date because of a cascading or non-cascading dependency.
This commit uses that information to decide what files might need to be
rebuilt even if a particular input doesn't change and doesn't appear to
have any changed dependencies. The two interesting cases are:
- A file was going to be built last time, but the build was halted
because of an error. Build it this time.
- One of the files was removed and thus we've lost a source of dependency
information; rebuild everything!
rdar://problem/19270980
Swift SVN r24018
"private" is a very overloaded term already. "Cascading" instead of
"non-private" is a bit more clear about what will happen with this sort
of lookup.
No functionality change. There are some double negatives I plan to clean
up in the next commit, but this one was supposed to be very mechanical.
Swift SVN r23969
This is important because we might get part-way through the full
compilation, overwriting swiftdeps files as we go, and then encounter an
error. We don't want to lose information about any decls that have been
removed since the previous compile, so we propagate forward the information
we already have by saving it to a "build record" file.
More simply, this is necessary to track when a file is removed from a target.
The next commit will handle reading in this file at the start of a build.
Swift SVN r23968
Previously, the driver waited for the first set of known-dirty jobs to
finish before doing /any/ dependency analysis. This was correct, but
could take a lot longer (consider waiting for one touched file to compile
and then finding out that it affects three others, when all four could
have been built in parallel). This only affects the incremental build.
Swift SVN r23967
Specifically, we care about the case where a job is run because of a private
dependency, and then a non-private dependency turns out to be dirty. In
this case, we still need to make sure to build all downstream files.
With this the driver support for private dependencies should be complete
and correct.
Swift SVN r23853
- Add flags to dependency entries in DependencyGraph.
- Don't traverse past private dependencies in markTransitive.
- Only mark dependent jobs after a build if the build was triggered
(a) explicitly (because the file is out of date), or
(b) because of a non-private dependency.
This still isn't fully correct because of new non-private dependencies
discovered /after/ building an individual file, but it's on the way there.
Solving that problem will require tracking which dependencies have already
been marked dirty (next commit).
Swift SVN r23852
- Give loadWithPath an enum result that includes "NeedsRebuilding".
This will be returned when a new dependency is discovered that
retroactively affects the graph.
- Don't clear the "provides" set for a node when it gets reloaded;
just append to it. This lets us avoid calling markTransitive twice.
- Use proper types for "depends" and "provides" entries instead of std::pair.
- Use swift::OptionSet instead of a manual bitmask.
- Use separate "depends" and "provides" callbacks when parsing dependency
files.
No expected functionality change.
Swift SVN r23851
This teaches the driver's Compilation to not run jobs where the base input
is older than the main output (r23221) when we're tracking dependencies.
After a compile command finishes, anything that depended on the file that
just got compiled will get scheduled.
This has the nice side effect of trying to rebuild changed files first.
The tests here aren't really testing the dependency graph yet, because the
files don't include any dependencies. I'll be adding several more test
scenarios in the next few commits.
Part of rdar://problem/15353101
Swift SVN r23273
...and rename Command to Job (previously the name of the base class).
We never generated job lists directly contained in other job lists, so
let's not even worry about this case. We may some day need to break Job
out into separate subclasses (Clang has Command and FallbackCommand in
addition to JobList), but we should be able to keep the list separate.
No intended functionality change.
Swift SVN r23144
Instead, if we can't schedule a command, record why it was blocked. When the
blocking command completes, we then try to reschedule everything that was
blocked on it.
This is also more robust for cross-job-list dependencies---things like the
link job depending on the merge-module job and both depending on compile jobs.
Swift SVN r23143
When "-parseable-output" is passed to the driver, it will now emit output in a
parseable format. (This format is described in docs/DriverParseableOutput.rst,
which was added in a previous commit.)
This is achieved by adding four functions (one for each kind of message). These
are in a new swift::driver::parseable_output namespace, and given the right
parameters, will output the appropriate message in JSON to the given
llvm::raw_ostream. These functions are then called by
Compilation::performJobsInList:
- "began" messages are emitted by the taskBegan callback
- "finished" messages are emitted by the taskFinished callback
- "signalled" messages are emitted by the taskSignalled callback
- "skipped" messages are emitted by the handleCommandWhichDoesNotNeedToExecute
lambda
(Note that "skipped" messages will not be emitted in practice, since the driver
does not yet support partial compilation.)
This fixes <rdar://problem/15958329>.
Swift SVN r20873
This level is selected by -parseable-output. This flag is only accepted by
swiftc, since it does not make sense for any of the interactive modes.
(Currently, this level prints out the same information as Verbose, with a
"Command: " string prepended.)
Additionally, in Compilation::performJobs, set RequiresBufferedOutput to true if
parseable output was requested, since parseable output will require buffered
output.
Part of <rdar://problem/15958329>.
Swift SVN r20872
Previously, Verbose output was not produced for this case; it is now, and this
is achieved by making performSingleCommand() a private method on Compilation
instead of a static function.
Swift SVN r20850
If a temporary file is mentioned in the output map, it is also preserved.
We can make a nicer -save-temps later, but for now this will at least stop
leaving random files in /var/tmp.
<rdar://problem/16874893>
Swift SVN r17850
Added a TaskSignalledCallback to TaskQueue, which will be called instead of
TaskFinishedCallback if the task exited abnormally.
In Unix/TaskQueue.inc, check WIFSIGNALED if the task did not pass WIFEXITED,
and call the TaskSignalledCallback if necessary. In Default/TaskQueue.inc, check
for a return code of -2; if present, call the TaskSignalledCallback instead of
the TaskFinishedCallback.
Updated Compilation to pass a TaskSignalledCallback.
Added diagnostics to indicate when a command signalled as well as when a command
failed with either poor diagnostics or a non-1 exit code. (These match Clang’s
diagnostics.) Added tests to ensure these diagnostics are emitted when the
frontend crashes or fails an assertion (if assertions are available).
This fixes <rdar://problem/16012199>.
Swift SVN r13654
