Fix the non-deterministic .swiftdeps output. In contrary to the comments
in the `FineGrainedDependencies.h`, the non-determinism is not because
of the use of unordered_* data structure there. Those data structures
are not traversed so removing all the unused traversing methods to avoid
the confusion.
The true reason for the non-determinism is all the DenseSet in the
Evaluator dependency tracking, that causes the FineGrainedDependencies
to see arbitrary ordering. Use `SetVector` instead to keep track of the
insertion order to make dependency output to be deterministic.
rdar://104876331
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Using a virutal output backend to capture all the outputs from
swift-frontend invocation. This allows redirecting and/or mirroring
compiler outputs to multiple location using different OutputBackend.
As an example usage for the virtual outputs, teach swift compiler to
check its output determinism by running the compiler invocation
twice and compare the hash of all its outputs.
Virtual output will be used to enable caching in the future.
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`
The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.
rdar://102362022
This was a hole in the existing dependency tracking infrastructure. David managed to discover a way to exploit this bug to get a miscompile in rdar://74583179. There, members of extensions were not counted towards the interface hash of a type and so mutating them could lead to e.g. the wrong declaration being selected in an overload set.
To start tracking extensions, we need to add two new kinds of arcs:
1) A nominal arc to the extension
2) A member arc to the extension
Unfortunately, extensions are also unique in Swift in that they do not have a name to allow us to unique them. Luckily, we do have a way of identifying extensions: their fingerprint. These arcs are therefore emitted with the extended nominal type and the fingerprint of the extension as their context. This effectively invents a new nominal type for every extension.
It can be quite difficult to tell at a glance just how any particular decl is going to be converted into a key. The space of available template specializations is also 2-dimensional which adds an additional level of difficulty when the time comes to extend or refactor any of them. Unroll all of the templates into a builder that coalesces the commonalities of the ways DependencyKeys are built to combat this.
T *const does not prevent logically non-const accesses to the underlying data, it merely indicates that the pointer value itself is const. This modifier can be cast off by a copy, so it's not generally what you want here. Switch to const T * instead.
Invert the responsibility of the entrypoint so that FrontendTool is directing the actual serialization work. The entrypoint now solely exists to construct a dependency graph.
While I'm here, prepare the way for serializing dependency graphs for modules by optimistically modeling a ModuleOrSourceFile input.
Restructure fine-grained-dependencies to enable unit testing
Get frontend to emit correct swiftdeps file (fine-grained when needed) and only emit dot file for -emit-fine-grained-dependency-sourcefile-dot-files
Use deterministic order for more information outputs.
Set EnableFineGrainedDependencies consistently in frontend.
Tolerate errors that result in null getExtendedNominal()
Fix memory issue by removing node everywhere.
Break up print routine
Be more verbose so it will compile on Linux.
Sort batchable jobs, too.
Restructure fine-grained-dependencies to enable unit testing
Get frontend to emit correct swiftdeps file (fine-grained when needed) and only emit dot file for -emit-fine-grained-dependency-sourcefile-dot-files
Use deterministic order for more information outputs.
Set EnableFineGrainedDependencies consistently in frontend.
Tolerate errors that result in null getExtendedNominal()
Fix memory issue by removing node everywhere.
Break up print routine
Be more verbose so it will compile on Linux.
Sort batchable jobs, too.
