SIL differentiability witnesses are a new top-level SIL construct mapping
"original" SIL functions to derivative SIL functions.
SIL differentiability witnesses have the following components:
- "Original" `SILFunction`.
- SIL linkage.
- Differentiability parameter indices (`IndexSubset`).
- Differentiability result indices (`IndexSubset`).
- Derivative `GenericSignature` representing differentiability generic
requirements (optional).
- JVP derivative `SILFunction` (optional).
- VJP derivative `SILFunction` (optional).
- "Is serialized?" bit.
This patch adds the `SILDifferentiabilityWitness` data structure, with
documentation, parsing, and printing.
Resolves TF-911.
Todos:
- TF-1136: upstream `SILDifferentiabilityWitness` serialization.
- TF-1137: upstream `SILDifferentiabilityWitness` verification.
- TF-1138: upstream `SILDifferentiabilityWitness` SILGen from
`@differentiable` and `@derivative` attributes.
- TF-20: robust mangling for `SILDifferentiabilityWitness` names.
For more context, see:
1. https://github.com/apple/swift/pull/29239 - original PR which introduced
the change, including an LLDB-side change.
2. The immediately preceding commit, which reverted this change.
3. https://github.com/apple/swift/pull/29350 which revealed some breakage
caused by the changes in PR 29239 (unrelated to printing).
The cross-module-optimization can change the linkage of a function to public. Then the SILLinkage is "out of sync" with the linkage derived from the AST. We need to make sure to read the correct SILLinkage from the module file.
This is needed for cross-module-optimization: CMO marks functions as inlinable. If a private or internal method is referenced from such an inlinable function, it must not be eliminated by dead function elimination after serialization (a method is basically an AbstractFunctionDecl).
For SILFunctions we can do this by simply setting the linkage, but for methods we need another mechanism.
When compiling the OnoneSupport library, the compiler checks for @_semantics("prespecialize.X") attributes to pre-specialize function X.
rdar://problem/48924409
The ownership kind is Any for trivial types, or Owned otherwise, but
whether a type is trivial or not will soon depend on the resilience
expansion.
This means that a SILModule now uniques two SILUndefs per type instead
of one, and serialization uses two distinct sentinel IDs for this
purpose as well.
For now, the resilience expansion is not actually used here, so this
change is NFC, other than changing the module format.
Each call site will soon have to think about passing in the right expansion
instead of just assuming the default will be OK. But there are now only a
few call sites left, because most have been refactored to use convenience
APIs that pass in the right resilience expansion already.
This lets the SIL optimizer reason about types (e.g. if a type is loadable) in specific functions.
For example, a type might be loadable in a resilient function, but not in an inlinable function.
All callers were doing the same thing here, so move it inside the
function. Also, change getRootNormalConformance(), which is deprecated,
to getRootConformance().
We've been running doxygen with the autobrief option for a couple of
years now. This makes the \brief markers into our comments
redundant. Since they are a visual distraction and we don't want to
encourage more \brief markers in new code either, this patch removes
them all.
Patch produced by
for i in $(git grep -l '\\brief'); do perl -pi -e 's/\\brief //g' $i & done
The reason why I am doing this is now that once we have the
OperandOwnershipKindMap I can write this optimization in a more robust,
aggressive manner. My hope is that I can eliminate all copy_value of guaranteed
parameters all of whose uses could accept a guaranteed parameter. This is given
to me by the OperandOwnershipKindMap.
Additionally, I found that the way the analysis was using the OwnershipVerifier
was not sound on non-ownership verified SIL. Rather than fix it, I just turned
it off for that case.
rdar://44667493
No functionality change. Unfortunately we still need the flag in
SILModule itself because of the ability to create an empty SILModule
and parse SIL into it incrementally, which can happen before there's
a FileUnit to use as the associated DeclContext instead of a
CompilerInstance's main module.
This was only used by the integrated REPL, and is now a dead option.
The "StartElem" option for performTypeChecking is still used for
reading SIL files, which have AST and SIL blocks alternate.
A few places around the compiler were checking for this module by its
name. The implementation still checks by name, but at least that only
has to occur in one place.
(Unfortunately I can't eliminate the string constant altogether,
because the implicit import for SwiftOnoneSupport happens by name.)
No functionality change.
Previously SILModule contained two different pathways for the deserializer to
send notifications that it had created functions:
1. A list of function pointers that were called when a function's body was
deserialized. This was added recently so that access enforcement elimination is
run on newly deserialized SIL code if we have already eliminated access
enforcement from the module.
2. SILModule::SerializationCallback. This is an implementation of the full
callback interface and is used by the SILModule to update linkage and other
sorts of book keeping.
To fix the pass manager notification infrastructure, I need to be able to send
notifications to a SILPassManager when deserializing. I also need to be able to
eliminate these callbacks when a SILPassManager is destroyed. These requirements
are incompatible with the current two implementations since: (2) is an
implementation detail of SILModule and (1) only notifies on function bodies
being deserialized instead of the creation of new declarations (what the caller
analysis wants).
Rather than adding a third group of callbacks, this commit refactors the
infrastructure in such a way that all of these use cases can use one
implementation. This is done by:
1. Lifting the interface of SerializedSILLoader::Callback into a base
notification protocol for deserialization called
DeserializationNotificationHandlerBase and its base no-op implementation into an
implementation of the aforementioned protocol:
DeserializationNotificationHandler.
2. Changing SILModule::SerializationCallback to implement
DeserializationNotificationHandler.
3. Creating a class called FunctionBodyDeserializationNotificationHandler that
takes in a function pointer and uses that to just override the
didDeserializeFunctionBody. This eliminates the need for the specific function
body deserialization list.
4. Replacing the state associated with the two other pathways with a single
DeserializationNotificationHandlerSet class that contains a set of
DeserializationNotificationHandler and chains notifications to them. This set
implements DeserializationNotificationHandlerBase so we know that its
implementation will always be in sync with DeserializationNotificationHandler.
rdar://42301529
This commit does not modify those APIs or their usage. It just:
1. Moves the APIs onto SILFunctionBuilder and makes SILFunctionBuilder a friend
of SILModule.
2. Hides the APIs on SILModule so all users need to use SILFunctionBuilder to
create/destroy functions.
I am doing this in order to allow for adding/removing function notifications to
be enforced via the type system in the SILOptimizer. In the process of finishing
off CallerAnalysis for FSO, I discovered that we were not doing this everywhere
we need to. After considering various other options such as:
1. Verifying after all passes that the notifications were sent correctly and
asserting. Turned out to be expensive.
2. Putting a callback in SILModule. This would add an unnecessary virtual call.
I realized that by using a builder we can:
1. Enforce that users of SILFunctionBuilder can only construct composed function
builders by making the composed function builder's friends of
SILFunctionBuilder (notice I did not use the word subclass, I am talking
about a pure composition).
2. Refactor a huge amount of code in SILOpt/SILGen that involve function
creation onto a SILGenFunctionBuilder/SILOptFunctionBuilder struct. Many of
the SILFunction creation code in question are straight up copies of each
other with small variations. A builder would be a great way to simplify that
code.
3. Reduce the size of SILModule.cpp by 25% from ~30k -> ~23k making the whole
file easier to read.
NOTE: In this commit, I do not hide the constructor of SILFunctionBuilder since
I have not created the derived builder structs yet. Once I have created those in
a subsequent commit, I will hide that constructor.
rdar://42301529
* SILModule::isVisibleExternally utility for VarDecls.
* Fix the SIL parser so it doesn't drop global variable decls.
This information was getting lost in SIL printing/parsing.
Some passes rely on it. Regardless of whether passes should rely on it,
it is totally unacceptable for the SIL passes to have subtle differences
in behavior depending on the frontend mode. So, if we don't want passes
to rely on global variable decls, that needs to be enforced by the API
independent of how the frontend is invoked or how SIL is serialized.
* Use custom DemangleOptions to lookup global variable identifiers.
Fixed-ABI means that we can do value operations on the type without
any metadata: value-allocations, copies, and destroys. It's currently
equivalent to being fixed-size, but (1) being fixed-size isn't useful
by itself at the SIL level and (2) you can imagine resilience or generics
micro-optimizations where there's like an attribute that tells us the
size of a type without actually telling us how to copy it. All types
are fixed-ABI except:
- layout-unconstrained generic types,
- resilient value types, and
- value types which contain a subobject of such a type (except within
indirect enum cases).
ABI-accessible means that we can perform value operations at all.
We might not be able to if the type is not fixed-ABI and it is private
to a different file (in non-WMO builds) or internal to a different
module, because in such cases we will not be able to access its metadata.
In general, we can't use such types `T` directly, but we may be able to
use types `C` that contain such types as subobjects. Furthermore, we
may be reasonably exposed to SIL that performs operations that treat `C`
as non-opaque, e.g. if `C` is frozen (as it will be by default for
modules in Swift 5). We can still achieve correctness in these cases
as long as we don't either:
- inline code that contains value operations on `T` or
- attempt to recursively expand a value operation on `T` into value
operations on its subobjects.
The SIL optimizer currently never tries to expand value operations on
objects in memory. However, IRGen always recursively expands value
operations on frozen types; that will be fixed in a follow-up patch.
The SIL verification that I've added here is definitely incomplete.
It was only used in a few tests. Those tests now use -emit-sil instead
of -emit-silgen, with some functions marked @_transparent and a few
CHECK: lines changed now that the mandatory optimizations get to run.
Code may end up indirectly using a witness table for a Clang-imported type by inlining code that used the conformance from another module, in which case we need to ensure we have a local definition at hand in the inlining module so we can have something to link against independently. This needs to be fixed from both sides:
- During serialization, serialize not only witness tables from the current module, but from Clang-imported modules too
- During deserialization, when the SILLinker walks a loaded module, ensure that all shared conformances get deserialized, including those from ApplyInsts and inherited/associated type protocol requirements.
Fixes rdar://problem/38687726.
Code may end up indirectly using a witness table for a Clang-imported type by inlining code that used the conformance from another module, in which case we need to ensure we have a local definition at hand in the inlining module so we can have something to link against independently. This needs to be fixed from both sides:
- During serialization, serialize not only witness tables from the current module, but from Clang-imported modules too, so that their definitions can be used by other modules that inline code from the current module
- During IRGen, when we emit a reference to a SILWitnessTable or SILFunction declaration with shared linkage, attempt to deserialize the definition on demand
Fixes rdar://problem/38687726.
This patch moves the ownership of profiling state from SILGenProfiling
to SILFunction, where it always belonged. Similarly, it moves ownership
of the profile reader from SILGenModule to SILModule.
The refactor sets us up to fix a few outstanding code coverage bugs and
does away with sad hacks like ProfilerRAII. It also allows us to locally
guarantee that a profile counter increment actually corresponds to the
SILFunction at hand.
That local guarantee causes a bugfix to accidentally fall out of this
refactor: we now set up the profiling state for delayed functions
correctly. Previously, we would set up a ProfilerRAII for the delayed
function, but its counter increment would never be emitted :(. This fix
constitutes the only functional change in this patch -- the rest is NFC.
As a follow-up, I plan on removing some dead code in the profiling
logic and fixing a few naming inconsistencies. I've left that for later
to keep this patch simple.
This patch moves the ownership of profiling state from SILGenProfiling
to SILFunction, where it always belonged. Similarly, it moves ownership
of the profile reader from SILGenModule to SILModule.
The refactor sets us up to fix a few outstanding code coverage bugs and
does away with sad hacks like ProfilerRAII. It also allows us to locally
guarantee that a profile counter increment actually corresponds to the
SILFunction at hand.
That local guarantee causes a bugfix to accidentally fall out of this
refactor: we now set up the profiling state for delayed functions
correctly. Previously, we would set up a ProfilerRAII for the delayed
function, but its counter increment would never be emitted :(. This fix
constitutes the only functional change in this patch -- the rest is NFC.
As a follow-up, I plan on removing some dead code in the profiling
logic and fixing a few naming inconsistencies. I've left that for later
to keep this patch simple.
This brings the capability from clang to save remarks in an external YAML files.
YAML files can be viewed with tools like the opt-viewer.
Saving the remarks is activated with the new option -save-optimization-record.
Similarly to -emit-tbd, I've only added support for single-compile mode for now.
In this case the default filename is determined by
getOutputFilenameFromPathArgOrAsTopLevel, i.e. unless explicitly specified
with -save-optimization-record-path, the file is placed in the directory of the
main output file as <modulename>.opt.yaml.
