This adds initial support for differentiation of functions that may produce `Error` result.
Essentially we wrap the pullback into `Optional` and emit a diamond-shape control flow pattern depending on whether the pullback value is available or not. VJP emission was modified to accommodate for this. In addition to this, some additional tricks are required as `try_apply` result is not available in the instruction parent block, it is available in normal successor basic block.
As a result we can now:
- differentiate an active `try_apply` result (that would be produced from `do ... try .. catch` constructions)
- `try_apply` when error result is unreachable (usually `try!` and similar source code constructs)
- Support (some) throwing functions with builtin differentiation operators. stdlib change will follow. Though we cannot support typed throws here (yet)
- Correctly propagate error types during currying around differentiable functions as well as type-checking for `@derivative(of:)` attribute, so we can register custom derivatives for functions producing error result
- Added custom derivative for `Optional.??` operator (note that support here is not yet complete as we cannot differentiate through autoclosures, so `x ?? y` works only if `y` is not active, e.g. a constant value).
Some fixes here and there
In #84704, some tests were XFAIL'ed since they've become failing. The
root cause of failures is lack of ownership info on the 2nd run of
AutoDiff closure specialization pass. Ownership info is required for the
pass run after #84704, so at the moment only the 1st run of the pass is
effective.
Several test cases still remain passing because for some cases we are
lucky and all the inlining required for specialization is done before
the 1st pass run. This patch enables such test cases back so we have at
least some test coverage.
Beside supporting OSSA, this change significantly simplifies the pass.
The main change is that instead of starting at a closure (e.g. `partial_apply`) and finding all call sites, we now start at a call site and look for closures for all arguments. This makes a lot of things much simpler, e.g. not so many intermediate data structures are required to track all the states.
I needed to remove the 3 unit tests because the things those tests were testing are not there anymore. However, the pass is tested with a lot of sil tests (and I added quite a few), which should give good test coverage.
The old ClosureSpecializer pass is still kept in place, because at that point in the pipeline we don't have OSSA, yet. Once we have that, we can replace the old pass withe the new one.
However, the autodiff closure specializer already runs in the OSSA pipeline and there the new changes take effect.
Previously, AutoDiff closure specialization pass was triggered only on
VJPs containing single basic block. However, the pass logic allows
running on arbitrary VJPs. This PR enables the pass for all VJPs
unconditionally. So, if the pullback corresponding to multiple-BB VJP
accepts some closures directly as arguments, these closures might become
specialized by the pass. Closures passed via payload of branch tracing
enum are not specialized - this is subject for future changes.
The PR contains several commits.
1. The thing named "call site" in the code is partial_apply of pullback
corresponding to the VJP. This might appear only once, so we drop
support for multiple "call sites".
2. Enhance existing SILOptimizer tests for the pass.
3. Add validation-tests for single basic block case.
4. The change itself - delete check against single basic block.
5. Add validation-tests for multiple basic block case.
6. Add SILOptimizer tests for multiple basic block case.
Consider an `@_alwaysEmitIntoClient` function and a custom derivative
defined
for it. Previously, such a combination resulted different errors under
different
circumstances.
Sometimes, there were linker errors due to missing derivative function
symbol -
these occurred when we tried to find the derivative in a module, while
it
should have been emitted into client's code (and it did not happen).
Sometimes, there were SIL verification failures like this:
```
SIL verification failed: internal/private function cannot be serialized or serializable: !F->isAnySerialized() || embedded
```
Linkage and serialization options for the derivative were not handled
properly,
and, instead of PublicNonABI linkage, we had Private one which is
unsupported
for serialization - but we need to serialize `@_alwaysEmitIntoClient`
functions
so the client's code is able to see them.
This patch resolves the issue and adds proper handling of custom
derivatives
of `@_alwaysEmitIntoClient` functions. Note that either both the
function and
its custom derivative or none of them should have
`@_alwaysEmitIntoClient`
attribute, mismatch in this attribute is not supported.
The following cases are handled (assume that in each case client's code
uses
the derivative).
1. Both the function and its derivative are defined in a single file in
one module.
2. Both the function and its derivative are defined in different files
which
are compiled to a single module.
3. The function is defined in one module, its derivative is defined in
another
module.
4. The function and the derivative are defined as members of a protocol
extension in two separate modules - one for the function and one for the
derivative. A struct conforming the protocol is defined in the third
module.
5. The function and the derivative are defined as members of a struct
extension in two separate modules - one for the function and one for the
derivative.
The changes allow to define derivatives for methods of `SIMD`.
Fixes#54445
<!--
If this pull request is targeting a release branch, please fill out the
following form:
https://github.com/swiftlang/.github/blob/main/PULL_REQUEST_TEMPLATE/release.md?plain=1
Otherwise, replace this comment with a description of your changes and
rationale. Provide links to external references/discussions if
appropriate.
If this pull request resolves any GitHub issues, link them like so:
Resolves <link to issue>, resolves <link to another issue>.
For more information about linking a pull request to an issue, see:
https://docs.github.com/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue
-->
<!--
Before merging this pull request, you must run the Swift continuous
integration tests.
For information about triggering CI builds via @swift-ci, see:
https://github.com/apple/swift/blob/main/docs/ContinuousIntegration.md#swift-ci
Thank you for your contribution to Swift!
-->
Type annotations for instruction operands are omitted, e.g.
```
%3 = struct $S(%1, %2)
```
Operand types are redundant anyway and were only used for sanity checking in the SIL parser.
But: operand types _are_ printed if the definition of the operand value was not printed yet.
This happens:
* if the block with the definition appears after the block where the operand's instruction is located
* if a block or instruction is printed in isolation, e.g. in a debugger
The old behavior can be restored with `-Xllvm -sil-print-types`.
This option is added to many existing test files which check for operand types in their check-lines.
The patch adds lowering of partial_apply instructions for coroutines.
This pattern seems to trigger a lot of type mismatch errors in IRGen, because
coroutine functions are not substituted in the same way as regular functions
(see the patch 07f03bd2 "Use pattern substitutions to consistently abstract
yields" for more details).
Other than that, lowering of partial_apply for coroutines is straightforward: we
generate another coroutine that captures arguments passed to the partial_apply
instructions. It calls the original coroutine for yields (first return) and
yields the resulting values. Then it calls the original function's continuation
for return or unwind, and forwards them to the caller as well.
After IRGen, LLVM's Coroutine pass transforms the generated coroutine (along with
all other coroutines) and eliminates llvm.coro.* intrinsics. LIT tests check
LLVM IR after this transformation.
Co-authored-by: Anton Korobeynikov <anton@korobeynikov.info>
Co-authored-by: Arnold Schwaighofer <aschwaighofer@apple.com>
Add a new mandatory BooleanLiteralFolding pass which constant folds conditional branches with boolean literals as operands.
```
%1 = integer_literal -1
%2 = apply %bool_init(%1) // Bool.init(_builtinBooleanLiteral:)
%3 = struct_extract %2, #Bool._value
cond_br %3, bb1, bb2
```
->
```
...
br bb1
```
This pass is intended to run before DefiniteInitialization, where mandatory inlining and constant folding didn't run, yet (which would perform this kind of optimization).
This optimization is required to let DefiniteInitialization handle boolean literals correctly.
For example in infinite loops:
```
init() {
while true { // DI need to know that there is no loop exit from this while-statement
if some_condition {
member_field = init_value
break
}
}
}
```
Introduce the notion of "semantic result parameter". Handle differentiation of inouts via semantic result parameter abstraction. Do not consider non-wrt semantic result parameters as semantic results
Fixes#67174
Linear maps are captured in vjp routine via callee-guaranteed partial apply and are passed as @owned references to the enclosing pullback that finally consumes them. Necessary retains are inserted by a partial apply forwarder.
However, this is not the case when the function being differentiated contains loops as heap-allocated context is used and bare pointer is captured by the pullback partial apply. As a result, partial apply forwarder does not retain the linear maps that are owned by a heap-allocated context, however, they are still treated as @owned references and therefore are released in the pullback after the first call. As a result, subsequent pullback calls release linear maps and we'd end with possible use-after-free.
Ensure we retain values when we load values from the context.
Reproducible only when:
* Loops (so, heap-allocated context)
* Pullbacks of thick functions (so context is non-zero)
* Multiple pullback calls
* Some cleanup while there
Fixes#64257
This essentially passes the members of a linear map tuple as individual arguments. It yields few nice simplifications:
* No linear map tuples at all for getters / setters
* No tuple formation / deconstruction around pullbacks
* Pullbacks with loops still use heap-allocated tuples
The changes are intentionally were made close to the original implementation w/o possible simplifications to ease the review
Fixes#63207, supersedes #63379 (and fixes#63234)
Apparently #60467 changed the semantics of store_borrow as it started to produce a value. This change was not documented in SIL spec and not all places were updated to new semantics.
Now the adjoint of store_borrow should be generated for the value of instruction itself, not the destination address
@differentiable function is actually a triple (function, jvp, vjp). Previously normal thick function value witness table was used. As a result, for example, only function was copied, but none of differential components.
This was the cause of uninitialized memory accesses and subsequent segfaults.
Should fix now unavailable TF-1122
In `TangentBuilder` temporary allocations are emitted in order to make generic calls to `AdditiveArithmetic.zero` and `AdditiveArithmetic.+`. When we perform accumulation of two adjoint values that correspond to a lexical value in the original function, the stack allocations require a debug variable, as we use the original instruction's location. However such a debug variable wouldn't really make sense as these temporary allocations do not represent the original variable's adjoint. With lexical borrow scopes, we hit a crasher when creating temporary allocations when accumulating adjoints for lexical `begin_borrow` instructions.
This PR makes `TangentBuilder`'s temporary allocations use an auto-generated location. The existing debug info emission behavior is not changed, as we still emit `debug_value` when materializing an `AdjointValue` after any necessary adjoint accumulation.
The effect of passing -enable-copy-propagation is both to enable the
CopyPropagation pass to shorten object lifetimes and also to enable
lexical lifetimes to ensure that object lifetimes aren't shortened while
a variable is still in scope and used.
Add a new flag, -enable-lexical-borrow-scopes=true to override
-enable-copy-propagation's effect (setting it to ::ExperimentalLate) on
SILOptions::LexicalLifetimes that sets it to ::Early even in the face of
-enable-copy-propagation. The old flag -disable-lexical-lifetimes is
renamed to -enable-lexical-borrow-scopes=false but continues to set that
option to ::Off even when -enable-copy-propagation is passed.
The latest Long Term Support NDK finally removed binutils, including the bfd/gold
linkers and libgcc. This simplifies our Android support, including making lld the
default linker for Android. Disable three reflection tests that now fail, likely
related to issues with swift-reflection-dump and switching to lld.
The latest Long Term Support NDK finally removed binutils, including the bfd/gold
linkers and libgcc. This simplifies our Android support, including making lld the
default linker for Android. Disable three reflection tests that now fail, likely
related to issues with swift-reflection-dump and switching to lld.
Also, add the libatomic dependency for Android armv7, just as on linux.
1. When calculating the differential type of an original function with an inout parameter and when the inout parameter has a type parameter, the inout parameter should get a generic parameter in the subst generic signature of the differential but it currently doesn't. This causes SILGen to attempt to reabstract the differential value in the JVP protocol witness thunk, whilst the generic signature is lacking requirements, leading to a requirement machine error. This patch fixes the calculation so that the JVP's result type (the differential type) always matches the witness thunk's result type.
Wrong type:
```swift
sil private [transparent] [thunk] [ossa] @... <τ_0_0 where τ_0_0 : Differentiable> (...) -> @owned @callee_guaranteed @substituted <τ_0_0, τ_0_1> (@in_guaranteed τ_0_0) -> @out τ_0_1 for <τ_0_0.TangentVector, τ_0_0.TangentVector> {
%6 = differentiable_function_extract [jvp] %5 : $@differentiable(reverse) @convention(method) <τ_0_0 where τ_0_0 : Differentiable> (@in_guaranteed τ_0_0, @noDerivative @inout τ_0_0, @noDerivative SR_13305_Struct) -> () // user: %7
HERE ====> %7 = apply %6<τ_0_0>(%0, %1, %3) : $@convention(method) <τ_0_0 where τ_0_0 : Differentiable> (@in_guaranteed τ_0_0, @inout τ_0_0, SR_13305_Struct) -> @owned @callee_guaranteed @substituted <τ_0_0> (@in_guaranteed τ_0_0) -> @out τ_0_0 for <τ_0_0.TangentVector>
```
Should be:
```swift
%7 = apply %6<τ_0_0>(%0, %1, %3) : $@convention(method) <τ_0_0 where τ_0_0 : Differentiable> (@in_guaranteed τ_0_0, @inout τ_0_0, SR_13305_Struct) -> @owned @callee_guaranteed @substituted <τ_0_0, τ_0_1> (@in_guaranteed τ_0_0) -> @out τ_0_1 for <τ_0_0.TangentVector, τ_0_0.TangentVector>
```
2. `TypeConverter::makeConstantInterfaceType` is not passing down the derivative generic signature to `SILFunctionType::getAutoDiffDerivativeFunctionType` for class methods, and this was caught by RequirementMachine during vtable emission. This patch fixes that.
Partially resolves rdar://82549134. The only remaining tests that require `-requirement-machine=off` are SILOptimizer/semantic_member_accessors_sil.swift and SILOptimizer/differentiation_diagnostics.swift which I will fix next. Then I'll do a proper fix for workaround #39416.
The SIL type lowering logic for AutoDiff gets the substituted generic signature
mixed up with the invocation generic signature, so it tries to ask questions
about DependentMemberTypes in a signature with no requirements. This triggers
assertions when the requirement machine is enabled.
Disable the requirement machine until this is fixed.
