This adds initial support for differentiation of functions that may produce `Error` result.
Essentially we wrap the pullback into `Optional` and emit a diamond-shape control flow pattern depending on whether the pullback value is available or not. VJP emission was modified to accommodate for this. In addition to this, some additional tricks are required as `try_apply` result is not available in the instruction parent block, it is available in normal successor basic block.
As a result we can now:
- differentiate an active `try_apply` result (that would be produced from `do ... try .. catch` constructions)
- `try_apply` when error result is unreachable (usually `try!` and similar source code constructs)
- Support (some) throwing functions with builtin differentiation operators. stdlib change will follow. Though we cannot support typed throws here (yet)
- Correctly propagate error types during currying around differentiable functions as well as type-checking for `@derivative(of:)` attribute, so we can register custom derivatives for functions producing error result
- Added custom derivative for `Optional.??` operator (note that support here is not yet complete as we cannot differentiate through autoclosures, so `x ?? y` works only if `y` is not active, e.g. a constant value).
Some fixes here and there
1. When differentiable nested function (closure) is specialized by capture promotion pass ensure we generate a differentiability witness for the specialized function as well. Ensure the original witness is removed if the original function becomes dead.
2. Differentiability witnesses for a function could originate either from its `@differentiable` attribute or from explicit `@derivative(of:)` attribute on the derivative. In the latter case the derivative itself might not be emitted, while original function is (e.g. original function is `@inlineable`, but derivative is `@usableFromInline`). Previously both cases were handled only when function body was emitted. As a result we missed witness in the aforementioned case. Ensure the
differentiability witness originating from `@derivative(of:)` is emitted even if we're not going to emit body of the derivative.
Fixes#59135
For example:
```
%0 = load %1
copy_addr %1 to %2
```
->
```
%0 = load %1
store %0 to %2
```
This is important for MandatoryRedundantLoadElimination to be able to create statically initialized globals in the mandatory pipeline.
For example:
```
public struct MyStruct {
public static let r: Range<Int> = 1 ..< 3
}
```
gets a statically initialized global, even at Onone, with this improvement.
rdar://149356742
Inside fragile functions, we expect function derivatives to be public, which could be achieved by either explicitly marking the functions as differentiable or having a public explicit derivative defined for them. This is obviously not
possible for single and double curry thunks which are a special case of `AutoClosureExpr`.
Instead of looking at the thunk itself, we unwrap it and look at the function being wrapped. While the thunk itself and its differentiability witness will not have public visibility, it's not an issue for the case where the function being wrapped (and its witness) have public visibility.
Fixes#54819Fixes#75776
As autodiff happens on function types it is not in general possible to determine the real expansion context of the function being differentiated. Use of minimal context is a conservative approach that should work even when libraty evolution mode is enabled.
Fixes#55179
Type annotations for instruction operands are omitted, e.g.
```
%3 = struct $S(%1, %2)
```
Operand types are redundant anyway and were only used for sanity checking in the SIL parser.
But: operand types _are_ printed if the definition of the operand value was not printed yet.
This happens:
* if the block with the definition appears after the block where the operand's instruction is located
* if a block or instruction is printed in isolation, e.g. in a debugger
The old behavior can be restored with `-Xllvm -sil-print-types`.
This option is added to many existing test files which check for operand types in their check-lines.
Use the `%target-swift-5.1-abi-triple` substitution to compile the tests for
deployment to the minimum OS versions required for use of opaque types, instead
of disabling availability checking.
[serialized_for_package] if Package CMO is enabled. The latter kind
allows a function to be serialized even if it contains loadable types,
if Package CMO is enabled. Renamed IsSerialized_t as SerializedKind_t.
The tri-state serialization kind requires validating inlinability
depending on the serialization kinds of callee vs caller; e.g. if the
callee is [serialized_for_package], the caller must be _not_ [serialized].
Renamed `hasValidLinkageForFragileInline` as `canBeInlinedIntoCaller`
that takes in its caller's SerializedKind as an argument. Another argument
`assumeFragileCaller` is also added to ensure that the calle sites of
this function know the caller is serialized unless it's called for SIL
inlining optimization passes.
The [serialized_for_package] attribute is allowed for SIL function, global var,
v-table, and witness-table.
Resolves rdar://128406520
Except for the async context, where it is needed (arguments
within an async function).
We don't support dbg.declare in optimized code, as variables can
be moved by SIL optimization passes. If a partial store is
eliminated, we want a dbg.value on the allocation, and another
dbg.value with a fragment in place of the partial store.
rdar://128155050
Unreachable blocks possess some challenges to autodiff since in reverse pass (pullback generation) we need to execute the function backwards, pushing the values from return BB back to entry block. As a result, unreachable blocks might become reachable from the return BB and this might cause all kind of issues.
Add a new mandatory BooleanLiteralFolding pass which constant folds conditional branches with boolean literals as operands.
```
%1 = integer_literal -1
%2 = apply %bool_init(%1) // Bool.init(_builtinBooleanLiteral:)
%3 = struct_extract %2, #Bool._value
cond_br %3, bb1, bb2
```
->
```
...
br bb1
```
This pass is intended to run before DefiniteInitialization, where mandatory inlining and constant folding didn't run, yet (which would perform this kind of optimization).
This optimization is required to let DefiniteInitialization handle boolean literals correctly.
For example in infinite loops:
```
init() {
while true { // DI need to know that there is no loop exit from this while-statement
if some_condition {
member_field = init_value
break
}
}
}
```
Add `Differentiable` requirements to pattern substitutions / pattern generic signature when calculating constrained function type. Also, add requirements for differentiable results as well.
Fixes#65487
Introduce the notion of "semantic result parameter". Handle differentiation of inouts via semantic result parameter abstraction. Do not consider non-wrt semantic result parameters as semantic results
Fixes#67174
The substituon map might contain proper specialization (e.g. t_0_0 -> Double) that is required to fully specify a derivate type (e.g. during reabstraction conversion from fully specified function type to a substituted one)
Fixes#65073
This essentially passes the members of a linear map tuple as individual arguments. It yields few nice simplifications:
* No linear map tuples at all for getters / setters
* No tuple formation / deconstruction around pullbacks
* Pullbacks with loops still use heap-allocated tuples
The changes are intentionally were made close to the original implementation w/o possible simplifications to ease the review
Fixes#63207, supersedes #63379 (and fixes#63234)
Single input variable might yield multiple adjoint buffers if control flow is involved. Therefore we cannot simply transfer debug info from the input variable: it will be invalid as we will end with multiple locations for a single "source" variable, and, even worse, might end with conflicting debug info as different buffers might be optimized
differently.
We do:
- Drop input argument number. This must be unique and we're not
- Correct variable name
