Semantically, an `inout` parameter is both a parameter and a result.
`@differentiable` and `@derivative` attributes now support original functions
with one "semantic result": either a formal result or an `inout` parameter.
Derivative typing rules for functions with `inout` parameters are now defined.
The differential/pullback type of a function with `inout` differentiability
parameters also has `inout` parameters. This is ideal for performance.
Differential typing rules:
- Case 1: original function has no `inout` parameters.
- Original: `(T0, T1, ...) -> R`
- Differential: `(T0.Tan, T1.Tan, ...) -> R.Tan`
- Case 2: original function has a non-wrt `inout` parameter.
- Original: `(T0, inout T1, ...) -> Void`
- Differential: `(T0.Tan, ...) -> T1.Tan`
- Case 3: original function has a wrt `inout` parameter.
- Original: `(T0, inout T1, ...) -> Void`
- Differential: `(T0.Tan, inout T1.Tan, ...) -> Void`
Pullback typing rules:
- Case 1: original function has no `inout` parameters.
- Original: `(T0, T1, ...) -> R`
- Pullback: `R.Tan -> (T0.Tan, T1.Tan, ...)`
- Case 2: original function has a non-wrt `inout` parameter.
- Original: `(T0, inout T1, ...) -> Void`
- Pullback: `(T1.Tan) -> (T0.Tan, ...)`
- Case 3: original function has a wrt `inout` parameter.
- Original: `(T0, inout T1, ...) -> Void`
- Pullback: `(inout T1.Tan) -> (T0.Tan, ...)`
Resolves TF-1164.
Using `DenseMap<SILAutoDiffDerivativeFunctionKey, ...>` in a file that includes
AutoDiff.h but not Types.h produced the following error:
```
error: invalid application of 'alignof' to an incomplete type 'swift::SILFunctionType'
```
Fix the error by:
- Including TypeAlignments.h in AutoDiff.h.
- Adding an alignment forward declaration for `SILFunctionType` to TypeAlignments.h.
The `differentiability_witness_function` instruction looks up a
differentiability witness function (JVP, VJP, or transpose) for a referenced
function via SIL differentiability witnesses.
Add round-trip parsing/serialization and IRGen tests.
Notes:
- Differentiability witnesses for linear functions require more support.
`differentiability_witness_function [transpose]` instructions do not yet
have IRGen.
- Nothing currently generates `differentiability_witness_function` instructions.
The differentiation transform does, but it hasn't been upstreamed yet.
Resolves TF-1141.
Add `SILFunctionType::getAutoDiffTransposeFunctionType`.
It computes the transpose `SILFucntionType` for an original `SILFunctionType`,
given:
- Linearity parameter indices
- Transpose function generic signature (optional)
- Other auxiliary parameters
Add doc comments explaining typing rules, preconditions, and other details.
Add `isTranspose` flag to `autodiff::getConstrainedDerivativeGenericSignature`.
Partially resolves TF-1125.
Unblocks TF-1141: upstream `differentiability_witness_function` instruction.
SIL differentiability witnesses are a new top-level SIL construct mapping
"original" SIL functions to derivative SIL functions.
SIL differentiability witnesses have the following components:
- "Original" `SILFunction`.
- SIL linkage.
- Differentiability parameter indices (`IndexSubset`).
- Differentiability result indices (`IndexSubset`).
- Derivative `GenericSignature` representing differentiability generic
requirements (optional).
- JVP derivative `SILFunction` (optional).
- VJP derivative `SILFunction` (optional).
- "Is serialized?" bit.
This patch adds the `SILDifferentiabilityWitness` data structure, with
documentation, parsing, and printing.
Resolves TF-911.
Todos:
- TF-1136: upstream `SILDifferentiabilityWitness` serialization.
- TF-1137: upstream `SILDifferentiabilityWitness` verification.
- TF-1138: upstream `SILDifferentiabilityWitness` SILGen from
`@differentiable` and `@derivative` attributes.
- TF-20: robust mangling for `SILDifferentiabilityWitness` names.
`TangentSpace` represents the tangent space of a type.
- For `Differentiable`-conforming types:
- The tangent space is the `TangentVector` associated type.
- For tuple types:
- The tangent space is a tuple of the elements' tangent space types, for the
elements that have a tangent space.
- Other types have no tangent space.
`TypeBase::getAutoDiffTangentSpace` gets the tangent space of a type.
`TangentSpace` is used to:
- Compute the derivative function type of a given original function type.
- Compute the type of tangent/adjoint values during automatic differentiation.
Progress towards TF-828: upstream `@differentiable` attribute type-checking.
This is needed to build with VS2019:
```
swift\include\swift\AST\AutoDiff.h(172): error C2593: 'operator ==' is ambiguous
swift\include\swift\Basic\AnyValue.h(129): note: could be 'bool swift::operator ==(const swift::AnyValue &,const swift::AnyValue &)' [found using argument-dependent lookup]
swift\include\swift\AST\AutoDiff.h(172): note: or 'built-in C++ operator==(swift::AutoDiffDerivativeFunctionKind::innerty, swift::AutoDiffDerivativeFunctionKind::innerty)'
swift\include\swift\AST\AutoDiff.h(172): note: while trying to match the argument list '(const swift::AutoDiffDerivativeFunctionKind, const swift::AutoDiffDerivativeFunctionKind)'
```
The `@derivative` attribute registers a function as a derivative of another
function-like declaration: a `func`, `init`, `subscript`, or `var` computed
property declaration.
The `@derivative` attribute also has an optional `wrt:` clause specifying the
parameters that are differentiated "with respect to", i.e. the differentiation
parameters. The differentiation parameters must conform to the `Differentiable`
protocol.
If the `wrt:` clause is unspecified, the differentiation parameters are inferred
to be all parameters that conform to `Differentiable`.
`@derivative` attribute type-checking verifies that the type of the derivative
function declaration is consistent with the type of the referenced original
declaration and the differentiation parameters.
The `@derivative` attribute is gated by the
`-enable-experimental-differentiable-programming` flag.
Resolves TF-829.
The `@derivative(of:)` attribute registers a function as a derivative of another
function. This patch adds the `@derivative(of:)` attribute definition, syntax,
parsing, and printing.
Resolves TF-826.
Todos:
- Type-checking (TF-829).
- Serialization (TF-837).
This PR introduces `@differentiable` attribute to mark functions as differentiable. This PR only contains changes related to parsing the attribute. Type checking and other changes will be added in subsequent patches.
See https://github.com/apple/swift/pull/27506/files#diff-f3216f4188fd5ed34e1007e5a9c2490f for examples and tests for the new attribute.
