Commit Graph

4 Commits

Author SHA1 Message Date
Richard Wei
0b53a02544 [AutoDiff] Rename 'in:' to 'of:' in differential operators.
Rename the argument label `in:` in `gradient(at:in:)`, `pullback(at:in:)`, etc to `of:`, as suggested in the [pitch thread](https://forums.swift.org/t/differentiable-programming-for-gradient-based-machine-learning/42147).
2021-02-24 01:33:42 -05:00
Richard Wei
af8942d940 [AutoDiff] Rename '@differentiable' to '@differentiable(reverse)'.
Compiler:
- Add `Forward` and `Reverse` to `DifferentiabilityKind`.
- Expand `DifferentiabilityMask` in `ExtInfo` to 3 bits so that it now holds all 4 cases of `DifferentiabilityKind`.
- Parse `@differentiable(reverse)` and `@differentiable(_forward)` declaration attributes and type attributes.
- Emit a warning for `@differentiable` without `reverse`.
- Emit an error for `@differentiable(_forward)`.
- Rename `@differentiable(linear)` to `@differentiable(_linear)`.
- Make `@differentiable(reverse)` type lowering go through today's `@differentiable` code path. We will specialize it to reverse-mode in a follow-up patch.

ABI:
- Add `Forward` and `Reverse` to `FunctionMetadataDifferentiabilityKind`.
- Extend `TargetFunctionTypeFlags` by 1 bit to store the highest bit of differentiability kind (linear). Note that there is a 2-bit gap in `DifferentiabilityMask` which is reserved for `AsyncMask` and `ConcurrentMask`; `AsyncMask` is ABI-stable so we cannot change that.

_Differentiation module:
- Replace all occurrences of `@differentiable` with `@differentiable(reverse)`.
- Delete `_transpose(of:)`.

Resolves rdar://69980056.
2021-02-07 14:09:46 -08:00
Richard Wei
ffe6064101 Mangle derivative functions and linear maps.
- `Mangle::ASTMangler::mangleAutoDiffDerivativeFunction()` and `Mangle::ASTMangler::mangleAutoDiffLinearMap()` accept original function declarations and return a mangled name for a derivative function or linear map. This is called during SILGen and TBDGen.
- `Mangle::DifferentiationMangler` handles differentiation function mangling in the differentiation transform. This part is necessary because we need to perform demangling on the original function and remangle it as part of a differentiation function mangling tree in order to get the correct substitutions in the mangled derivative generic signature.

A mangled differentiation function name includes:
- The original function.
- The differentiation function kind.
- The parameter indices for differentiation.
- The result indices for differentiation.
- The derivative generic signature.
2021-01-07 02:21:10 -08:00
Dan Zheng
2cf7d63a5e [AutoDiff] Fix ownership error in VJPCloner::visitApplyInst. (#35003)
Change the following code pattern:

```
%x1 = convert_function %x0 : $@differentiable (...) -> ...
%x2 = begin_borrow %x1
... // use %x2
%x3 = end_borrow %x2
destroy_value %x0
```

To the following:

```
%x1 = begin_borrow %x0 : $@differentiable (...) -> ...
%x2 = convert_function %x0
... // use %x2
%x3 = end_borrow %x1
destroy_value %x0
```

Resolves SR-13933: "multiple consuming users" ownership error caused by
`VJPCloner::visitApply` related to `@differentiable`-function-typed callees.

Also upstream test/AutoDiff/SILOptimizer/generics.swift from tensorflow branch.
2020-12-08 19:48:31 -05:00