Commit Graph

5 Commits

Author SHA1 Message Date
Richard Wei
0b53a02544 [AutoDiff] Rename 'in:' to 'of:' in differential operators.
Rename the argument label `in:` in `gradient(at:in:)`, `pullback(at:in:)`, etc to `of:`, as suggested in the [pitch thread](https://forums.swift.org/t/differentiable-programming-for-gradient-based-machine-learning/42147).
2021-02-24 01:33:42 -05:00
Richard Wei
e494df2ee6 [AutoDiff] Add differentiability kind to differentiability witnesses and mangle them.
Differentiability witnesses are now keyed by the original function name, the differentiability kind, and the autodiff config.

Updated SIL syntax:
```
differentiability-kind ::= 'forward' | 'reverse' | 'normal' | 'linear'
sil-differentiability-witness ::=
    'sil_differentiability_witness'
    sil-linkage?
    '[' differentiability-kind ']'
    '[' 'parameters' sil-differentiability-witness-function-index-list ']'
    '[' 'results' sil-differentiability-witness-function-index-list ']'
    generic-parameter-clause?
    sil-function-name ':' sil-type
    sil-differentiability-witness-body?
sil-instruction ::=
    'differentiability_witness_function'
    '[' sil-differentiability-witness-function-kind ']'
    '[' differentiability-kind ']'
    '[' 'parameters' sil-differentiability-witness-function-index-list ']'
    '[' 'results' sil-differentiability-witness-function-index-list ']'
    generic-parameter-clause?
    sil-function-name ':' sil-type
```
```console
sil_differentiability_witness [reverse] [parameters 0 1] [results 0] <T where T: Differentiable> @foo : <T> $(T) -> T
differentiability_witness_function [vjp] [reverse] [parameters 0] [results 0] <T where T: Differentiable> @foo : $(T) -> T
```

New mangling:
```swift
  global ::= global generic-signature? 'WJ' DIFFERENTIABILITY-KIND INDEX-SUBSET 'p' INDEX-SUBSET 'r' // differentiability witness
```
```console
$s13test_mangling3fooyS2f_S2ftFWJrSpSr ---> reverse differentiability witness for test_mangling.foo(Swift.Float, Swift.Float, Swift.Float) -> Swift.Float with respect to parameters {0} and results {0}
```

Resolves rdar://74380324.
2021-02-17 18:27:42 -05:00
Richard Wei
af8942d940 [AutoDiff] Rename '@differentiable' to '@differentiable(reverse)'.
Compiler:
- Add `Forward` and `Reverse` to `DifferentiabilityKind`.
- Expand `DifferentiabilityMask` in `ExtInfo` to 3 bits so that it now holds all 4 cases of `DifferentiabilityKind`.
- Parse `@differentiable(reverse)` and `@differentiable(_forward)` declaration attributes and type attributes.
- Emit a warning for `@differentiable` without `reverse`.
- Emit an error for `@differentiable(_forward)`.
- Rename `@differentiable(linear)` to `@differentiable(_linear)`.
- Make `@differentiable(reverse)` type lowering go through today's `@differentiable` code path. We will specialize it to reverse-mode in a follow-up patch.

ABI:
- Add `Forward` and `Reverse` to `FunctionMetadataDifferentiabilityKind`.
- Extend `TargetFunctionTypeFlags` by 1 bit to store the highest bit of differentiability kind (linear). Note that there is a 2-bit gap in `DifferentiabilityMask` which is reserved for `AsyncMask` and `ConcurrentMask`; `AsyncMask` is ABI-stable so we cannot change that.

_Differentiation module:
- Replace all occurrences of `@differentiable` with `@differentiable(reverse)`.
- Delete `_transpose(of:)`.

Resolves rdar://69980056.
2021-02-07 14:09:46 -08:00
Dan Zheng
07f632acaa [AutoDiff] NFC: document test suite. (#35194)
Add README files explaining the differentiable programming test suite.

Add lit.local.cfg files so individual tests do not need to add
`REQUIRES: asserts` in these directories:

- test/AutoDiff/compiler_crashers
- test/AutoDiff/compiler_crashers_fixed

Motivation: it is important for compiler crasher tests to require assertions
enabled, because many of these tests crash on compiler assertions.
2020-12-23 07:19:19 -05:00
marcrasi
ce5e0e774c [AutoDiff] Fix SR-12641: Handle address-only types in derivative fn types (#31496)
* The update in `SILFunctionType.cpp` fixes SR-12641 by making address-only parameters/results in differentials have indirect convention.
* I updated the crasher test to use a resilient struct defined in the test, instead of `Tracked<Float>`, so that the test does not need to depend on `DifferentiationUnittest`.
* The update in `VJPEmitter.cpp` fixes a similar issue with pullbacks that I discovered while investigating.
* I added code that exposes this new issue to the SR-12641 crasher test.
2020-05-05 12:29:10 -07:00