Dan Zheng 697c722a5f [AutoDiff] Type-checking support for inout parameter differentiation. (#29959) ...

Semantically, an `inout` parameter is both a parameter and a result.

`@differentiable` and `@derivative` attributes now support original functions
with one "semantic result": either a formal result or an `inout` parameter.

Derivative typing rules for functions with `inout` parameters are now defined.

The differential/pullback type of a function with `inout` differentiability
parameters also has `inout` parameters. This is ideal for performance.

Differential typing rules:
- Case 1: original function has no `inout` parameters.
  - Original:     `(T0, T1, ...) -> R`
  - Differential: `(T0.Tan, T1.Tan, ...) -> R.Tan`
- Case 2: original function has a non-wrt `inout` parameter.
  - Original:     `(T0, inout T1, ...) -> Void`
  - Differential: `(T0.Tan, ...) -> T1.Tan`
- Case 3: original function has a wrt `inout` parameter.
  - Original:     `(T0, inout T1, ...) -> Void`
  - Differential: `(T0.Tan, inout T1.Tan, ...) -> Void`

Pullback typing rules:
- Case 1: original function has no `inout` parameters.
  - Original: `(T0, T1, ...) -> R`
  - Pullback: `R.Tan -> (T0.Tan, T1.Tan, ...)`
- Case 2: original function has a non-wrt `inout` parameter.
  - Original: `(T0, inout T1, ...) -> Void`
  - Pullback: `(T1.Tan) -> (T0.Tan, ...)`
- Case 3: original function has a wrt `inout` parameter.
  - Original: `(T0, inout T1, ...) -> Void`
  - Pullback: `(inout T1.Tan) -> (T0.Tan, ...)`

Resolves TF-1164.

2020-02-21 09:47:53 -08:00

14 KiB

Raw Blame History

View Raw