* initial changes
* Add tests, undo unnecessary changes.
* Fixing up computed properties accessors and adding tests for getters.
* Adding nested type testcase
* Fixing error message for when accessor is referenced but not acutally found.
* Cleanup.
- Improve diagnostic message.
- Clean up code and tests.
- Delete unrelated nested type `@derivative` attribute tests.
* Temporarily disable class subscript setter derivative registration test.
Blocked by SR-13096.
* Adding libsyntax integration and fixing up an error message.
* Added a helper function for checking if the next token is an accessor label.
* Update utils/gyb_syntax_support/AttributeNodes.py
Co-authored-by: Dan Zheng <danielzheng@google.com>
* Update lib/Parse/ParseDecl.cpp
Co-authored-by: Dan Zheng <danielzheng@google.com>
* Add end-to-end derivative registration tests.
* NFC: run `git clang-format`.
* NFC: clean up formatting.
Re-apply `git clang-format`.
* Clarify parsing ambiguity FIXME comments.
* Adding couple of more testcases and fixing up error message for when accessor is not found on functions resolved.
* Update lib/Sema/TypeCheckAttr.cpp
Co-authored-by: Dan Zheng <danielzheng@google.com>
Co-authored-by: Dan Zheng <danielzheng@google.com>
Currently it's possible to have a type conflict between different
requirements deduced as the same type which leads to incorrect
diagnostics. To mitigate that let's adjust how "fixed" requirements
are stored - instead of using resolved type for the left-hand side,
let's use originating generic parameter type.
Fix false `Differentiable` derived conformances warning on `@differentiable`
property-wrapped properties.
Check property mutability using `VarDecl::isSettable` instead of
`VarDecl::getAccessor(AccessorKind::Set)`.
Some settable properties do not have setters, depending on the underlying
`WriteImplKind`.
Resolves SR-13071.
`Differentiable` conformance derivation now supports
`Differentiable.zeroTangentVectorInitializer`.
There are two potential cases:
1. Memberwise derivation: done when `TangentVector` can be initialized memberwise.
2. `{ TangentVector.zero }` derivation: done as a fallback.
`zeroTangentVectorInitializer` is a closure that produces a zero tangent vector,
capturing minimal necessary information from `self`.
It is an instance property, unlike the static property `AdditiveArithmetic.zero`,
and should be used by the differentiation transform for correctness.
Remove `Differentiable.zeroTangentVectorInitializer` dummy default implementation.
Update stdlib `Differentiable` conformances and tests.
Clean up DerivedConformanceDifferentiable.cpp cruft.
Resolves TF-1007.
Progress towards TF-1008: differentiation correctness for projection operations.
In `Differentiable` derived conformances, determine whether a property-wrapped property is mutable by checking whether a setter exists, instead of calling `VarDecl::getPropertyWrapperMutability()`.
Resolves rdar://63577692 (SR-12873) where class-typed property wrappers with a mutable `wrappedValue` are rejected by synthesis (not included in the `TangentVector`). Also improve the property-wrapper-spe
cific immutability warning message.
Unify type-checking using `AnyFunctionType::getAutoDiffDerivativeFunctionLinearMapType`.
Delete `checkDifferentiabilityParameters` helper, which is subsumed.
Update tests with minor diagnostic changes.
Create `DerivativeFunctionTypeError` representing all potential derivative
function type calculation errors.
Make `AnyFunctionType::getAutoDiffDerivativeFunctionLinearMapType` return
`llvm::Expected<AnyFunctionType *>`. This is much safer and caller-friendly
than performing assertions.
Delete hacks in `@differentiable` and `@derivative` attribute type-checking
for verifying that `Differentiable.TangentVector` type witnesses are valid:
this is no longer necessary.
Robust fix for TF-521: invalid `Differentiable` conformances during
`@derivative` attribute type-checking.
Resolves SR-12793: bad interaction between `@differentiable` and `@derivative`
attribute type-checking and `Differentiable` derived conformances.
Make derivative forwarding thunks use original function's linkage instead of the
derivative function's, stripping external.
This is consistent with the linkage of differentiability witnesses.
Clarify AutoDiff linkage-related comments.
Resolves TF-1160: TBDGen error due to incorrect derivative thunk linkage.
Differentiable conformance derivation now "peers through" property wrappers.
Synthesized TangentVector structs contain wrapped properties' TangentVectors as
stored properties, not wrappers' TangentVectors.
Property wrapper types are not required to conform to `Differentiable`.
Property wrapper types are required to provide `wrappedValue.set`, which is
needed to synthesize `mutating func move(along:)`.
```
import _Differentiation
@propertyWrapper
struct Wrapper<Value> {
var wrappedValue: Value
}
struct Struct: Differentiable {
@Wrapper var x: Float = 0
// Compiler now synthesizes:
// struct TangentVector: Differentiable & AdditiveArithmetic {
// var x: Float
// ...
// }
}
```
Resolves SR-12638.
Require `@derivative` functions and their original functions to have the same
access level. Public original functions may also have internal
`@usableFromInline` derivatives, as a special case.
Diagnose access level mismatches. Produce a fix-it to change the derivative
function's access level.
This simplifies derivative registration rules, and may enable simplification of
AutoDiff symbol linkage rules.
Resolves TF-1099 and TF-1160.
Support `@differentiable` function conversion for `init` references, in
addition to `func` references and literal closures. Minor usability improvement.
Resolves SR-12562.
Lift temporary cross-file derivative registration restriction.
`@derivative` attribute type-checking simplications coming soon: TF-1099.
Original function and derivative function must have same access level, with one
exception: public original functions may have internal `@usableFromInline`
derivatives.
Previously, two conditions were necessary to enable differentiable programming:
- Using the `-enable-experimental-differentiable-programming` frontend flag.
- Importing the `_Differentiation` module.
Importing the `_Differentiation` module is the true condition because it
contains the required compiler-known `Differentiable` protocol. The frontend
flag is redundant and cumbersome.
Now, the frontend flag is removed.
Importing `_Differentiation` is the only condition.
The `@transpose` attribute registers a function as the transpose of another
function-like declaration: a `func`, `init`, `subscript`, or `var` computed
property declaration.
The `@transpose` attribute also has an optional `wrt:` clause specifying the
linearity parameters, i.e. the parameters that are transposed with respect to.
The linearity parameters must conform to the `Differentiable` protocol and
satisfy `Self == TangentVector`.
If the `wrt:` clause is unspecified, the linearity parameters are inferred to be
all parameters that conform to `Differentiable` and that satisfy
`Self == TangentVector`.
`@transpose` attribute type-checking verifies that the type of the transpose
function declaration is consistent with the type of the referenced original
declaration and the linearity parameters.
Resolves TF-830.
Add `AdditiveArithmetic` derived conformances for structs and classes, gated by
the `-enable-experimental-differentiable-programming` flag.
Structs and classes whose stored properties all conform to `Differentiable` can
derive `Differentiable`:
- `associatedtype TangentVector: Differentiable & AdditiveArithmetic`
- Member `TangentVector` structs are synthesized whose stored properties are
all `var` stored properties that conform to `Differentiable` and that are
not `@noDerivative`.
- `mutating func move(along: TangentVector)`
The `@noDerivative` attribute may be declared on stored properties to opt out of
inclusion in synthesized `TangentVector` structs.
Some stored properties cannot be used in `TangentVector` struct synthesis and
are implicitly marked as `@noDerivative`, with a warning:
- `let` stored properties.
- These cannot be updated by `mutating func move(along: TangentVector)`.
- Non-`Differentiable`-conforming stored properties.
`@noDerivative` also implies `@_semantics("autodiff.nonvarying")`, which is
relevant for differentiable activity analysis.
Add type-checking and SILGen tests.
Resolves TF-845.
Add the `@differentiable` function conversion pipeline:
- New expressions that convert between `@differentiable`,
`@differentiable(linear)`, and non-`@differentiable` functions:
- `DifferentiableFunction`
- `LinearFunction`
- `DifferentiableFunctionExtractOriginal`
- `LinearFunctionExtractOriginal`
- `LinearToDifferentiableFunction`
- All the AST handling (e.g. printing) necessary for those expressions.
- SILGen for those expressions.
- CSApply code that inserts these expressions to implicitly convert between
the various function types.
- Sema tests for the implicit conversions.
- SILGen tests for the SILGen of these expressions.
Resolves TF-833.
Add type checking for `@differentiable` function types:
- Check that parameters and results conform to `Differentiable`.
- Implicitly conform parameters and results whose types are generic parameters
to `Differentiable`.
- Upstream most of the differentiable_func_type_type_checking.swift test from
`tensorflow` branch. A few function conversion tests have not been added
because they depend on the `@differentiable` function conversion pipeline.
Diagnose gracefully when the `Differentiable` protocol is unavailable because
`_Differentiation` has not been imported.
Resolves TF-823 and TF-1219.
Add `AdditiveArithmetic` derived conformances for structs, gated by the
`-enable-experimential-additive-arithmetic-derivation` flag.
Structs whose stored properties all conform to `AdditiveArithmetic` can derive
`AdditiveArithmetic`:
- `static var zero: Self`
- `static func +(lhs: Self, rhs: Self) -> Self`
- `static func -(lhs: Self, rhs: Self) -> Self`
- An "effective memberwise initializer":
- Either a synthesized memberwise initializer or a user-defined initializer
with the same type.
Effective memberwise initializers are used only by derived conformances for
`Self`-returning protocol requirements like `AdditiveArithmetic.+`, which
require memberwise initialization.
Resolves TF-844.
Unblocks TF-845: upstream `Differentiable` derived conformances.
Previously, all witnesses of a `@differentiable` protocol requirement were
required to have the same attribute (or one with superset parameter indices).
However, this leads to many annotations on witnesses and is not ideal for
usability. `@differentiable` attributes are really only significant on
public witnesses, so that they are clearly `@differentiable` at a glance (in
source code, interface files, and API documentation), without looking through
protocol conformance hierarchies.
Now, only *public* witnesses of `@differentiable` protocol requirements are
required to have the same attribute (or one with superset parameter indices).
For less-visible witnesses, an implicit `@differentiable` attribute is created
with the same configuration as the requirement's.
Resolves TF-1117.
Upstreams #29771 from tensorflow branch.
Delete `@differentiable` attribute `jvp:` and `vjp:` arguments for derivative
registration. `@derivative` attribute is now the canonical way to register
derivatives.
Resolves TF-1001.
Add test/AutoDiff/lit.local.cfg: run tests only when `differentiable_programming`
is enabled in lit. With this, individual tests no longer need
`REQUIRES: differentiable_programming`.
Move multi-functionality SIL tests from test/AutoDiff/SIL/Serialization to
test/AutoDiff/SIL.
Garden test filenames.
Semantically, an `inout` parameter is both a parameter and a result.
`@differentiable` and `@derivative` attributes now support original functions
with one "semantic result": either a formal result or an `inout` parameter.
Derivative typing rules for functions with `inout` parameters are now defined.
The differential/pullback type of a function with `inout` differentiability
parameters also has `inout` parameters. This is ideal for performance.
Differential typing rules:
- Case 1: original function has no `inout` parameters.
- Original: `(T0, T1, ...) -> R`
- Differential: `(T0.Tan, T1.Tan, ...) -> R.Tan`
- Case 2: original function has a non-wrt `inout` parameter.
- Original: `(T0, inout T1, ...) -> Void`
- Differential: `(T0.Tan, ...) -> T1.Tan`
- Case 3: original function has a wrt `inout` parameter.
- Original: `(T0, inout T1, ...) -> Void`
- Differential: `(T0.Tan, inout T1.Tan, ...) -> Void`
Pullback typing rules:
- Case 1: original function has no `inout` parameters.
- Original: `(T0, T1, ...) -> R`
- Pullback: `R.Tan -> (T0.Tan, T1.Tan, ...)`
- Case 2: original function has a non-wrt `inout` parameter.
- Original: `(T0, inout T1, ...) -> Void`
- Pullback: `(T1.Tan) -> (T0.Tan, ...)`
- Case 3: original function has a wrt `inout` parameter.
- Original: `(T0, inout T1, ...) -> Void`
- Pullback: `(inout T1.Tan) -> (T0.Tan, ...)`
Resolves TF-1164.
- Support `@differentiable` and `@derivative` attributes for original
initializers in final classes. Reject original initializers in non-final
classes.
- Synchronize tests.
Attempt to look up original function before checking whether the `value:` result
conforms to `Differentiable`.
This improves diagnostics: "original function not found" should be diagnosed as
early as possible.
Previously, `@derivative` attribute type-checking produced a confusing error
referencing unbound types `T` and `U`:
```
'@derivative(of:)' attribute requires function to return a two-element tuple of
type '(value: T..., pullback: (U.TangentVector) -> T.TangentVector...)' or
'(value: T..., differential: (T.TangentVector...) -> U.TangentVector)'
```
Now, the error is less confusing:
```
'@derivative(of:)' attribute requires function to return a two-element tuple;
first element must have label 'value:' and second element must have label
'pullback:' or 'differential:'
```
For protocol requirements and class members with `@differentiable` attribute,
conforming types and subclasses must have the same `@differentiable` attribute
(or one with a superset of differentiability parameters) on implementing/
overriding declarations.
For implementing/overriding declarations that are missing a `@differentiable`
attribute, emit a fix-it that adds the missing attribute.
Resolves TF-1118.
The `@differentiable` attribute marks a function as differentiable.
Example:
```
@differentiable(wrt: x, jvp: derivativeFoo where T: Differentiable)
func id<T>(_ x: T) -> T { x }
```
The `@differentiable` attribute has an optional `wrt:` clause specifying the
parameters that are differentiated "with respect to", i.e. the differentiability
parameters. The differentiability parameters must conform to the
`Differentiable` protocol.
If the `wrt:` clause is unspecified, the differentiability parameters are
currently inferred to be all parameters that conform to `Differentiable`.
The `@differentiable` attribute also has optional `jvp:` and `vjp:` labels
for registering derivative functions. These labels are deprecated in favor of
the `@derivative` attribute and will be removed soon.
The `@differentiable` attribute also has an optional `where` clause, specifying
extra differentiability requirements for generic functions.
The `@differentiable` attribute is gated by the
`-enable-experimental-differentiable-programming` flag.
Code changes:
- Add `DifferentiableAttributeTypeCheckRequest`.
- Currently, the request returns differentiability parameter indices, while
also resolving `JVPFunction`, `VJPFunction`, and
`DerivativeGenericSignature` and mutating them in-place in
`DifferentiableAttr`. This was the simplest approach that worked without
introducing request cycles.
- Add "is type-checked" bit to `DifferentiableAttr`.
- Alternatively, I tried changing `DifferentiableAttributeTypeCheckRequest` to
use `CacheKind::Cache` instead of `CacheKind::SeparatelyCached`, but it did
not seem to work: `@differentiable` attributes in non-primary-files were
left unchecked.
Type-checking rules (summary):
- `@differentiable` attribute must be declared on a function-like "original"
declaration: `func`, `init`, `subscript`, `var` (computed properties only).
- Parsed differentiability parameters must be valid (if they exist).
- Parsed `where` clause must be valid (if it exists).
- Differentiability parameters must all conform to `Differentiable`.
- Original result must all conform to `Differentiable`.
- If JVP/VJP functions are specified, they must match the expected type.
- `@differentiable(jvp:vjp:)` for derivative registration is deprecated in
favor of `@derivative` attribute, and will be removed soon.
- Duplicate `@differentiable` attributes with the same differentiability
parameters are invalid.
- For protocol requirements and class members with `@differentiable` attribute,
conforming types and subclasses must have the same `@differentiable` attribute
(or one with a superset of differentiability parameter indices) on
implementing/overriding declarations.
Enable qualified declaration names in `@derivative` attribute, just like
`@transpose` attribute.
`DerivativeAttr` now stores a base type `TypeRepr *`, which is non-null for
parsed attributes that reference a qualified original declaration.
Add `TypeResolutionFlags::AllowModule` flag to enable module lookup via
`TypeChecker::lookupMember` given a `ModuleType`.
Add tests for type-qualified and module-qualified declaration names.
Resolves TF-1058.
The new _Differentiable module is not available in any shipping OS release, but its public API currently doesn’t have availability, either.
Temporarily disable tests that import it when we’re testing with OS-provided libraries.
The typecheck test test/AutoDiff/stdlib/differentiable_protocol.swift imports _Differentable but still somehow succeeds in these configs, so leave that one enabled.
rdar://57975086
The `@derivative` attribute registers a function as a derivative of another
function-like declaration: a `func`, `init`, `subscript`, or `var` computed
property declaration.
The `@derivative` attribute also has an optional `wrt:` clause specifying the
parameters that are differentiated "with respect to", i.e. the differentiation
parameters. The differentiation parameters must conform to the `Differentiable`
protocol.
If the `wrt:` clause is unspecified, the differentiation parameters are inferred
to be all parameters that conform to `Differentiable`.
`@derivative` attribute type-checking verifies that the type of the derivative
function declaration is consistent with the type of the referenced original
declaration and the differentiation parameters.
The `@derivative` attribute is gated by the
`-enable-experimental-differentiable-programming` flag.
Resolves TF-829.
Adds parsing for a type attribute `@differentiable`, which is optionally allowed to have argument `@differentiable(linear)`.
The typechecker currently rejects all uses of `@differentiable` with "error: attribute does not apply to type". Future work (https://bugs.swift.org/browse/TF-871https://bugs.swift.org/browse/TF-873) will update the typechecker to allow this attribute in places where it is allowed.
Resolves https://bugs.swift.org/browse/TF-822.
