Add mangling scheme for `@differentiable` and `@differentiable(linear)` function
types. Mangling support is important for debug information, among other things.
Update docs and add tests.
Resolves TF-948.
The `@transpose` attribute registers a function as the transpose of another
function-like declaration: a `func`, `init`, `subscript`, or `var` computed
property declaration.
The `@transpose` attribute also has an optional `wrt:` clause specifying the
linearity parameters, i.e. the parameters that are transposed with respect to.
The linearity parameters must conform to the `Differentiable` protocol and
satisfy `Self == TangentVector`.
If the `wrt:` clause is unspecified, the linearity parameters are inferred to be
all parameters that conform to `Differentiable` and that satisfy
`Self == TangentVector`.
`@transpose` attribute type-checking verifies that the type of the transpose
function declaration is consistent with the type of the referenced original
declaration and the linearity parameters.
Resolves TF-830.
Switch the direct operator lookup logic over to
querying the SourceLookupCache, then switch the
main operator lookup logic over to calling the
direct lookup logic rather than querying the
operator maps on the SourceFile.
This then allows us to remove the SourceFile
operator maps, in addition to the logic from
NameBinding that populated them. This requires
redeclaration checking to be implemented
separately.
Finally, to compensate for the caching that the old
operator maps were providing for imported results,
turn the operator lookup requests into cached
requests.
Query the SourceLookupCache for the operator decls,
and use ModuleDecl::getOperatorDecls for both
frontend stats and to clean up some code
completion logic.
In addition, this commit switches getPrecedenceGroups
over to querying SourceLookupCache.
Serialize derivative function configurations per module.
`@differentiable` and `@derivative` attributes register derivatives for
`AbstractFunctionDecl`s for a particular "derivative function configuration":
parameter indices and dervative generic signature.
To find `@derivative` functions registered in other Swift modules, derivative
function configurations must be serialized per module. When configurations for
a `AbstractFunctionDecl` are requested, all configurations from imported
modules are deserialized. This module serialization technique has precedent: it
is used for protocol conformances (e.g. extension declarations for a nominal
type) and Obj-C members for a class type.
Add `AbstractFunctionDecl::getDerivativeFunctionConfigurations` entry point
for accessing derivative function configurations.
In the differentiation transform: use
`AbstractFunctionDecl::getDerivativeFunctionConfigurations` to implement
`findMinimalDerivativeConfiguration` for canonical derivative function
configuration lookup, replacing `getMinimalASTDifferentiableAttr`.
Resolves TF-1100.
Add `AdditiveArithmetic` derived conformances for structs and classes, gated by
the `-enable-experimental-differentiable-programming` flag.
Structs and classes whose stored properties all conform to `Differentiable` can
derive `Differentiable`:
- `associatedtype TangentVector: Differentiable & AdditiveArithmetic`
- Member `TangentVector` structs are synthesized whose stored properties are
all `var` stored properties that conform to `Differentiable` and that are
not `@noDerivative`.
- `mutating func move(along: TangentVector)`
The `@noDerivative` attribute may be declared on stored properties to opt out of
inclusion in synthesized `TangentVector` structs.
Some stored properties cannot be used in `TangentVector` struct synthesis and
are implicitly marked as `@noDerivative`, with a warning:
- `let` stored properties.
- These cannot be updated by `mutating func move(along: TangentVector)`.
- Non-`Differentiable`-conforming stored properties.
`@noDerivative` also implies `@_semantics("autodiff.nonvarying")`, which is
relevant for differentiable activity analysis.
Add type-checking and SILGen tests.
Resolves TF-845.
Add the `@differentiable` function conversion pipeline:
- New expressions that convert between `@differentiable`,
`@differentiable(linear)`, and non-`@differentiable` functions:
- `DifferentiableFunction`
- `LinearFunction`
- `DifferentiableFunctionExtractOriginal`
- `LinearFunctionExtractOriginal`
- `LinearToDifferentiableFunction`
- All the AST handling (e.g. printing) necessary for those expressions.
- SILGen for those expressions.
- CSApply code that inserts these expressions to implicitly convert between
the various function types.
- Sema tests for the implicit conversions.
- SILGen tests for the SILGen of these expressions.
Resolves TF-833.
Add type checking for `@differentiable` function types:
- Check that parameters and results conform to `Differentiable`.
- Implicitly conform parameters and results whose types are generic parameters
to `Differentiable`.
- Upstream most of the differentiable_func_type_type_checking.swift test from
`tensorflow` branch. A few function conversion tests have not been added
because they depend on the `@differentiable` function conversion pipeline.
Diagnose gracefully when the `Differentiable` protocol is unavailable because
`_Differentiation` has not been imported.
Resolves TF-823 and TF-1219.
A request is intended to be a pure function of its inputs. That function could, in theory, fail. In practice, there were basically no requests taking advantage of this ability - the few that were using it to explicitly detect cycles can just return reasonable defaults instead of forwarding the error on up the stack.
This is because cycles are checked by *the Evaluator*, and are unwound by the Evaluator.
Therefore, restore the idea that the evaluate functions are themselves pure, but keep the idea that *evaluation* of those requests may fail. This model enables the best of both worlds: we not only keep the evaluator flexible enough to handle future use cases like cancellation and diagnostic invalidation, but also request-based dependencies using the values computed at the evaluation points. These aforementioned use cases would use the llvm::Expected interface and the regular evaluation-point interface respectively.
Introduce evaluator::SideEffect, the type of a request that performs
some operation solely to execute its side effects. Thankfully, there are
precious few requests that need to use this type in practice, but it's
good to call them out explicitly so we can get around to making them
behave much more functionally in the future.
Property wrappers are allowed to infer the type of a variable, but this
only worked when the property wrapper was provided with an explicit
initialization, e.g.,
@WrapsAnInt() var x // infers type Int from WrapsAnInt.wrappedValue
However, when default initialization is supported by the property wrapper,
dropping the parentheses would produce an error about the missing type
annotation
@WrapsAnInt var x
Make this second case behave like the first, so that default initialization
works consistently with the explicitly-specified version.
Fixes rdar://problem/59471019.
RedundantPhiElimination eliminates block phi-arguments which have the same value as other arguments of the same block.
This also works with cycles, like two equivalent loop induction variables. Such patterns are generated e.g. when using stdlib's enumerated() on Array.
preheader:
br bb1(%initval, %initval)
header(%phi1, %phi2):
%next1 = builtin "add" (%phi1, %one)
%next2 = builtin "add" (%phi2, %one)
cond_br %loopcond, header(%next1, %next2), exit
exit:
is replaced with
preheader:
br bb1(%initval)
header(%phi1):
%next1 = builtin "add" (%phi1, %one)
%next2 = builtin "add" (%phi1, %one) // dead: will be cleaned-up later
cond_br %loopcond, header(%next1), exit
exit:
Any remaining dead or "trivially" equivalent instructions will then be cleaned-up by DCE and CSE, respectively.
rdar://problem/33438123
Add a request to lookup all implied conformances for use while
typechecking the primary. This provides a cache-point for
evaluator-based dependency tracking.
Add `linear_function` and `linear_function_extract` instructions.
`linear_function` creates a `@differentiable(linear)` function-typed value from
an original function operand and a transpose function operand (optional).
`linear_function_extract` extracts either the original or transpose function
value from a `@differentiable(linear)` function.
Resolves TF-1142 and TF-1143.
This indirectly hardens the `swift_reflection_infoForTypeRef` API
against being invoked with a null TypeRef pointer. The API already
handles a nullptr being returned from `TypeConverter::getTypeInfo`
by converting it into a standard "UNKNOWN" type info descriptor.
Resolves rdar://60633988
Several tests related to indexing system modules were taking a considerable
amount of time (100+ seconds in the worst case) indexing the standard library.
This adds a frontend option to skip it and updates those tests to pass it.
Add `AdditiveArithmetic` derived conformances for structs, gated by the
`-enable-experimential-additive-arithmetic-derivation` flag.
Structs whose stored properties all conform to `AdditiveArithmetic` can derive
`AdditiveArithmetic`:
- `static var zero: Self`
- `static func +(lhs: Self, rhs: Self) -> Self`
- `static func -(lhs: Self, rhs: Self) -> Self`
- An "effective memberwise initializer":
- Either a synthesized memberwise initializer or a user-defined initializer
with the same type.
Effective memberwise initializers are used only by derived conformances for
`Self`-returning protocol requirements like `AdditiveArithmetic.+`, which
require memberwise initialization.
Resolves TF-844.
Unblocks TF-845: upstream `Differentiable` derived conformances.
Previously, all witnesses of a `@differentiable` protocol requirement were
required to have the same attribute (or one with superset parameter indices).
However, this leads to many annotations on witnesses and is not ideal for
usability. `@differentiable` attributes are really only significant on
public witnesses, so that they are clearly `@differentiable` at a glance (in
source code, interface files, and API documentation), without looking through
protocol conformance hierarchies.
Now, only *public* witnesses of `@differentiable` protocol requirements are
required to have the same attribute (or one with superset parameter indices).
For less-visible witnesses, an implicit `@differentiable` attribute is created
with the same configuration as the requirement's.
Resolves TF-1117.
Upstreams #29771 from tensorflow branch.
Define type signatures and SILGen for the following builtins:
```
/// Applies the {jvp|vjp} of `f` to `arg1`, ..., `argN`.
func applyDerivative_arityN_{jvp|vjp}(f, arg1, ..., argN) -> jvp/vjp return type
/// Applies the transpose of `f` to `arg`.
func applyTranspose_arityN(f, arg) -> transpose return type
/// Makes a differentiable function from the given `original`, `jvp`, and
/// `vjp` functions.
func differentiableFunction_arityN(original, jvp, vjp)
/// Makes a linear function from the given `original` and `transpose` functions.
func linearFunction_arityN(original, transpose)
```
Add SILGen FileCheck tests for all builtins.
Remove logic for parsing and diagnosing `jvp:` and `vjp:` arguments for
`@differentiable` attribute. No logic remains for handling those arguments.
Follow-up to TF-1001.
Delete `@differentiable` attribute `jvp:` and `vjp:` arguments for derivative
registration. `@derivative` attribute is now the canonical way to register
derivatives.
Resolves TF-1001.
HOW THIS WAS DONE: I did this by refactoring the last usages of checkValue into
a higher level API that uses checkValue as an implementation detail:
completeConsumingUseSet(...). All of these places in
MandatoryInlining/PredictableMemOpts all wanted behavior where we complete a set
of consuming uses for a value, detecting if the consuming use is in a different
loop nest from the value.
WHY DO THIS: The reason why I wanted to do this is that checkValue is a lower
level API that drives the actual low level computation. We shouldn't expose its
interface to the LinearLifetimeChecker's users since it is our own private
implementation detail that also has some sharp edges.
AN ADDITIONAL BENEFIT: Additionally by hiding the declaration of checkValue, the
last public use of LinearLifetimeError and ErrorBehaviorKind was not
private. This allowed me to then move the declarations of those two to a private
header (./lib/SIL/LinearLifetimeCheckerPrivate.h) and make their declarations
private to LinearLifetimeChecker as well. As such, I renamed them to
LinearLifetimeChecker::Error and LinearLifetimeChecker::ErrorBehaviorKind.
Factor out the lookup side of TypeChecker::conformsToProtocol so we have
a dependency registration point available for evaluator-based
dependencies that doesn't have re-entrancy problems.
Introduce DirectOperatorLookupRequest &
DirectPrecedenceGroupLookupRequest that lookup
operator and precedence groups within a given
file or module without looking through imports.
These will eventually be used as the basis for the
new operator lookup implementation, but for now
just use them when querying lookup results from
serialized module files.
Add `differentiable_function` and `differentiable_function_extract`
instructions.
`differentiable_function` creates a `@differentiable` function-typed
value from an original function operand and derivative function operands
(optional).
`differentiable_function_extract` extracts either the original or
derivative function value from a `@differentiable` function.
The differentiation transform canonicalizes `differentiable_function`
instructions, filling in derivative function operands if missing.
Resolves TF-1139 and TF-1140.
