Consider an `@_alwaysEmitIntoClient` function and a custom derivative
defined
for it. Previously, such a combination resulted different errors under
different
circumstances.
Sometimes, there were linker errors due to missing derivative function
symbol -
these occurred when we tried to find the derivative in a module, while
it
should have been emitted into client's code (and it did not happen).
Sometimes, there were SIL verification failures like this:
```
SIL verification failed: internal/private function cannot be serialized or serializable: !F->isAnySerialized() || embedded
```
Linkage and serialization options for the derivative were not handled
properly,
and, instead of PublicNonABI linkage, we had Private one which is
unsupported
for serialization - but we need to serialize `@_alwaysEmitIntoClient`
functions
so the client's code is able to see them.
This patch resolves the issue and adds proper handling of custom
derivatives
of `@_alwaysEmitIntoClient` functions. Note that either both the
function and
its custom derivative or none of them should have
`@_alwaysEmitIntoClient`
attribute, mismatch in this attribute is not supported.
The following cases are handled (assume that in each case client's code
uses
the derivative).
1. Both the function and its derivative are defined in a single file in
one module.
2. Both the function and its derivative are defined in different files
which
are compiled to a single module.
3. The function is defined in one module, its derivative is defined in
another
module.
4. The function and the derivative are defined as members of a protocol
extension in two separate modules - one for the function and one for the
derivative. A struct conforming the protocol is defined in the third
module.
5. The function and the derivative are defined as members of a struct
extension in two separate modules - one for the function and one for the
derivative.
The changes allow to define derivatives for methods of `SIMD`.
Fixes#54445
<!--
If this pull request is targeting a release branch, please fill out the
following form:
https://github.com/swiftlang/.github/blob/main/PULL_REQUEST_TEMPLATE/release.md?plain=1
Otherwise, replace this comment with a description of your changes and
rationale. Provide links to external references/discussions if
appropriate.
If this pull request resolves any GitHub issues, link them like so:
Resolves <link to issue>, resolves <link to another issue>.
For more information about linking a pull request to an issue, see:
https://docs.github.com/issues/tracking-your-work-with-issues/linking-a-pull-request-to-an-issue
-->
<!--
Before merging this pull request, you must run the Swift continuous
integration tests.
For information about triggering CI builds via @swift-ci, see:
https://github.com/apple/swift/blob/main/docs/ContinuousIntegration.md#swift-ci
Thank you for your contribution to Swift!
-->
Type annotations for instruction operands are omitted, e.g.
```
%3 = struct $S(%1, %2)
```
Operand types are redundant anyway and were only used for sanity checking in the SIL parser.
But: operand types _are_ printed if the definition of the operand value was not printed yet.
This happens:
* if the block with the definition appears after the block where the operand's instruction is located
* if a block or instruction is printed in isolation, e.g. in a debugger
The old behavior can be restored with `-Xllvm -sil-print-types`.
This option is added to many existing test files which check for operand types in their check-lines.
The patch adds lowering of partial_apply instructions for coroutines.
This pattern seems to trigger a lot of type mismatch errors in IRGen, because
coroutine functions are not substituted in the same way as regular functions
(see the patch 07f03bd2 "Use pattern substitutions to consistently abstract
yields" for more details).
Other than that, lowering of partial_apply for coroutines is straightforward: we
generate another coroutine that captures arguments passed to the partial_apply
instructions. It calls the original coroutine for yields (first return) and
yields the resulting values. Then it calls the original function's continuation
for return or unwind, and forwards them to the caller as well.
After IRGen, LLVM's Coroutine pass transforms the generated coroutine (along with
all other coroutines) and eliminates llvm.coro.* intrinsics. LIT tests check
LLVM IR after this transformation.
Co-authored-by: Anton Korobeynikov <anton@korobeynikov.info>
Co-authored-by: Arnold Schwaighofer <aschwaighofer@apple.com>
Add a new mandatory BooleanLiteralFolding pass which constant folds conditional branches with boolean literals as operands.
```
%1 = integer_literal -1
%2 = apply %bool_init(%1) // Bool.init(_builtinBooleanLiteral:)
%3 = struct_extract %2, #Bool._value
cond_br %3, bb1, bb2
```
->
```
...
br bb1
```
This pass is intended to run before DefiniteInitialization, where mandatory inlining and constant folding didn't run, yet (which would perform this kind of optimization).
This optimization is required to let DefiniteInitialization handle boolean literals correctly.
For example in infinite loops:
```
init() {
while true { // DI need to know that there is no loop exit from this while-statement
if some_condition {
member_field = init_value
break
}
}
}
```
Introduce the notion of "semantic result parameter". Handle differentiation of inouts via semantic result parameter abstraction. Do not consider non-wrt semantic result parameters as semantic results
Fixes#67174
Linear maps are captured in vjp routine via callee-guaranteed partial apply and are passed as @owned references to the enclosing pullback that finally consumes them. Necessary retains are inserted by a partial apply forwarder.
However, this is not the case when the function being differentiated contains loops as heap-allocated context is used and bare pointer is captured by the pullback partial apply. As a result, partial apply forwarder does not retain the linear maps that are owned by a heap-allocated context, however, they are still treated as @owned references and therefore are released in the pullback after the first call. As a result, subsequent pullback calls release linear maps and we'd end with possible use-after-free.
Ensure we retain values when we load values from the context.
Reproducible only when:
* Loops (so, heap-allocated context)
* Pullbacks of thick functions (so context is non-zero)
* Multiple pullback calls
* Some cleanup while there
Fixes#64257
This essentially passes the members of a linear map tuple as individual arguments. It yields few nice simplifications:
* No linear map tuples at all for getters / setters
* No tuple formation / deconstruction around pullbacks
* Pullbacks with loops still use heap-allocated tuples
The changes are intentionally were made close to the original implementation w/o possible simplifications to ease the review
Fixes#63207, supersedes #63379 (and fixes#63234)
Apparently #60467 changed the semantics of store_borrow as it started to produce a value. This change was not documented in SIL spec and not all places were updated to new semantics.
Now the adjoint of store_borrow should be generated for the value of instruction itself, not the destination address
@differentiable function is actually a triple (function, jvp, vjp). Previously normal thick function value witness table was used. As a result, for example, only function was copied, but none of differential components.
This was the cause of uninitialized memory accesses and subsequent segfaults.
Should fix now unavailable TF-1122
In `TangentBuilder` temporary allocations are emitted in order to make generic calls to `AdditiveArithmetic.zero` and `AdditiveArithmetic.+`. When we perform accumulation of two adjoint values that correspond to a lexical value in the original function, the stack allocations require a debug variable, as we use the original instruction's location. However such a debug variable wouldn't really make sense as these temporary allocations do not represent the original variable's adjoint. With lexical borrow scopes, we hit a crasher when creating temporary allocations when accumulating adjoints for lexical `begin_borrow` instructions.
This PR makes `TangentBuilder`'s temporary allocations use an auto-generated location. The existing debug info emission behavior is not changed, as we still emit `debug_value` when materializing an `AdjointValue` after any necessary adjoint accumulation.
The effect of passing -enable-copy-propagation is both to enable the
CopyPropagation pass to shorten object lifetimes and also to enable
lexical lifetimes to ensure that object lifetimes aren't shortened while
a variable is still in scope and used.
Add a new flag, -enable-lexical-borrow-scopes=true to override
-enable-copy-propagation's effect (setting it to ::ExperimentalLate) on
SILOptions::LexicalLifetimes that sets it to ::Early even in the face of
-enable-copy-propagation. The old flag -disable-lexical-lifetimes is
renamed to -enable-lexical-borrow-scopes=false but continues to set that
option to ::Off even when -enable-copy-propagation is passed.
The latest Long Term Support NDK finally removed binutils, including the bfd/gold
linkers and libgcc. This simplifies our Android support, including making lld the
default linker for Android. Disable three reflection tests that now fail, likely
related to issues with swift-reflection-dump and switching to lld.
The latest Long Term Support NDK finally removed binutils, including the bfd/gold
linkers and libgcc. This simplifies our Android support, including making lld the
default linker for Android. Disable three reflection tests that now fail, likely
related to issues with swift-reflection-dump and switching to lld.
Also, add the libatomic dependency for Android armv7, just as on linux.
1. When calculating the differential type of an original function with an inout parameter and when the inout parameter has a type parameter, the inout parameter should get a generic parameter in the subst generic signature of the differential but it currently doesn't. This causes SILGen to attempt to reabstract the differential value in the JVP protocol witness thunk, whilst the generic signature is lacking requirements, leading to a requirement machine error. This patch fixes the calculation so that the JVP's result type (the differential type) always matches the witness thunk's result type.
Wrong type:
```swift
sil private [transparent] [thunk] [ossa] @... <τ_0_0 where τ_0_0 : Differentiable> (...) -> @owned @callee_guaranteed @substituted <τ_0_0, τ_0_1> (@in_guaranteed τ_0_0) -> @out τ_0_1 for <τ_0_0.TangentVector, τ_0_0.TangentVector> {
%6 = differentiable_function_extract [jvp] %5 : $@differentiable(reverse) @convention(method) <τ_0_0 where τ_0_0 : Differentiable> (@in_guaranteed τ_0_0, @noDerivative @inout τ_0_0, @noDerivative SR_13305_Struct) -> () // user: %7
HERE ====> %7 = apply %6<τ_0_0>(%0, %1, %3) : $@convention(method) <τ_0_0 where τ_0_0 : Differentiable> (@in_guaranteed τ_0_0, @inout τ_0_0, SR_13305_Struct) -> @owned @callee_guaranteed @substituted <τ_0_0> (@in_guaranteed τ_0_0) -> @out τ_0_0 for <τ_0_0.TangentVector>
```
Should be:
```swift
%7 = apply %6<τ_0_0>(%0, %1, %3) : $@convention(method) <τ_0_0 where τ_0_0 : Differentiable> (@in_guaranteed τ_0_0, @inout τ_0_0, SR_13305_Struct) -> @owned @callee_guaranteed @substituted <τ_0_0, τ_0_1> (@in_guaranteed τ_0_0) -> @out τ_0_1 for <τ_0_0.TangentVector, τ_0_0.TangentVector>
```
2. `TypeConverter::makeConstantInterfaceType` is not passing down the derivative generic signature to `SILFunctionType::getAutoDiffDerivativeFunctionType` for class methods, and this was caught by RequirementMachine during vtable emission. This patch fixes that.
Partially resolves rdar://82549134. The only remaining tests that require `-requirement-machine=off` are SILOptimizer/semantic_member_accessors_sil.swift and SILOptimizer/differentiation_diagnostics.swift which I will fix next. Then I'll do a proper fix for workaround #39416.
The SIL type lowering logic for AutoDiff gets the substituted generic signature
mixed up with the invocation generic signature, so it tries to ask questions
about DependentMemberTypes in a signature with no requirements. This triggers
assertions when the requirement machine is enabled.
Disable the requirement machine until this is fixed.
* Attempt at a patch for SR-14053 and SR-14218, based on Dan Zheng's initial fix.
* Incorporated Dan's cleanup suggestions.
* Converting a bool SmallVector to a SmallBitVector.
* Testing if Windows issues with this test are due to runtime support.
* Simplifying test case.
Repurpose mangling operator `Y` as an umbrella operator that covers new attributes on function types. Free up operators `J`, `j`, and `k`.
```
async ::= 'Ya' // 'async' annotation on function types
sendable ::= 'Yb' // @Sendable on function types
throws ::= 'K' // 'throws' annotation on function types
differentiable ::= 'Yjf' // @differentiable(_forward) on function type
differentiable ::= 'Yjr' // @differentiable(reverse) on function type
differentiable ::= 'Yjd' // @differentiable on function type
differentiable ::= 'Yjl' // @differentiable(_linear) on function type
```
Resolves rdar://76299796.
`@noDerivative` was not mangled in function types, and was resolved incorrectly when there's an ownership specifier. It is fixed by this patch with the following changes:
* Add `NoDerivative` demangle node represented by a `k` operator.
```
list-type ::= type identifier? 'k'? 'z'? 'h'? 'n'? 'd'? // type with optional label, '@noDerivative', inout convention, shared convention, owned convention, and variadic specifier
```
* Fix `NoDerivative`'s overflown offset in `ParameterTypeFlags` (`7` -> `6`).
* In type decoder and type resolver where attributed type nodes are processed, add support for nested attributed nodes, e.g. `inout @noDerivative T`.
* Add `TypeResolverContext::InoutFunctionInput` so that when we resolve an `inout @noDerivative T` parameter, the `@noDerivative T` checking logic won't get a `TypeResolverContext::None` set by the caller.
Resolves rdar://75916833.
