The reason why I am doing this is that in certain cases the AST captures indices
will never actually line up with partial apply capture indices since we seem to
"smush" together closures and locally defined functions.
NOTE: The reason for the really small amount of test changes is that this change
does not change the actual output by design. The only cases I had to change were
a case where we began to emit a better diagnostic and also where I added code
coverage around _ and let _ since those require ignored_use to be implemented so
that they would be diagnosed (previously we just did not emit anything so we
couldn't emit the diagnostic at the SIL level).
rdar://142661388
In terms of the test suite the only difference is that we allow for non-Sendable
types to be returned from nonisolated functions. This is safe due to the rules
of rbi. We do still error when we return non-Sendable functions across isolation
boundaries though.
The reason that I am doing this now is that I am implementing a prototype that
allows for nonisolated functions to inherit isolation from their caller. This
would have required me to implement support both in Sema for results and
arguments in SIL. Rather than implement results in Sema, I just finished the
work of transitioning the result checking out of Sema and into SIL. The actual
prototype will land in a subsequent change.
rdar://127477211
The reason that I am modifying this error is that in situations like the
following one can have a Sendable type that triggers this error since the box
containing the value is non-Sendable.
```
func methodConsuming(x: consuming SendableKlass) async throws {
try await withThrowingTaskGroup(of: Void.self) { group in
group.addTask { // expected-tns-warning {{passing closure as a 'sending' parameter risks causing data races between code in the current task and concurrent execution of the closure}}
useValue(x) // expected-tns-note {{closure captures reference to mutable var 'x' which is accessible to code in the current task}}
}
try await group.waitForAll()
}
}
```
rdar://133813644
(cherry picked from commit 36c2b3cc1330c07dcf9715f8ae88d31f9dba58c4)
This requires two major changes.
The first is that we need to teach SILGen that the isolation of an initializer
is essentially dynamic (as far as SILGen is concerned) --- that it needs to emit
code in order to get the isolation reference. To make this work, I needed to
refactor how we store the expected executor of a function so that it's not
always a constant value; instead, we'll need to emit code that DI will lower
properly. Fortunately, I can largely build on top of the work that Doug previously
did to support #isolation in these functions. The SIL we emit here around delegating
initializer calls is not ideal --- the breadcrumb hop ends up jumping to the
generic executor, and then DI actually emits the hop to the actor. This is a little
silly, but it's hard to eliminate without special-casing the self-rebinding, which
honestly we should consider rather than the weirdly global handling of that in
SILGen today. The optimizer should eliminate this hop pretty reliably, at least.
The second is that we need to teach DI to handle the pattern of code we get in
delegating initializers, where the builtin actually has to be passed the self var
rather than a class reference. This is because we don't *have* a class reference
that's consistently correct in these cases. This ended up being a fairly
straightforward generalization.
I also taught the hop_to_executor optimizer to skip over the initialization of
the default-actor header; there are a lot of simple cases where we still do emit
the prologue generic-executor hop, but at least the most trivial case is handled.
To do this better, we'd need to teach this bit of the optimizer that the properties
of self can be stored to in an initializer prior to the object having escaped, and
we don't have that information easily at hand, I think.
Fixes rdar://87485045.
Specifically:
I changed the main error message to focus on the closure and that the closure
is being accessed concurrently.
If we find that we captured a value that is the actual isolation source, we
emit that the capture is actually actor isolated.
If the captured value is in the same region as the isolated value but is not
isolated, we instead say that the value is accessible from *-isolated code or
code within the current task.
If we find multiple captures and we do not which is the actual value that was
in the same region before we formed the partial apply, we just emit a note on
the captures saying that the closure captures the value.
I changed the diagnostics from using the phrase "task-isolated" to use some
variant of accessible to code in the current task.
The idea is that in all situations we provide a breadcrumb that the user can
start investigating rather than just saying that the closure is "task-isolated".
From a preconcurrency perspective, I made it so that we apply the preconcurrency
behavior of all of the captures. This means that if one of the captures is
preconcurrency, we apply the preconcurrency restriction to the closure. This is
one step towards making it so that preconcurrency applies at the region level...
we just are not completely there yet.
rdar://133798044
The changes to allow for partial consumption unintentionally also allowed for
`self` to be consumed as a whole during `deinit`, which we don't yet want to
allow because it could lead to accidental "resurrection" and/or accidental
infinite recursion if the consuming method lets `deinit` be implicitly run
again. This makes it an error again. The experimental feature
`ConsumeSelfInDeinit` will allow it for test coverage or experimentation
purposes. rdar://132761460
Otherwise, we will assume that an async let autoclosure infers isolation from
its DeclContext... which we do not want. An async let autoclosure should always
be nonisolated + sending.
The diagnostic change that I mentioned in the header is that we were emitting
unfortunate "sending task or actor isolated could result in races" error. I
eliminated this by adding a new diagnostic for transfer non transferrable errors
happening in autoclosures. So now we emit this:
```swift
func asyncLetInferAsNonIsolated<T : Actor>(
isolation actor: isolated T
) async throws {
async let subTask: Void = {
await useValueAsyncNoReturnWithInstance(self, actor)
// expected-warning @-1:47 {{sending 'self' risks causing data races}}
// expected-note @-2 {{sending 'actor'-isolated 'self' into async let risks causing data races between nonisolated and 'actor'-isolated uses}}
}()
await subTask
```
I also noticed that we did not have enough test cases for autoclosures in
general so I also added a bunch of tests just so we can see what the current
behavior is. I think there are a few issues therein (I believe some may have
been reported due to '??').
rdar://130151318
Otherwise IRGen would crash.
It needs a bit of work to support alloc_box of generic non-copyable structs/enums with deinit, because we need to specialize the deinit functions, though they are not explicitly referenced in SIL.
Until this is supported, give an error in such cases.
Fixes a compiler crash in IRGen
rdar://130283111
Replace the assert-check if a vtable is available with a regular error message.
This cannot occur in regular builds - only if built with embedded swift and without wmo.
The command line compiler prevents this combination, but it can happen in SourceKit.
rdar://130167087
Specifically:
1. We error now if one transfers an 'inout sending' parameter and does not
reinitialize it before the end of the function.
2. We error now if one merges an 'inout sending' parameter into an actor
isolated region and do not reinitialize it with a non-actor isolated value
before the end of the function.
rdar://126303739
This isn't fully implemented yet so it would crash eventually, so instead of
letting the compiler crash put up a proper diagnostic indicating this isn't
yet implemented. rdar://129034189
Instance properties of non-sendable types cannot safely be
accessed within deinitializers. Make sure we respect `@preconcurrency`
when diagnosing these.
Just trying to slice off a larger change where I change these tests to actually
use 'sending'. This is nice to do now since it is algebraic to do.
rdar://128216574
To avoid breaking early adopters of this feature, accept attempts to `return`
a `let` binding in a noncopyable `switch` when it would be treated as a
borrow normally, with a warning that this behavior will change soon.
rdar://126775241
Specifically, I am transforming it from "may cause a race" -> "may cause a data
race". Adding data is a small thing, but it adds a bunch of nice clarity.
This PR implements first set of changes required to support autodiff for coroutines. It mostly targeted to `_modify` accessors in standard library (and beyond), but overall implementation is quite generic.
There are some specifics of implementation and known limitations:
- Only `@yield_once` coroutines are naturally supported
- VJP is a coroutine itself: it yields the results *and* returns a pullback closure as a normal return. This allows us to capture values produced in resume part of a coroutine (this is required for defers and other cleanups / commits)
- Pullback is a coroutine, we assume that coroutine cannot abort and therefore we execute the original coroutine in reverse from return via yield and then back to the entry
- It seems there is no semantically sane way to support `_read` coroutines (as we will need to "accept" adjoints via yields), therefore only coroutines with inout yields are supported (`_modify` accessors). Pullbacks of such coroutines take adjoint buffer as input argument, yield this buffer (to accumulate adjoint values in the caller) and finally return the adjoints indirectly.
- Coroutines (as opposed to normal functions) are not first-class values: there is no AST type for them, one cannot e.g. store them into tuples, etc. So, everywhere where AST type is required, we have to hack around.
- As there is no AST type for coroutines, there is no way one could register custom derivative for coroutines. So far only compiler-produced derivatives are supported
- There are lots of common things wrt normal function apply's, but still there are subtle but important differences. I tried to organize the code to enable code reuse, still it was not always possible, so some code duplication could be seen
- The order of how pullback closures are produced in VJP is a bit different: for normal apply's VJP produces both value and pullback closure via a single nested VJP apply. This is not so anymore with coroutine VJP's: yielded values are produced at `begin_apply` site and pullback closure is available only from `end_apply`, so we need to track the order in which pullbacks are produced (and arrange consumption of the values accordingly – effectively delay them)
- On the way some complementary changes were required in e.g. mangler / demangler
This patch covers the generation of derivatives up to SIL level, however, it is not enough as codegen of `partial_apply` of a coroutine is completely broken. The fix for this will be submitted separately as it is not directly autodiff-related.
---------
Co-authored-by: Andrew Savonichev <andrew.savonichev@gmail.com>
Co-authored-by: Richard Wei <rxwei@apple.com>
This diagnostic is useful around silgen_name where it validates that we do not
have any weird collisions. Sadly, it just points where one of the conflicting
elements is... with this patch, we also emit an error on the other function if
we have a SILLocation.
