The resignID call within the initializer moved into DI, because an assignment to
the actorSystem, and thus initialization of the id, is no longer guaranteed to happen.
Thus, while we previously could model the resignation as a clean-up emitted on all
unwind paths in an initializer, now it's conditional, based on whether the id was
initialized. This is exactly what DI is designed to do, so we inject the resignation
call just before we call the identity's deinit.
This attribute is designed for let-bound variables whose initializing
assignment is synthesized by the compiler. This assignment is
expected to happen at some point before DefiniteInitialization has
run, which is the pass that verifies whether the compiler truly
initialized the variable.
I generally expect that this will never be a user-facing feature, and
that the synthesized assignment happens in SILGen.
After https://github.com/apple/swift/pull/40793, alloc_boxes all have
their lifetimes protected by a lexical borrow scope. In that PR, DI had
been updated to see through begin_borrow instructions from a project_box
to a mark_uninitialized. It did not, however, correct the end_borrow
instructions when destroy_values of mark_uninitializeds were replaced
with destroy_addrs of project_boxes. That is done here.
In a bit more detail, in the following context
%box = alloc_box
%mark_uninit = mark_uninitialized %box
%lifetime = begin_borrow [lexical] %mark_uninit
%proj_box = project_box %lifetime
When it is not statically known whether a field is initialized, we are
replacing the instruction
// before
destroy_value %mark_uninit
// after
with the following diamond
// before
%initialized = load
cond_br %initialized, yes, no
yes:
destroy_addr %proj_box
br bottom
no:
br bottom
bottom:
dealloc_box %box
br keep_going
keep_going:
// after
Doing so is problematic, though, because by SILGen construction the
destroy_value is always preceded by an end_borrow:
end_borrow %lifetime
destroy_value %mark_uninit
Previously, that end_borrow remained above the
%initialized = load
instruction in the above. That was invalid because the the newly
introduced
destroy_addr %proj_box
was a use of the borrow scope (%proj_box is a projection of the
begin_borrow) and consequently must be within the borrow scope.
Note also that it would not be sufficient to simply emit the diamond
before the end_borrow. The end_borrow must come before the destruction
of the value whose lifetime it is protecting (%box), and the diamond
contains the instruction to destroy that value (dealloc_box) in its
bottom block.
To resolve this issue, just move the end_borrow instruction from where
it was to before the dealloc box. (This is actually done by moving it to
the top of the diamond's "continue" block prior to the emission of that
dealloc_box instruction.)
rdar://89984216
The distributed case is distinguishable from the non-distributed case
based on the actor type itself for those rare cases where we care. The
vast majority of code is simplified by treating this identically to
`ActorInstance`.
This is a combination of fixes:
- inject hops in self-isolated delegating actor initializers
after self becomes initialized.
- fix sendable and isolation for convenience inits
- fix bug in distributed actor inits that I introduced when
implementing flow-isolation.
- fix / add test coverage.
Flow-isolation is a diagnostic SIL pass that finds
unsafe accesses to properties in initializers and
deinitializers that cannot gain isolation to otherwise
protect those accesses from concurrent modifications.
See SE-327 for more details about how and why it exists.
This commit includes changes and features like:
- The removal of the escaping-use restriction
- Flow-isolation that works properly with `defer` statements
- Flow-isolation with an emphasis on helpful diagnostics.
It also includes known issues like:
- Local / nonescaping functions are not analyzed by
flow-isolation, despite it being technically possible.
The main challenge in supporting it efficiently is that
such functions do not have a single exit-point, like
a `defer`. In particular, arbitrary functions can throw
so there are points where nonisolation should _not_ flow
out of the function at a call-site in the initializer, etc.
- The implementation of the flow-isolation pass is not
particularly memory efficient; it relies on BitDataflow
even though the particular flow problem is simple.
So, a more efficient implementation would be specialized for
this particular problem, etc.
There are also some changes to the Swift language itself: defer
will respect its context when deciding its property access kind.
Previously, a defer in an initializer would always access a stored
property through its accessor methods, instead of doing so directly
like its enclosing function might. This inconsistency is unfortunate,
so for Swift 6+ we make this consistent. For Swift 5, only a defer
in a function that is a member of the following kinds of types
will gain this consistency:
- an actor type
- any nominal type that is actor-isolated, excluding UnsafeGlobalActor.
These types are still rather new, so there is much less of a chance of
breaking expected behaviors around defer. In particular, the danger is
that users are relying on the behavior of defer triggering a property
observer within an init or deinit, when it would not be triggering it
without the defer.
* [Distributed] towards DistributedActorSystem; synthesize the id earlier, since Identifiable.id
* Fix execute signature to what Pavel is working with
* funcs are ok in sil
* fixed lifetime of id in inits
* fix distributed_actor_deinit
* distributed_actor_local
* update more tests
fixing tests
fix TBD test
fix Serialization/distributed
fix irgen test
Fix null pointer crashes
* prevent issues with null func ptrs and fix Distributed prorotocol test
* fix deinit sil test
This instruction is similar to a copy_addr except that it marks a move of an
address that has to be checked. In order to keep the memory lifetime verifier
happy, the semantics before the checker runs are the mark_unresolved_move_addr is
equivalent to copy_addr [init] (not copy_addr [take][init]).
The use of this instruction is that Mandatory Inlining converts builtin "move"
to a mark_unresolved_move_addr when inlining the function "_move" (the only
place said builtin is invoked).
This is then run through a special checker (that is later in this PR) that
either proves that the mark_unresolved_move_addr can actually be a move in which
case it converts it to copy_addr [take][init] or if it can not be a move, emit
an error and convert the instruction to a copy_addr [init]. After this is done
for all instructions, we loop back through again and emit an error on any
mark_unresolved_move_addr that were not processed earlier allowing for us to
know that we have completeness.
NOTE: The move kills checker for addresses is going to run after Mandatory
Inlining, but before predictable memory opts and friends.
The returnsSelf check in DI was missing proper handling for failable inits.
This patch fixes that by specifically recognizing the tail-end of the success path
in a failable init, and treating that block as "returning self".
Previously, we were ignoring the explicit `nonisolated`
attribute on an actor initializer. This only has a
noticable effect if the initializer is `async`: an actor
hop is being emitted even though isolation was explicitly
not requested.
While we internally treat the isolation of an unadorned
actor initializer as if it were `nonisolated`, if the
programmer explicitly requests it, then it should be honored
in an async init too. Thus, this patch tells definite initialization
to skip the insertion of the hop, instead treating a `nonisolated`
initializer as though it has the escaping-use restriction.
resolves rdar://84164173
Refactor the code that generates SIL to call into the distributed actor
transport to eliminate duplication and better cope with concrete actor
transports. Centralize the knowledge of which actor transport is used
with a given distributed actor type.
Immediately after the hop_to_executor in an async, distributed
actor init, we need to notify the transport that the actor is ready.
This patch does not yet account for more complex cases. In particular,
we will need a mechanism to prevent multiple calls to actorReady,
which can happen if a loop appears in the init:
distributed actor Dactor {
var x: Int
init(tport: ActorTransport, count: Int) async {
var i = count
repeat {
self.x = count
// hop is injected here
i -= 1
} while i > 0
}
}
Those instructions should not be "visible" for debugging and for diagnostic messages.
Fixes a wrong "unreachable code" warning.
https://bugs.swift.org/browse/SR-14873
rdar://80237870
Because `self` can be used unrestricted after it is
fully-initialized in an async actor init, we perform
a hop_to_executor(self) immediately after `self`
is fully-initialized on all paths within the
initializer.
If we did not, then code could race with the
initializer if it, e.g., spawns a task from
the initializer and mutates actor state that
the initializer then reads. By hopping to the
executor, we prevent that task from running
until _after_ the initializer completes.
A uniqueness rule for 'self' is enforced throughout
the following types of actor initializers:
1. synchronous
2. global-actor isolated
This means that 'self' cannot be used for anything,
even after 'self' is fully initialized, other than
for access to stored properties. Method calls are
considered escaping uses of 'self', since it is passed
implicitly to the method of computed property when invoked.
Also, an actor's convenience init now treats self as
nonisolated.
This provides a way to perform follow-up work after
self is fully initialized, when creating the
actor from a synchronous context.
Resolves rdar://78790369
* [Distributed] Initial distributed checking
* [Distributed] initial types shapes and conform to DistributedActor
* [Distributed] Require Codable params and return types
* [Distributed] initial synthesis of fields and constructors
* [Distributed] Field and initializer synthesis
* [Distributed] Codable requirement on distributed funcs; also handle <T: Codable>
* [Distributed] handle generic type params which are Codable in dist func
[Distributed] conformsToProtocol after all
* [Distributed] Implement remote flag on actors
* Implement remote flag on actors
* add test
* actor initializer that sets remote flag
[Distributed] conformances getting there
* [Distributed] dont require async throws; cleanup compile tests
* [Distributed] do not synthesize default implicit init, only our special ones
* [Distributed] properly synth inits and properties; mark actorTransport as _distributedActorIndependent
Also:
- do not synthesize default init() initializer for dist actor
* [Distributed] init(transport:) designated and typechecking
* [Distributed] dist actor initializers MUST delegate to local-init
* [Distributed] check if any ctors in delegation call init(transport:)
* [Distributed] check init(transport:) delegation through many inits; ban invoking init(resolve:using:) explicitly
* [Distributed] disable IRGen test for now
* [Distributed] Rebase cleanups
* [Concurrent] transport and address are concurrent value
* [Distributed] introduce -enable-experimental-distributed flag
* rebase adjustments again
* rebase again...
* [Distributed] distributed functions are implicitly async+throws outside the actor
* [Distributed] implicitly throwing and async distributed funcs
* remove printlns
* add more checks to implicit function test
* [Distributed] resolve initializer now marks the isRemote actor flag
* [Distributed] distributedActor_destroy invoked instead, rather than before normal
* [Distributed] Generate distributed thunk for actors
* [distributed] typechecking for _remote_ functions existing, add tests for remote funcs
* adding one XFAIL'ed task & actor lifetime test
The `executor_deinit1` test fails 100% of the time
(from what I've seen) so I thought we could track
and see when/if someone happens to fix this bug.
Also, added extra coverage for #36298 via `executor_deinit2`
* Fix a memory issue with actors in the runtime system, by @phausler
* add new test that now passes because of patch by @phausler
See previous commit in this PR.
Test is based on one from rdar://74281361
* fix all tests that require the _remote_ function stubs
* Do not infer @actorIndependent onto `let` decls
* REVERT_ME: remove some tests that hacky workarounds will fail
* another flaky test, help build toolchain
* [Distributed] experimental distributed implies experimental concurrency
* [Distributed] Allow distributed function that are not marked async or throws
* [Distributed] make attrs SIMPLE to get serialization generated
* [Distributed] ActorAddress must be Hashable
* [Distributed] Implement transport.actorReady call in local init
* cleanup after rebase
* [Distributed] add availability attributes to all distributed actor code
* cleanup - this fixed some things
* fixing up
* fixing up
* [Distributed] introduce new Distributed module
* [Distributed] diagnose when missing 'import _Distributed'
* [Distributed] make all tests import the module
* more docs on address
* [Distributed] fixup merge issues
* cleanup: remove unnecessary code for now SIMPLE attribute
* fix: fix getActorIsolationOfContext
* [Distributed] cmake: depend on _concurrency module
* fixing tests...
* Revert "another flaky test, help build toolchain"
This reverts commit 83ae6654dd.
* remove xfail
* clenup some IR and SIL tests
* cleanup
* [Distributed] fix cmake test and ScanDependencies/can_import_with_map.swift
* [Distributed] fix flags/build tests
* cleanup: use isDistributed wherever possible
* [Distributed] don't import Dispatch in tests
* dont link distributed in stdlib unittest
* trying always append distributed module
* cleanups
* [Distributed] move all tests to Distributed/ directory
* [lit] try to fix lit test discovery
* [Distributed] update tests after diagnostics for implicit async changed
* [Distributed] Disable remote func tests on Windows for now
* Review cleanups
* [Distributed] fix typo, fixes Concurrency/actor_isolation_objc.swift
* [Distributed] attributes are DistributedOnly (only)
* cleanup
* [Distributed] cleanup: rely on DistributedOnly for guarding the keyword
* Update include/swift/AST/ActorIsolation.h
Co-authored-by: Doug Gregor <dgregor@apple.com>
* introduce isAnyThunk, minor cleanup
* wip
* [Distributed] move some type checking to TypeCheckDistributed.cpp
* [TypeCheckAttr] remove extra debug info
* [Distributed/AutoDiff] fix SILDeclRef creation which caused AutoDiff issue
* cleanups
* [lit] remove json import from lit test suite, not needed after all
* [Distributed] distributed functions only in DistributedActor protocols
* [Distributed] fix flag overlap & build setting
* [Distributed] Simplify noteIsolatedActorMember to not take bool distributed param
* [Distributed] make __isRemote not public
* [Distributed] Fix availability and remove actor class tests
* [actorIndependent] do not apply actorIndependent implicitly to values where it would be illegal to apply
* [Distributed] disable tests until issue fixed
Co-authored-by: Dario Rexin <drexin@apple.com>
Co-authored-by: Kavon Farvardin <kfarvardin@apple.com>
Co-authored-by: Doug Gregor <dgregor@apple.com>
* Revert "[Distributed] disable tests until issue fixed"
This reverts commit 0a04278920.
* Revert "[Distributed] Initial `distributed` actors and functions and new module (#37109)"
This reverts commit 814ede0cf3.
* [Distributed] Initial distributed checking
* [Distributed] initial types shapes and conform to DistributedActor
* [Distributed] Require Codable params and return types
* [Distributed] initial synthesis of fields and constructors
* [Distributed] Field and initializer synthesis
* [Distributed] Codable requirement on distributed funcs; also handle <T: Codable>
* [Distributed] handle generic type params which are Codable in dist func
[Distributed] conformsToProtocol after all
* [Distributed] Implement remote flag on actors
* Implement remote flag on actors
* add test
* actor initializer that sets remote flag
[Distributed] conformances getting there
* [Distributed] dont require async throws; cleanup compile tests
* [Distributed] do not synthesize default implicit init, only our special ones
* [Distributed] properly synth inits and properties; mark actorTransport as _distributedActorIndependent
Also:
- do not synthesize default init() initializer for dist actor
* [Distributed] init(transport:) designated and typechecking
* [Distributed] dist actor initializers MUST delegate to local-init
* [Distributed] check if any ctors in delegation call init(transport:)
* [Distributed] check init(transport:) delegation through many inits; ban invoking init(resolve:using:) explicitly
* [Distributed] disable IRGen test for now
* [Distributed] Rebase cleanups
* [Concurrent] transport and address are concurrent value
* [Distributed] introduce -enable-experimental-distributed flag
* rebase adjustments again
* rebase again...
* [Distributed] distributed functions are implicitly async+throws outside the actor
* [Distributed] implicitly throwing and async distributed funcs
* remove printlns
* add more checks to implicit function test
* [Distributed] resolve initializer now marks the isRemote actor flag
* [Distributed] distributedActor_destroy invoked instead, rather than before normal
* [Distributed] Generate distributed thunk for actors
* [distributed] typechecking for _remote_ functions existing, add tests for remote funcs
* adding one XFAIL'ed task & actor lifetime test
The `executor_deinit1` test fails 100% of the time
(from what I've seen) so I thought we could track
and see when/if someone happens to fix this bug.
Also, added extra coverage for #36298 via `executor_deinit2`
* Fix a memory issue with actors in the runtime system, by @phausler
* add new test that now passes because of patch by @phausler
See previous commit in this PR.
Test is based on one from rdar://74281361
* fix all tests that require the _remote_ function stubs
* Do not infer @actorIndependent onto `let` decls
* REVERT_ME: remove some tests that hacky workarounds will fail
* another flaky test, help build toolchain
* [Distributed] experimental distributed implies experimental concurrency
* [Distributed] Allow distributed function that are not marked async or throws
* [Distributed] make attrs SIMPLE to get serialization generated
* [Distributed] ActorAddress must be Hashable
* [Distributed] Implement transport.actorReady call in local init
* cleanup after rebase
* [Distributed] add availability attributes to all distributed actor code
* cleanup - this fixed some things
* fixing up
* fixing up
* [Distributed] introduce new Distributed module
* [Distributed] diagnose when missing 'import _Distributed'
* [Distributed] make all tests import the module
* more docs on address
* [Distributed] fixup merge issues
* cleanup: remove unnecessary code for now SIMPLE attribute
* fix: fix getActorIsolationOfContext
* [Distributed] cmake: depend on _concurrency module
* fixing tests...
* Revert "another flaky test, help build toolchain"
This reverts commit 83ae6654dd.
* remove xfail
* clenup some IR and SIL tests
* cleanup
* [Distributed] fix cmake test and ScanDependencies/can_import_with_map.swift
* [Distributed] fix flags/build tests
* cleanup: use isDistributed wherever possible
* [Distributed] don't import Dispatch in tests
* dont link distributed in stdlib unittest
* trying always append distributed module
* cleanups
* [Distributed] move all tests to Distributed/ directory
* [lit] try to fix lit test discovery
* [Distributed] update tests after diagnostics for implicit async changed
* [Distributed] Disable remote func tests on Windows for now
* Review cleanups
* [Distributed] fix typo, fixes Concurrency/actor_isolation_objc.swift
* [Distributed] attributes are DistributedOnly (only)
* cleanup
* [Distributed] cleanup: rely on DistributedOnly for guarding the keyword
* Update include/swift/AST/ActorIsolation.h
Co-authored-by: Doug Gregor <dgregor@apple.com>
* introduce isAnyThunk, minor cleanup
* wip
* [Distributed] move some type checking to TypeCheckDistributed.cpp
* [TypeCheckAttr] remove extra debug info
* [Distributed/AutoDiff] fix SILDeclRef creation which caused AutoDiff issue
* cleanups
* [lit] remove json import from lit test suite, not needed after all
* [Distributed] distributed functions only in DistributedActor protocols
* [Distributed] fix flag overlap & build setting
* [Distributed] Simplify noteIsolatedActorMember to not take bool distributed param
* [Distributed] make __isRemote not public
Co-authored-by: Dario Rexin <drexin@apple.com>
Co-authored-by: Kavon Farvardin <kfarvardin@apple.com>
Co-authored-by: Doug Gregor <dgregor@apple.com>
* Refactoring: replace "Destination" and the ownership qualifier by a single "Mode". This represents much better the mode how the instruction is to be lowered. NFC
* Make assign_by_wrapper printable and parseable.
* Fix lowering of the assign modes for indirect results of the init-closure: The indirect result was initialized and not assigned to. The fix is to insert a destroy_addr before calling the init closure. This fixes a memory lifetime error and/or a memory leak. Found by inspection.
* Fix an iterator-invalidation crash in RawSILInstLowering
* Add tests for lowering assign_by_wrapper.
Specifically before this PR, if a caller did not customize a specific callback
of InstModCallbacks, we would store a static default std::function into
InstModCallbacks. This means that we always would have an indirect jump. That is
unfortunate since this code is often called in loops.
In this PR, I eliminate this problem by:
1. I made all of the actual callback std::function in InstModCallback private
and gave them a "Func" postfix (e.x.: deleteInst -> deleteInstFunc).
2. I created public methods with the old callback names to actually call the
callbacks. This ensured that as long as we are not escaping callbacks from
InstModCallback, this PR would not result in the need for any source changes
since we are changing a call of a std::function field to a call to a method.
3. I changed all of the places that were escaping inst mod's callbacks to take
an InstModCallback. We shouldn't be doing that anyway.
4. I changed the default value of each callback in InstModCallbacks to be a
nullptr and changed the public helper methods to check if a callback is
null. If the callback is not null, it is called, otherwise the getter falls
back to an inline default implementation of the operation.
All together this means that the cost of a plain InstModCallback is reduced and
one pays an indirect function cost price as one customizes it further which is
better scalability.
P.S. as a little extra thing, I added a madeChange field onto the
InstModCallback. Now that we have the helpers calling the callbacks, I can
easily insert instrumentation like this, allowing for users to pass in
InstModCallback and see if anything was RAUWed without needing to specify a
callback.
LifetimeChecker::shouldEmitError in order to avoid incorrectly
populating EmittedErrorLocs when no error is emitted.
This fixes a few corner cases where DI would error out or crash
while assigning to a wrapped property with a nonmutating setter.
It would be more abstractly correct if this got DI support so
that we destroy the member if the constructor terminates
abnormally, but we can get to that later.
In a constructor, SILGen lowers type(of: self) to a value_metatype
instruction, but 'self' might not have been initialized yet if
this is a convenience initializer, so we replace it with a use of
the 'self' metatype argument.
However getDynamicSelfMetadata() asserts if 'self' is a non-class
type. Since we know we're inside of a proper method here and not a
closure that captured 'self', we can use getSelfArgument() instead.
Fixes https://bugs.swift.org/browse/SR-12665 / rdar://problem/62481587.
DI had trouble with this pattern:
%s = struct_element_addr ...
%t0 = tuple_element_addr %s, 0
%t1 = tuple_element_addr %s, 1
%f = function_ref ...
apply %f(%t0, %t1)
This is because the NonLoadUses map only stored a single use of a
tuple element per instruction, preventing instructions such as
'apply' from being able to use multiple tuple elements.
In other cases where this comes up, such as with 'assign' and
'copy_addr', DI scalarizes the tuple operation by projecting
each component, however clearly this can't be easily done with
an 'apply'.
Instead, we can get DI out of the business of scalarization, at
least for instructions which are known to perform an unconditional
initialization.
We do this by changing the NonLoadUses map to store a vector of
DIMemoryUse IDs instead of a single value.
In a designated initializer of a non-root class, 'self' becomes
fully initialized after the 'super.init' call, at which point
escaping uses of 'self' become valid, and releases of 'self' are
lowered to a 'strong_release' instruction, which runs the
deinitializer.
In a root class, 'self' becomes fully initialized after all stored
properties have been initialized, at which point escaping uses of
'self' become valid.
However, DI would still lower a conditional destroy of 'self' by
destroying any initialized stored properties and freeing the object
with 'dealloc_partial_ref'. This is incorrect, because 'self' may
have escaped.
In the non-conditional destroy case, we correctly lowered the
release to a 'strong_release' if all stored properties are known
to be initialized.
Fix DI to handle the conditional destroy case by first checking if all
bits in the control variable are set, and releasing the instance with
'strong_release' if so. The 'dealloc_partial_ref' is only emitted
if not at least one stored property was not initialized.
This ensures that we don't deallocate an instance that may have
escaped.
Fixes <https://bugs.swift.org/browse/SR-13439>, <rdar://problem/67746791>,
<https://bugs.swift.org/browse/SR-13355>, <rdar://problem/67361228>.
While we don't need to destroy these fields, we still need to know
when the class is _semantically_ fully initialized, since this will
determine if we're going to release the class, running the deinitializer,
or if we're going to deallocate the memory for the instance directly.
Previously we would just forward the cleanup and create a normal "destroy"
cleanup resulting in early deallocation and use after frees along error paths.
As part of this I also had to tweak how DI recognizes self init writebacks to
not use SILLocations. This is approach is more robust (since we aren't relying
on SourceLocs/have verifiers to make sure we don't violate SIL) and also avoids
issues from the write back store not necessarily have the same SILLocation.
<rdar://problem/59830255>
This is achieved in 3 steps:
1. CSApply detects assignments to property wrappers inside constructors, and produces an `inout` expr instead of a `load`, which it normally would because nonmutating setters take the `self` by-value. This is necessary becasue the assign_by_wrapper instruction expects an address type for its $1 operand.
2. SILGenLValue now emits the assign_by_wrapper pattern for such setters, ignoring the fact that they capture `self` by value. It also introduces an additional load instruction for the setter patrial_apply because the setter signature still expects a value and we now have an address (because of (1)).
3. DefiniteInitialization specifically ignores load instructions used to produce a `self` value for a setter referenced on assign_by_wrapper because it will be deleted by lowering anyway.
Resolves rdar://problem/60600911 and rdar://problem/52280477
