`Builtin.once` has type `(Builtin.RawPointer, (Builtin.RawPointer) -> ())`
at Swift level, but lazy global init passes its initializer as `() -> ()`,
so their callee and caller signatures doesn't match.
* [Distributed] dist actor always has default executor (currently)
* [Distributed] extra test for missing makeEncoder
* [DistributedDecl] Add DistributedActorSystem to known SDK types
* [DistributedActor] ok progress on getting the system via witness
* [Distributed] allow hop-to `let any: any X` where X is DistActor
* [Distributed] AST: Add an accessor to determine whether type is distributed actor
- Classes have specialized method on their declarations
- Archetypes and existentials check their conformances for
presence of `DistributedActor` protocol.
* [Distributed] AST: Account for distributed members declared in class extensions
`getConcreteReplacementForProtocolActorSystemType` should use `getSelfClassDecl`
otherwise it wouldn't be able to find actor if the member is declared in an extension.
* [Distributed] fix ad-hoc requirement checks for 'mutating'
[PreChecker] LookupDC might be null, so account for that
* [Distributed] Completed AST synthesis for dist thunk
* [Distributed][ASTDumper] print pretty distributed in right color in AST dumps
* wip on making the local/remote calls
* using the _local to mark the localCall as known local
* [Distributed] fix passing Never when not throwing
* fix lifetime of mangled string
* [Distributed] Implement recordGenericSubstitution
* [Distributed] Dont add .
* [Distributed] dont emit thunk when func broken
* [Distributed] fix tests; cleanups
* [Distributed] cleanup, move is... funcs to DistributedDecl
* [Distributed] Remove SILGen for distributed thunks, it is in Sema now!
* [Distributed] no need to check stored props in protocols
* remote not used flag
* fix mangling test
* [Distributed] Synthesis: Don't re-use AST nodes for `decodeArgument` references
* [Distributed] Synthesis: Make sure that each thunk parameter has an internal name
* [Distributed/Synthesis] NFC: Add a comment regarding empty internal parameter names
* [Distributed] NFC: Adjust distributed thunk manglings in the accessor section test-cases
* cleanup
* [Distributed] NFC: Adjust distributed thunk manglings in the accessor thunk test-cases
* review follow ups
* xfail some linux tests for now so we can land the AST thunk
* Update distributed_actor_remote_functions.swift
Co-authored-by: Pavel Yaskevich <xedin@apache.org>
The main point of this change is to make sure that a shared function always has a body: both, in the optimizer pipeline and in the swiftmodule file.
This is important because the compiler always needs to emit code for a shared function. Shared functions cannot be referenced from outside the module.
In several corner cases we missed to maintain this invariant which resulted in unresolved-symbol linker errors.
As side-effect of this change we can drop the shared_external SIL linkage and the IsSerializable flag, which simplifies the serialization and linkage concept.
Lowering hops to actors expected the executor to either be an optional
builtin executor type or an actor type. The type coming from
`getMainExecutor` is just a builtin executor, not an optional. As a
result, `LowerHopToActor::emitGetExecutor` would get called in
`LowerHopToActor::processHop`, and would try looking up the builtin
executor type, thinking it was an actor type. This would fail because we
didn't have an actor type.
```
auto actorConf = module->lookupConformance(actorType, actorProtocol);
assert(actorConf &&
"hop_to_executor with actor that doesn't conform to Actor");
```
The end result was hitting this assert here, saying that the "actor"
doesn't conform to actor.
The fix is to wrap the executor in an optional.
Async functions are now expected to set ExpectedExecutor in their
prologue (and, generally, immediately hop to it). I updated the
prologue code for a bunch of function emission, most of which was
uninteresting. Top-level code was not returning to the main
executor, which is now fixed; fortunately, we weren't assuming
that we were on the main executor yet.
We had some code that only kicked in when an ExpectedExecutor
wasn't set which made us capture the current executor before
a hop and then return to it later. This code has been removed;
there's no situation in which save-and-return is the semantically
correct thing to do given the possibility of hop optimization.
I suspect it could also have led to crashes if the current
executor is being kept alive only because it's currently running
code. If we ever add async functions that are supposed to inherit
their caller's executor, we should have the caller pass the right
executor down to it.
This is the first half of SE-0338; the second, sendability
enforcement, is much more complicated, and Doug has volunteered
to do it.
Fixes rdar://79284465, as well as some tests that were XFAILed
on Windows.
This patch adds the SILGen side of generating the asynchronous main
entrypoint for top-level code. The behavior is the same as with the
asynchronous MainType entrypoint.
Leaks checking is not thread safe and e.g. lldb creates multiple SILModules in multiple threads, which would result in false alarms.
Ideally we would make it thread safe, e.g. by putting the instruction counters in the SILModule, but this would be a big effort and it's not worth doing it. Leaks checking in the frontend's and SILOpt's SILModule (not including SILModules created for module interface building) is a good enough test.
rdar://84688015
Fix a few minor issues in the type checker and SILGen to properly cope with
distributed functions defined within extensions of distributed actors.
While here, centralize the logic that adds the "_remote_" function.
Fixes rdar://84325525.
This was a relict from the -sil-serialize-all days. This linkage doesn't make any sense because a private function cannot be referenced from another module (or file, in case of non-wmo compilation).
This patch updates the asynchronous main function to run the first thunk
of the function synchronously through a call to `swift_job_run`.
The runloop is killed by exiting or aborting the task that it is running
on. As such, we need to ensure that the task contains an async function
that either calls exit explicitly or aborts. The AsyncEntryPoint, that
contains this code, was added in the previous patch. This patch adds the
pieces for the actual implementation of this behaviour as well as adding
the necessary code to start the runloop.
There are now four layers of main functions before hitting the "real"
code.
@main: This is the actual main entrypoint of the program. This
constructs the task containing @async_main, grabs the main executor,
runs swift_job_run to run the first part synchronously, and finally
kicks off the runloop with a call to _asyncMainDrainQueue. This is
generated in the call to `emitAsyncMainThreadStart`.
@async_main: This thunk exists to ensure that the main function calls
`exit` at some point so that the runloop stops. It also handles emitting
an error if the user-written main function throws.
e.g:
```
func async_main() async -> () {
do {
try await Main.$main()
exit(0)
} catch {
_errorInMain(error)
}
}
```
Main.$main(): This still has the same behaviour as with the
synchronous case. It just calls `try await Main.main()` and exists to
simplify typechecking.
Main.main(): This is the actual user-specified main. It serves the same
purpose as in the synchronous, allowing the programmer to write code,
but it's async!
The control flow in `emitFunctionDefinition` is a little confusing (to
me anyway), so here it is spelled out:
If the main function is synchronous, the `constant.kind` will be a
`SILDeclRef::Kind::EntryPoint`, but the `decl` won't be async, so it
drops down to `emitArtificalTopLevel` anyway.
If the main function is async and we're generating `@main`, the
`constant.kind` will be `SILDeclRef::Kind::AsyncEntryPoint`, so we also
call `emitArtificalTopLevel`. `emitArtificalTopLevel` is responsible for
detecting whether the decl is async and deciding whether to emit code to
extract the argc/argv variables that get passed into the actual main
entrypoint to the program. If we're generating the `@async_main` body,
the kind will be `SILDeclRef::Kind::EntryPoint` and the `decl` will be
async, so we grab the mainEntryPoint decl and call
`emitAsyncMainThreadStart` to generate the wrapping code.
Note; there is a curious change in `SILLocation::getSourceLoc()`
where instead of simply checking `isFilenameAndLocation()`, I change it
to `getStorageKind() == FilenameAndLocationKind`. This is because the
SILLocation returned is to a FilenameAndLocationKind, but the actual
storage returns true for the call to `isNull()` inside of the
`isFilenameAndLocation()` call. This results in us incorrectly falling
through to the `getASTNode()` call below that, which asserts when asked
to get the AST node of a location.
I also did a little bit of refactoring in the SILGenModule for grabbing
intrinsics. Previously, there was only a `getConcurrencyIntrinsic`
function, which would only load FuncDecls out of the concurrency
module. The `exit` function is in the concurrency shims module, so I
refactored the load code to take a ModuleDecl to search from.
The emitBuiltinCreateAsyncTask function symbol is exposed from
SILGenBuiltin so that it is available from SILGenFunction. There is a
fair bit of work involved going from what is available at the SGF to
what is needed for actually calling the CreateAsyncTask builtin, so in
order to avoid additional maintenance, it's good to re-use that.
The AsyncEntryPoint represents the thunk that is wrapped in a task. This
thunk is used to ensure that the main function explicitly calls "exit",
and to properly unwrap and report any unhandled errors returned from the
user-written main. The function takes on the name `@async_main` in the
emitted SIL.
Literal closures are only ever directly referenced in the context of the expression they're written in,
so it's wasteful to emit them at their fully-substituted calling convention and then reabstract them if
they're passed directly to a generic function. Avoid this by saving the abstraction pattern of the context
before emitting the closure, and then lowering its main entry point's calling convention at that
level of abstraction. Generalize some of the prolog/epilog code to handle converting arguments and returns
to the correct representation for a different abstraction level.
Literal closures are only ever directly referenced in the context of the expression they're written in,
so it's wasteful to emit them at their fully-substituted calling convention and then reabstract them if
they're passed directly to a generic function. Avoid this by saving the abstraction pattern of the context
before emitting the closure, and then lowering its main entry point's calling convention at that
level of abstraction. Generalize some of the prolog/epilog code to handle converting arguments and returns
to the correct representation for a different abstraction level.
Literal closures are only ever directly referenced in the context of the expression they're written in,
so it's wasteful to emit them at their fully-substituted calling convention and then reabstract them if
they're passed directly to a generic function. Avoid this by saving the abstraction pattern of the context
before emitting the closure, and then lowering its main entry point's calling convention at that
level of abstraction. Generalize some of the prolog/epilog code to handle converting arguments and returns
to the correct representation for a different abstraction level.
Change the code generation patterns for `async let` bindings to use an ABI based on the following
functions:
- `swift_asyncLet_begin`, which starts an `async let` child task, but which additionally
now associates the `async let` with a caller-owned buffer to receive the result of the task.
This is intended to allow the task to emplace its result in caller-owned memory, allowing the
child task to be deallocated after completion without invalidating the result buffer.
- `swift_asyncLet_get[_throwing]`, which replaces `swift_asyncLet_wait[_throwing]`. Instead of
returning a copy of the value, this entry point concerns itself with populating the local buffer.
If the buffer hasn't been populated, then it awaits completion of the task and emplaces the
result in the buffer; otherwise, it simply returns. The caller can then read the result out of
its owned memory. These entry points are intended to be used before every read from the
`async let` binding, after which point the local buffer is guaranteed to contain an initialized
value.
- `swift_asyncLet_finish`, which replaces `swift_asyncLet_end`. Unlike `_end`, this variant
is async and will suspend the parent task after cancelling the child to ensure it finishes
before cleaning up. The local buffer will also be deinitialized if necessary. This is intended
to be used on exit from an `async let` scope, to handle cleaning up the local buffer if necessary
as well as cancelling, awaiting, and deallocating the child task.
- `swift_asyncLet_consume[_throwing]`, which combines `get` and `finish`. This will await completion
of the task, leaving the result value in the result buffer (or propagating the error, if it
throws), while destroying and deallocating the child task. This is intended as an optimization
for reading `async let` variables that are read exactly once by their parent task.
To avoid an epoch break with existing swiftinterfaces and ABI clients, the old builtins and entry
points are kept intact for now, but SILGen now only generates code using the new interface.
This new interface fixes several issues with the old async let codegen, including use-after-free
crashes if the `async let` was never awaited, and the inability to read from an `async let` variable
more than once.
rdar://77855176
* [Distributed] Initial distributed checking
* [Distributed] initial types shapes and conform to DistributedActor
* [Distributed] Require Codable params and return types
* [Distributed] initial synthesis of fields and constructors
* [Distributed] Field and initializer synthesis
* [Distributed] Codable requirement on distributed funcs; also handle <T: Codable>
* [Distributed] handle generic type params which are Codable in dist func
[Distributed] conformsToProtocol after all
* [Distributed] Implement remote flag on actors
* Implement remote flag on actors
* add test
* actor initializer that sets remote flag
[Distributed] conformances getting there
* [Distributed] dont require async throws; cleanup compile tests
* [Distributed] do not synthesize default implicit init, only our special ones
* [Distributed] properly synth inits and properties; mark actorTransport as _distributedActorIndependent
Also:
- do not synthesize default init() initializer for dist actor
* [Distributed] init(transport:) designated and typechecking
* [Distributed] dist actor initializers MUST delegate to local-init
* [Distributed] check if any ctors in delegation call init(transport:)
* [Distributed] check init(transport:) delegation through many inits; ban invoking init(resolve:using:) explicitly
* [Distributed] disable IRGen test for now
* [Distributed] Rebase cleanups
* [Concurrent] transport and address are concurrent value
* [Distributed] introduce -enable-experimental-distributed flag
* rebase adjustments again
* rebase again...
* [Distributed] distributed functions are implicitly async+throws outside the actor
* [Distributed] implicitly throwing and async distributed funcs
* remove printlns
* add more checks to implicit function test
* [Distributed] resolve initializer now marks the isRemote actor flag
* [Distributed] distributedActor_destroy invoked instead, rather than before normal
* [Distributed] Generate distributed thunk for actors
* [distributed] typechecking for _remote_ functions existing, add tests for remote funcs
* adding one XFAIL'ed task & actor lifetime test
The `executor_deinit1` test fails 100% of the time
(from what I've seen) so I thought we could track
and see when/if someone happens to fix this bug.
Also, added extra coverage for #36298 via `executor_deinit2`
* Fix a memory issue with actors in the runtime system, by @phausler
* add new test that now passes because of patch by @phausler
See previous commit in this PR.
Test is based on one from rdar://74281361
* fix all tests that require the _remote_ function stubs
* Do not infer @actorIndependent onto `let` decls
* REVERT_ME: remove some tests that hacky workarounds will fail
* another flaky test, help build toolchain
* [Distributed] experimental distributed implies experimental concurrency
* [Distributed] Allow distributed function that are not marked async or throws
* [Distributed] make attrs SIMPLE to get serialization generated
* [Distributed] ActorAddress must be Hashable
* [Distributed] Implement transport.actorReady call in local init
* cleanup after rebase
* [Distributed] add availability attributes to all distributed actor code
* cleanup - this fixed some things
* fixing up
* fixing up
* [Distributed] introduce new Distributed module
* [Distributed] diagnose when missing 'import _Distributed'
* [Distributed] make all tests import the module
* more docs on address
* [Distributed] fixup merge issues
* cleanup: remove unnecessary code for now SIMPLE attribute
* fix: fix getActorIsolationOfContext
* [Distributed] cmake: depend on _concurrency module
* fixing tests...
* Revert "another flaky test, help build toolchain"
This reverts commit 83ae6654dd.
* remove xfail
* clenup some IR and SIL tests
* cleanup
* [Distributed] fix cmake test and ScanDependencies/can_import_with_map.swift
* [Distributed] fix flags/build tests
* cleanup: use isDistributed wherever possible
* [Distributed] don't import Dispatch in tests
* dont link distributed in stdlib unittest
* trying always append distributed module
* cleanups
* [Distributed] move all tests to Distributed/ directory
* [lit] try to fix lit test discovery
* [Distributed] update tests after diagnostics for implicit async changed
* [Distributed] Disable remote func tests on Windows for now
* Review cleanups
* [Distributed] fix typo, fixes Concurrency/actor_isolation_objc.swift
* [Distributed] attributes are DistributedOnly (only)
* cleanup
* [Distributed] cleanup: rely on DistributedOnly for guarding the keyword
* Update include/swift/AST/ActorIsolation.h
Co-authored-by: Doug Gregor <dgregor@apple.com>
* introduce isAnyThunk, minor cleanup
* wip
* [Distributed] move some type checking to TypeCheckDistributed.cpp
* [TypeCheckAttr] remove extra debug info
* [Distributed/AutoDiff] fix SILDeclRef creation which caused AutoDiff issue
* cleanups
* [lit] remove json import from lit test suite, not needed after all
* [Distributed] distributed functions only in DistributedActor protocols
* [Distributed] fix flag overlap & build setting
* [Distributed] Simplify noteIsolatedActorMember to not take bool distributed param
* [Distributed] make __isRemote not public
* [Distributed] Fix availability and remove actor class tests
* [actorIndependent] do not apply actorIndependent implicitly to values where it would be illegal to apply
* [Distributed] disable tests until issue fixed
Co-authored-by: Dario Rexin <drexin@apple.com>
Co-authored-by: Kavon Farvardin <kfarvardin@apple.com>
Co-authored-by: Doug Gregor <dgregor@apple.com>
* Revert "[Distributed] disable tests until issue fixed"
This reverts commit 0a04278920.
* Revert "[Distributed] Initial `distributed` actors and functions and new module (#37109)"
This reverts commit 814ede0cf3.
* [Distributed] Initial distributed checking
* [Distributed] initial types shapes and conform to DistributedActor
* [Distributed] Require Codable params and return types
* [Distributed] initial synthesis of fields and constructors
* [Distributed] Field and initializer synthesis
* [Distributed] Codable requirement on distributed funcs; also handle <T: Codable>
* [Distributed] handle generic type params which are Codable in dist func
[Distributed] conformsToProtocol after all
* [Distributed] Implement remote flag on actors
* Implement remote flag on actors
* add test
* actor initializer that sets remote flag
[Distributed] conformances getting there
* [Distributed] dont require async throws; cleanup compile tests
* [Distributed] do not synthesize default implicit init, only our special ones
* [Distributed] properly synth inits and properties; mark actorTransport as _distributedActorIndependent
Also:
- do not synthesize default init() initializer for dist actor
* [Distributed] init(transport:) designated and typechecking
* [Distributed] dist actor initializers MUST delegate to local-init
* [Distributed] check if any ctors in delegation call init(transport:)
* [Distributed] check init(transport:) delegation through many inits; ban invoking init(resolve:using:) explicitly
* [Distributed] disable IRGen test for now
* [Distributed] Rebase cleanups
* [Concurrent] transport and address are concurrent value
* [Distributed] introduce -enable-experimental-distributed flag
* rebase adjustments again
* rebase again...
* [Distributed] distributed functions are implicitly async+throws outside the actor
* [Distributed] implicitly throwing and async distributed funcs
* remove printlns
* add more checks to implicit function test
* [Distributed] resolve initializer now marks the isRemote actor flag
* [Distributed] distributedActor_destroy invoked instead, rather than before normal
* [Distributed] Generate distributed thunk for actors
* [distributed] typechecking for _remote_ functions existing, add tests for remote funcs
* adding one XFAIL'ed task & actor lifetime test
The `executor_deinit1` test fails 100% of the time
(from what I've seen) so I thought we could track
and see when/if someone happens to fix this bug.
Also, added extra coverage for #36298 via `executor_deinit2`
* Fix a memory issue with actors in the runtime system, by @phausler
* add new test that now passes because of patch by @phausler
See previous commit in this PR.
Test is based on one from rdar://74281361
* fix all tests that require the _remote_ function stubs
* Do not infer @actorIndependent onto `let` decls
* REVERT_ME: remove some tests that hacky workarounds will fail
* another flaky test, help build toolchain
* [Distributed] experimental distributed implies experimental concurrency
* [Distributed] Allow distributed function that are not marked async or throws
* [Distributed] make attrs SIMPLE to get serialization generated
* [Distributed] ActorAddress must be Hashable
* [Distributed] Implement transport.actorReady call in local init
* cleanup after rebase
* [Distributed] add availability attributes to all distributed actor code
* cleanup - this fixed some things
* fixing up
* fixing up
* [Distributed] introduce new Distributed module
* [Distributed] diagnose when missing 'import _Distributed'
* [Distributed] make all tests import the module
* more docs on address
* [Distributed] fixup merge issues
* cleanup: remove unnecessary code for now SIMPLE attribute
* fix: fix getActorIsolationOfContext
* [Distributed] cmake: depend on _concurrency module
* fixing tests...
* Revert "another flaky test, help build toolchain"
This reverts commit 83ae6654dd.
* remove xfail
* clenup some IR and SIL tests
* cleanup
* [Distributed] fix cmake test and ScanDependencies/can_import_with_map.swift
* [Distributed] fix flags/build tests
* cleanup: use isDistributed wherever possible
* [Distributed] don't import Dispatch in tests
* dont link distributed in stdlib unittest
* trying always append distributed module
* cleanups
* [Distributed] move all tests to Distributed/ directory
* [lit] try to fix lit test discovery
* [Distributed] update tests after diagnostics for implicit async changed
* [Distributed] Disable remote func tests on Windows for now
* Review cleanups
* [Distributed] fix typo, fixes Concurrency/actor_isolation_objc.swift
* [Distributed] attributes are DistributedOnly (only)
* cleanup
* [Distributed] cleanup: rely on DistributedOnly for guarding the keyword
* Update include/swift/AST/ActorIsolation.h
Co-authored-by: Doug Gregor <dgregor@apple.com>
* introduce isAnyThunk, minor cleanup
* wip
* [Distributed] move some type checking to TypeCheckDistributed.cpp
* [TypeCheckAttr] remove extra debug info
* [Distributed/AutoDiff] fix SILDeclRef creation which caused AutoDiff issue
* cleanups
* [lit] remove json import from lit test suite, not needed after all
* [Distributed] distributed functions only in DistributedActor protocols
* [Distributed] fix flag overlap & build setting
* [Distributed] Simplify noteIsolatedActorMember to not take bool distributed param
* [Distributed] make __isRemote not public
Co-authored-by: Dario Rexin <drexin@apple.com>
Co-authored-by: Kavon Farvardin <kfarvardin@apple.com>
Co-authored-by: Doug Gregor <dgregor@apple.com>
Allow SILDeclRef to refer to the main program
entry-point, which will either be for a main
SourceFile, or a synthetic main such as an `@main`
decl. Adjust the various SILDeclRef related
functions to handle this new case, and change the
emission to go through `emitFunctionDefinition`.
This change will allow the entry-point for an `@main`
decl (and eventually a main SourceFile) to be
emitted on-demand from its symbol name.
Through various means, it is possible for a synchronous actor-isolated
function to escape to another concurrency domain and be called from
outside the actor. The problem existed previously, but has become far
easier to trigger now that `@escaping` closures and local functions
can be actor-isolated.
Introduce runtime detection of such data races, where a synchronous
actor-isolated function ends up being called from the wrong executor.
Do this by emitting an executor check in actor-isolated synchronous
functions, where we query the executor in thread-local storage and
ensure that it is what we expect. If it isn't, the runtime complains.
The runtime's complaints can be controlled with the environment
variable `SWIFT_UNEXPECTED_EXECUTOR_LOG_LEVEL`:
0 - disable checking
1 - warn when a data race is detected
2 - error and abort when a data race is detected
At an implementation level, this introduces a new concurrency runtime
entry point `_checkExpectedExecutor` that checks the given executor
(on which the function should always have been called) against the
executor on which is called (which is in thread-local storage). There
is a special carve-out here for `@MainActor` code, where we check
against the OS's notion of "main thread" as well, so that `@MainActor`
code can be called via (e.g.) the Dispatch library's
`DispatchQueue.main.async`.
The new SIL instruction `extract_executor` performs the lowering of an
actor down to its executor, which is implicit in the `hop_to_executor`
instruction. Extend the LowerHopToExecutor pass to perform said
lowering.
Prints a regular error instead of crashing.
The check is done in SILGen, because it's simple. We could also do it earlier, but I don't see a strong reason for this.
rdar://75950093
Previously, the name of the entry point function was always main. Here,
a new frontend flag is added to enable an arbitrary name to be
specified.
rdar://58275758
- `Mangle::ASTMangler::mangleAutoDiffDerivativeFunction()` and `Mangle::ASTMangler::mangleAutoDiffLinearMap()` accept original function declarations and return a mangled name for a derivative function or linear map. This is called during SILGen and TBDGen.
- `Mangle::DifferentiationMangler` handles differentiation function mangling in the differentiation transform. This part is necessary because we need to perform demangling on the original function and remangle it as part of a differentiation function mangling tree in order to get the correct substitutions in the mangled derivative generic signature.
A mangled differentiation function name includes:
- The original function.
- The differentiation function kind.
- The parameter indices for differentiation.
- The result indices for differentiation.
- The derivative generic signature.
