Function types aren't always trivially copyable, e.g. with address-discriminated signed pointers on ARM64e. Introduce a function_cast helper and use that instead.
Remove the multiple definitions of `std::bit_cast` into a header. While
this is still not great, it does reduce the duplication. This also
silently works towards reducing a bit of the UB introduced here by
adding an inline namespace for `std` which you are not technically
allowed to use. However, by doing this, we have a clear migration path
away from this once we adopt C++20.
The C++ standard does not guarantee that the code and data pointers are
interchangeable. Recent enhancements to clang now properly identify the
improper `reinterpret_cast` between function types. Silence the warning
by switching to `std::bit_cast`. Unfortunately, this is a C++20 feature
and we are still on C++17. In the case that the compiler doesn't have a
`bit_cast` implementation, fallback with some UB to a local definition.
The way that we include COMPATIBILITY_OVERRIDE_INCLUDE_PATH freaks out the
syntax highlighting of editors like emacs. It causes the whole file to be
highlighted like it is part of the include string.
To work around this, this patch creates a separate file called
CompatibilityOverrideIncludePath.h that just includes
COMPATIBILITY_OVERRIDE_INCLUDE_PATH. So its syntax highlighting is borked, but
at least in the actual files that contain real code, the syntax highlighting is
restored.
status record that is already registered with the task. Provide more
versatile removeStatusRecord functions and update clients to use them
Radar-Id: rdar://problem/101864092
done the load or who need the oldStatus information after adding the
status record.
Change some of the memory barrier logic since we can take advantage of
load-through HW address dependency.
Radar-Id: rdar://problem/105634683
field of an ActiveTaskStatus can also be modified while the
TaskStatusRecord list is being modified. Make the StatusRecordLock
reentrant.
Radar-Id: rdar://problem/88093007
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
Apply a blanket pass of including `new` for the placement new allocation
and namespacing the call to the global placement new allocator. This
should repair the Android ARMv7 builds.
Not all targets have a 16-byte type alignment guarantee. For the types
which are not naturally aligned, provide a type specific `operator new`
overload to ensure that we are properly aligning the type on allocation
as we run the risk of under-aligned allocations otherwise.
This should no longer be needed with C++17 and newer which do a two
phase `operator new` lookup preferring
`operator new(std::size, std::align_val_t)` if needed. The base type
would be fully pre-processed away. The empty base class optimization
should help ensure that we do not pay any extra size costs for the
alignment fixes.
As we are a C++14 codebase, we must locally implement some of the
standard type_traits utilities, namely `void_t`. We take the minimal
definition here, assuming that the compiler is up-to-date with C++14 DR
reports which fixed an issue in SFINAE. We use the SFINAE for detecting
the presence of the `operator new` overload to guide the over-alignment,
which is inherited through the new `swift::overaligned_type<>` base
type.
Annotate the known classes which request explicit alignment which is
non-pointer alignment. This list was identified by
`git grep ' alignas(.*) '`.
This change adds support for WASI in stdlib tests. Some tests that expect a crash to happen had to be disabled, since there's currently no way to observe such crash from a WASI host.
This change has two parts to it:
1. Add in a new interface (addStatusRecordWithChecks) for adding task
status records that also takes in a function ref. This function ref will
be used to evaluate if current state of the parent task has any changes
that need to be propagated to the child task that has been created.
This is necessary to prevent the following race between task creation
and concurrent cancellation and escalation:
a. Parent task create child task. It does lazy relaxed loads on its own
state while doing so and propagates this state to the child.
b. Child task is created but has not been attached to the parent
task/task group.
c. Parent task gets cancelled by another thread.
d. Child task gets linked into the parent’s task status records but no
reevaluation has happened to account for changes that might have happened to
the parent after (a).
2. Move status record management functions from the
Runtime/Concurrency.h to TaskPrivate.h. Remove any corresponding
overrides that are no longer needed. Remove unused tryAddStatusRecord
method whose functionality is provided by addStatusRecordWithChecks.
Radar-Id: rdar://problem/86347801
This macro takes the string and parameters directly, and is conditionally defined to either call fprintf or ignore its arguments. This makes the call sites a little more pleasant (no #if scattered about) and ensures every log includes the thread ID and a newline automatically.
If a future task has already completed, then task_future_wait doesn't populate the future context,
but asyncLet_finish was inappropriately relying on it to recover the result buffer pointer to
destroy the value inside. There's room in asyncLet_finish's own context to put this pointer, so
do that instead. Fixes rdar://81481317.
Change the code generation patterns for `async let` bindings to use an ABI based on the following
functions:
- `swift_asyncLet_begin`, which starts an `async let` child task, but which additionally
now associates the `async let` with a caller-owned buffer to receive the result of the task.
This is intended to allow the task to emplace its result in caller-owned memory, allowing the
child task to be deallocated after completion without invalidating the result buffer.
- `swift_asyncLet_get[_throwing]`, which replaces `swift_asyncLet_wait[_throwing]`. Instead of
returning a copy of the value, this entry point concerns itself with populating the local buffer.
If the buffer hasn't been populated, then it awaits completion of the task and emplaces the
result in the buffer; otherwise, it simply returns. The caller can then read the result out of
its owned memory. These entry points are intended to be used before every read from the
`async let` binding, after which point the local buffer is guaranteed to contain an initialized
value.
- `swift_asyncLet_finish`, which replaces `swift_asyncLet_end`. Unlike `_end`, this variant
is async and will suspend the parent task after cancelling the child to ensure it finishes
before cleaning up. The local buffer will also be deinitialized if necessary. This is intended
to be used on exit from an `async let` scope, to handle cleaning up the local buffer if necessary
as well as cancelling, awaiting, and deallocating the child task.
- `swift_asyncLet_consume[_throwing]`, which combines `get` and `finish`. This will await completion
of the task, leaving the result value in the result buffer (or propagating the error, if it
throws), while destroying and deallocating the child task. This is intended as an optimization
for reading `async let` variables that are read exactly once by their parent task.
To avoid an epoch break with existing swiftinterfaces and ABI clients, the old builtins and entry
points are kept intact for now, but SILGen now only generates code using the new interface.
This new interface fixes several issues with the old async let codegen, including use-after-free
crashes if the `async let` was never awaited, and the inability to read from an `async let` variable
more than once.
rdar://77855176
introduce new options parameter to all task spawning
[Concurrency] ABI for asynclet start to accept options
[Concurrency] fix unittest usages of changed task creation ABI
[Concurrency] introduce constants for parameter indexes in ownership
[Concurrency] fix test/SILOptimizer/closure_lifetime_fixup_concurrency.swift
Changes the task, taskGroup, asyncLet wait funtion call ABIs.
To reduce code size pass the context parameters and resumption function
as arguments to the wait function.
This means that the suspend point does not need to store parent context
and resumption to the suspend point's context.
```
void swift_task_future_wait_throwing(
OpaqueValue * result,
SWIFT_ASYNC_CONTEXT AsyncContext *callerContext,
AsyncTask *task,
ThrowingTaskFutureWaitContinuationFunction *resume,
AsyncContext *callContext);
```
The runtime passes the caller context to the resume entry point saving
the load of the parent context in the resumption function.
This patch adds a `Metadata *` field to `GroupImpl`. The await entry
pointer no longer pass the metadata pointer and there is a path through
the runtime where the task future is no longer available.
`pthread_self` is not portable to all platforms. Introduce a
`_swift_get_current_thread_id` to abstract over accessing the current
thread ID. On Windows, the thread ID and thread handle are two separate
entities, unlike POSIX threads which treats them the same.
This commit changes JobFlags storage to be 32bits, but leaves the runtime
API expressed in terms of size_t. This allows us to pack an Id in the
32bits we freed up.
The offset of this Id in the AsyncTask is an ABI constant. This way
introspection tools can extract the currently running task identifier
without any need for special APIs.
Also, do this for the initial slab for the task's allocator itself.
This avoids memory allocations for async-lets.
In case the async-task's memory demand does not exceed the initial slab size, it is now completely malloc-free.
The refcount bits of an async-let task are initialized to "immortal" so that ARC operations don't have an effect on the task.
The closure does not escape the startAsyncLet - endAsyncLet scope. Even though it's (potentially) running on a different thread.
The substantial change in the runtime is to not call swift_release on the closure context if it's a non-escaping closure.
