Commit Graph

81 Commits

Author SHA1 Message Date
Mike Ash
0116da253e [Concurrency] Emit async task running/waitingOn metadata inline into CPUTrace.
This allows tracking async code execution throughout a task's lifetime.

rdar://137230240
2025-09-10 11:32:10 -04:00
Konrad 'ktoso' Malawski
358c3e9dcf [Concurrency] a few missing overloads for immediate tasks 2025-07-10 12:40:41 +09:00
Saleem Abdulrasool
cab1f40218 Concurrency: silence warning on Windows
Prefer `strncpy_s` over `strncpy` which triggers a warning. This
function ensures that the copied string is null-terminated if the string
fits or simply returns an empty string (`\0`) if the string does not
fit. Prefer to use `_TRUNCATE` to copy as much of the name as fits and
ensure that it is null-terminated.
2025-05-13 09:48:33 -07:00
Mike Ash
325b66ab20 [Concurrency] Fix alreadyLocked in withStatusRecordLock.
If the preloaded status is locked, then we need to reload it in order to distinguish between the current thread holding the lock and another thread holding the lock. Without this, if another thread holds the lock, then we won't set the is-locked bit. We'll still actually hold the lock, but other threads may perform operations locklessly if the bit is not set, which can cause a crash. By reloading status in that case, we ensure that the bit is always set correctly.

This manifested as crashes in task cancellation but could cause other task-related issues as well.

Also remove an assert of !isStatusRecordLocked() in AsyncTask::complete(). We allow other threads to access tasks and take the lock for things like cancellation, so the lock may legitimately be held at that point.

rdar://150327908
2025-04-30 13:53:31 -04:00
Konrad 'ktoso' Malawski
94423da6d3 [Concurrency] Refactor _swift_taskGroup_cancelAllChildren -> _cancel
This way we do the right thing always when cancelling the group;
and we MAY visit the child tasks if we have to.
2025-04-14 17:30:28 +09:00
Konrad 'ktoso' Malawski
80f384958a [Concurrency] Parent task cancellation must cancel task group itself.
Seems that during refactorings of child cancellations we somehow missed
also cancelling the group itself. It seems we did not have good test
coverage of the addTaskUnlessCancelled somehow and thus this slipped
through.

This adds a regression test for addTaskUnlessCancelled and fixes how we
handle the cancellation effect in TaskStatus.

resolves #80789
resolves rdar://149177600
2025-04-14 17:06:44 +09:00
Konrad `ktoso` Malawski
28c4930f4f [Concurrency] Avoid inserting handler record in already cancelled task. (#80456)
This avoids the potential to race with the triggering coming from
task_cancel, because we first set the cancelled flag, and only THEN
take the lock and iterate over the inserted records. Because of this we
could: T1 flip the cancelled bit; T2 observes that, and triggers
"immediately" during installing the handler record. T1 then proceeds to
lock records and trigger it again, causing a double trigger of the
cancellation handler.

resolves https://github.com/swiftlang/swift/issues/80161
resolves rdar://147493150
2025-04-02 19:21:18 +09:00
Mike Ash
ecdcb82b97 Merge pull request #80316 from mikeash/lock-record-begone
[Concurrency] Eliminate StatusRecordLockRecord.
2025-03-28 17:18:40 -04:00
Mike Ash
7c3b06dc4f [Concurrency] Avoid de-escalating a task when racing to escalate.
Reload oldPriority each time through the compare_exchange loop. Without this, we might race with another escalating thread and end up trying to set a priority that's lower or equal to the priority set by the other thread. This results in an assertion failure when asserts are enabled, or attempt to lower the priority of the task when asserts are not enabled.

rdar://147888768
2025-03-26 15:44:57 -04:00
Mike Ash
6397e30856 [Concurrency] Eliminate StatusRecordLockRecord.
Move to a recursive lock inline in the Task. This avoids the need to allocate a lock record and simplifies the code somewhat.

Change Task's OpaquePrivateStorage to compute its size at build time based on the sizes of its components, rather than having it be a fixed size. It appears that the fixed size was intended to be part of the ABI, but that didn't happen and we're free to change this size. We need to expand it slightly when using pthread_mutex as the recursive lock, as pthread_mutex is pretty big. Other recursive locks allow it to shrink slightly.

We don't have a recursive mutex in our Threading support code, so add a RecursiveMutex type.

rdar://113898653
2025-03-26 14:52:37 -04:00
Mike Ash
1e265c24d7 Merge pull request #80096 from mikeash/task-group-cancellation-lock-fix
[Concurrency] Fix a race when using cancelAll on a task group concurrently with child tasks being removed.
2025-03-20 11:53:31 -04:00
Mike Ash
38c4fce1a8 [Concurrency] Fix a race when using cancelAll on a task group concurrently with child tasks being removed.
_swift_taskGroup_cancelAllChildren relies on there being no concurrent modification when called from the owning task, but this is not guaranteed.

Rearrange things to always take the owning task's status record lock when walking the group's children. Split _swift_taskGroup_cancelAllChildren into two functions, one which assumes/requires the lock is already held, and one which acquires the lock. We don't have the owning task in this case, but we can either get it from the current task, or by looking at the parent of the child task we're working on.

rdar://147172991
2025-03-18 14:52:07 -04:00
Konrad `ktoso` Malawski
85e4685f90 [Concurrency] Adjust task escalation APIs to SE accepted shapes (#79988)
* [Concurrency] Adjust task escalation APIs to SE accepted shapes

* adjust test a little bit

* Fix closure lifetime in withTaskPriorityEscalationHandler

* avoid bringing workaround func into abi by marking AEIC
2025-03-18 07:05:11 +09:00
Konrad `ktoso` Malawski
a92f37998e [freestanding] prefer strncpy with known length (#80005) 2025-03-14 04:44:09 -07:00
Konrad `ktoso` Malawski
fda7f539fb Reapply "Task names" (#79562) (#79600) 2025-03-08 10:58:49 +09:00
Konrad `ktoso` Malawski
09003d6f11 Revert "Merge pull request #77609 from ktoso/wip-task-names" (#79562)
This reverts commit 4ab5d2604f.
2025-02-23 22:59:21 -08:00
Konrad `ktoso` Malawski
4ab5d2604f Merge pull request #77609 from ktoso/wip-task-names
[Concurrency] Task names
2025-02-21 22:28:33 +09:00
Saleem Abdulrasool
8040430bc7 Concurrency: silence some unused variable warnings
This cleans up some warnings from the unused variables. The variables
were referenced by `assert` cases only.
2025-02-03 09:22:50 -08:00
Michael Gottesman
cb8e8b505a Fix syntax highlighting by changing how we include COMPATIBILITY_OVERRIDE_INCLUDE_PATH.
The way that we include COMPATIBILITY_OVERRIDE_INCLUDE_PATH freaks out the
syntax highlighting of editors like emacs. It causes the whole file to be
highlighted like it is part of the include string.

To work around this, this patch creates a separate file called
CompatibilityOverrideIncludePath.h that just includes
COMPATIBILITY_OVERRIDE_INCLUDE_PATH. So its syntax highlighting is borked, but
at least in the actual files that contain real code, the syntax highlighting is
restored.
2024-10-01 16:17:16 -07:00
Konrad `ktoso` Malawski
a3e0b173ea [Concurrency] TaskExecutors may be non-swift objects; dont swift_release them (#75059) 2024-07-09 02:55:11 -07:00
Konrad `ktoso` Malawski
2ec717b115 [Concurrency] TaskExecutor ownership fixes (#74000) 2024-06-14 22:56:33 +09:00
Konrad `ktoso` Malawski
e9b25d1091 [Concurrency] Move innermost record assertion to handle more gracefully 2024-05-28 16:58:38 +09:00
Mike Ash
0f226b6667 [Concurrency] Refine getResumeFunctionForLogging to avoid reading invalid future contexts.
When using a future adapter, the resume context may not be valid after the task starts running. Only peer through the adapter when we're starting to run.

rdar://126298035
2024-05-21 16:22:40 -04:00
cui fliter
127077b3aa chore: fix some comments
Signed-off-by: cui fliter <imcusg@gmail.com>
2024-03-05 17:23:22 +08:00
Rick van Voorden
f8ae46b3f3 [inclusive-language] changed sanity to soundness 2024-01-25 18:18:02 -08:00
Konrad `ktoso` Malawski
828f589be4 Initial Task Executor implementation Task(on:), addTask(on:) etc. (#68793)
Co-authored-by: John McCall <rjmccall@gmail.com>
2023-12-12 17:14:24 +09:00
Rokhini Prabhu
9b6c873239 Escalate an actor if a job in the queue of the actor has been escalated.
Radar-Id: rdar://problem/101864092
2023-02-28 13:52:26 -08:00
Rokhini Prabhu
534811c3dc When a task gets enqueued on an executor, start tracking which executor
it is enqueued on. This way, we have the necessary bookkeeping to
escalate an executor when a task that is enqueued, is escalated.

Radar-Id: rdar://problem/101864092
2023-02-25 17:01:03 -08:00
Rokhini Prabhu
d77306729b Improve TaskDependencyStatusRecord so that it can now track the executor
you are enqueued on

Radar-Id: rdar://problem/101864092
2023-02-25 16:58:02 -08:00
Rokhini Prabhu
d0aa55670a Provide an updateStatusRecord function which allows clients to update a
status record that is already registered with the task. Provide more
versatile removeStatusRecord functions and update clients to use them

Radar-Id: rdar://problem/101864092
2023-02-25 16:55:38 -08:00
Rokhini Prabhu
e8d9fd91f7 Tell TSAN that we know that we're doing with CONSUME semantics so that
it doesn't flag false positive issues

Radar-Id: rdar://problem/105634683
2023-02-23 17:42:00 -08:00
Rokhini Prabhu
3c0f4f79fe Provide a convenience for addStatusRecord for clients who have already
done the load or who need the oldStatus information after adding the
status record.

Change some of the memory barrier logic since we can take advantage of
load-through HW address dependency.

Radar-Id: rdar://problem/105634683
2023-02-20 23:14:47 -08:00
Rokhini Prabhu
e052ccef31 Create the notion of a TaskDependencyStatusRecord which tracks what a
task is blocked on. We can use this information to then perform live
priority escalation to a task future.

Radar-Id: rdar://problem/88093007
2023-02-08 17:29:55 -08:00
Rokhini Prabhu
2e6baf279d Add support to addStatusRecord and removeStatusRecord so that the flags
field of an ActiveTaskStatus can also be modified while the
TaskStatusRecord list is being modified. Make the StatusRecordLock
reentrant.

Radar-Id: rdar://problem/88093007
2023-02-08 17:29:05 -08:00
Rokhini Prabhu
62852262b4 Remove dead code with task nearest deadline logic
Radar-Id: rdar://problem/88093007
2023-02-07 14:34:29 -08:00
John McCall
7f737d235d Synchronize with cancellation when removing a task from a task group
We were detaching the child by just modifying the list, but the cancellation path was assuming that that would not be done without holding the task status lock.

This patch just fixes the current runtime; the back-deployment side is complicated.

Fixes rdar://88398824
2022-10-29 00:10:28 -04:00
Mike Ash
9c849e431c [Concurrency] Fix memory ordering around task cancellation.
Load task status with an acquire when canceling a task, to synchronize with the store-release that comes when updating a task's status.

Add explicit TSan calls in cancellation, as well as withStatusRecordLock and addStatusRecord, to avoid TSan complaining about data races when canceling a task.

Add a test that checks for TSan-reported data races when canceling a task.

rdar://93892417
2022-09-14 13:39:55 -04:00
Alastair Houghton
0e9318cec5 [Threading] Put everything through git clang-format.
Just formatting changes.

rdar://90776105
2022-06-07 07:39:53 +01:00
Alastair Houghton
f5bdb858e0 [Threading] Create new threading library and use it.
Moved all the threading code to one place.  Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.

rdar://90776105
2022-06-07 07:39:51 +01:00
Alex Hoppen
4aa2bbbf06 Revert "Merge pull request #42447 from al45tair/eng/PR-90776105"
This reverts commit 8bcb71140f, reversing
changes made to c4dd271d36.
2022-06-02 18:03:23 +02:00
Alastair Houghton
b5bd267ff1 [Threading] Put everything through git clang-format.
Just formatting changes.

rdar://90776105
2022-05-24 14:57:41 +01:00
Alastair Houghton
63a09007a1 [Threading] Create new threading library and use it.
Moved all the threading code to one place.  Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.

rdar://90776105
2022-05-24 14:57:39 +01:00
Kavon Farvardin
22323fbda9 correct format strings when building with SWIFT_TASK_DEBUG_LOG enabled 2022-05-10 13:00:45 -07:00
Rokhini Prabhu
98a8bdbd38 Task Escalation support on arm64_32 Apple platforms
Radar-Id: rdar://problem/88600541
2022-02-10 11:50:31 -08:00
Rokhini Prabhu
66d4af0b01 Task priority escalation on Apple platforms
A task can be in one of 4 states over its lifetime:

    (a) suspended
    (b) enqueued
    (c) running
    (d) completed

This change provides priority inversion avoidance support if a task gets
escalated when it is in state (a), (c), (d).

Radar-Id: rdar://problem/76127624
2022-02-07 16:34:46 -08:00
Rokhini Prabhu
519e60c43e Allow concurrent modifications to the ActiveTaskStatus while another
thread has the task status record lock.

Today, if a thread is holding the StatusRecordLock, then no other
modification of the task status is possible - including a thread
starting to execute the task or stopping execution of the task.
However, the TaskStatusRecordLock is really about protecting the linked
list that is maintained in the ActiveTaskStatus. As such, other
operations which don't need to look at that linked list of us records
really shouldn't have to block on the StatusRecordLock.

This change allows for concurrent modification of the veTaskStatus while
the TaskStatusRecordLock is held. In particular, a task can cancelled,
escalated, start and stop running, all while another ad is holding onto
the task's StatusRecordLock. In the event of cancellation and
escalation, the task's StatusRecordLock must be n in order to propagate
cancellation and escalation to its child tasks is not needed to cancel
or escalate the task itself.

Radar-Id: rdar://problem/76127624
2022-02-07 16:34:46 -08:00
Rokhini Prabhu
90bf37784c NFC: Rename isLocked to isStatusRecordLocked to more explicitly
distinguish from the drain lock of the task that will be added next

Radar-Id: rdar://problem/76127624
2022-02-04 13:45:40 -08:00
Rokhini Prabhu
a2a5c64d4b Change task's base and escalated priority propagation rules for
detached, unstructured and structured concurrency child tasks

Radar-Id: rdar://problem/86100376
2022-01-24 07:50:28 -08:00
Rokhini Prabhu
a4fe57f230 Merge pull request #40606 from apple/rokhinip/86347801-task-creation-escalation-race
Resolve race between task creation and concurrent escalation and cancellation.
2022-01-12 12:14:09 +08:00
Mike Ash
0977920217 [Concurrency] Add tracing for major operations in the concurrency runtime.
Each trace point is declared as a function in the new `Tracing.h` header. These functions are called from the appropriate places in the concurrency runtime.

On Darwin, an implementation of these functions is provided which uses the `os/signpost.h` API to emit signpost events/intervals.

When the signpost API is not available, no-op stub implementations are provided. Implementations for other OSes can be provided by providing implementations of the trace functions for that OS.

rdar://81858487
2022-01-07 11:35:40 -05:00