If the realloc for fda->entries succeeds but the realloc for fda->priv
fails, the error path frees the newly allocated entries.
However, fda->entries is neither updated to point to the new entries block
nor cleared to NULL. If realloc moved the allocation to a new block, the
old fda->entries pointer is now freed memory.
When fdarray__exit() is later called to clean up, it executes
free(fda->entries), which would trigger a double-free on that old pointer.
Reported-by: Sashiko Review <sashiko-bot@kernel.org>
Closes: https://lore.kernel.org/linux-perf-users/20260710200150.11FE71F00A3A@smtp.kernel.org
Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Under system-wide (-a) parallel streaming mode (--threads=cpu),
background recording threads can be inundated by a continuous
firehose of hardware samples generated by the OS. In this state,
a background thread's local hit count remains unequal to its
sample count, causing it to bypass the blocking fdarray__poll()
call entirely on each iteration of its recording loop.
Because the termination check relies on the POLLHUP event status
populated specifically by fdarray__poll(), bypassing it prevents
the background thread from ever recognizing that its control pipe
was closed by the main thread. This traps the background thread
in an infinite recording loop, hanging the main thread indefinitely
as it awaits a termination acknowledgment that never arrives.
Ensure teardown completion by adding explicit evlist__disable()
calls in the main thread's cleanup paths at out_child: and
out_child_no_flush:. Additionally, patch fdarray__filter() to
respect the fdarray_flag__nonfilterable flag, preventing it
from incorrectly setting the background thread's control pipe
file descriptor to -1 and clearing its revents mask upon
processing termination POLLHUP signals.
Fixes: f94563fac2 ("perf record: fix poll storm when monitored threads exit")
Assisted-by: Antigravity:gemini-3.5-flash
Signed-off-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
When `perf record` samples a multi-threaded process and one of the
target threads exits during the session, perf itself may start burning
100% CPU (up to 200% across two cores) until the session ends. A
single dead fd is sufficient to trigger this; it can be reproduced with
15 pthreads in a compute loop where one thread exits halfway through.
The root cause is two independent instances of the same defect: dead
perf_event ring-buffer fds are left in a pollfd array. When a monitored
thread exits, the kernel closes its ring-buffer fd, which then returns
POLLHUP. POSIX specifies that poll() always reports POLLHUP and POLLERR
regardless of the events mask, so any dead fd left in the array makes
poll() return immediately every time, spinning in a tight loop:
3 seconds: 256,600 poll() calls, 0 context switches, only 21 write()
Woken up count goes from ~0 to 1,300,000+
There are two affected poll paths, fixed together here:
1. Record main loop, via fdarray__filter() (tools/lib/api/fd/array.c).
Since commit 59b4412f27 ("libperf: Avoid internal moving of
fdarray fds") it only zeroes events/revents without setting fd to
-1, so poll() keeps reporting POLLHUP for the entry. Setting
fd = -1 makes poll() skip it, matching the pattern already used in
the control-fd path at tools/perf/builtin-record.c:1673.
2. BPF sideband thread, perf_evlist__poll_thread()
(tools/perf/util/sideband_evlist.c). This thread polls for
PERF_RECORD_BPF_EVENT but, unlike the main record loop, never calls
fdarray__filter() at all, so dead fds accumulate forever and it
spins at 100% CPU:
Before fix: dJiffies=101, wchan=0 (running)
After fix: dJiffies=0, wchan=do_sys_poll (blocking)
Fixed by calling the existing evlist__filter_pollfd() helper after
evlist__poll(), mirroring the main record loop. <poll.h> is
included for the POLLERR/POLLHUP macros (previously unused there).
The two fixes compose: fix 1 makes poll() ignore dead fds (fd=-1); fix
2 ensures the sideband thread actually performs the filtering. Both
paths are affected in all kernels from v5.1/v5.9 to the current master
(7.2-rc1); the source of both functions is byte-identical across them.
BPF event recording is preserved: after the fix, perf.data still
contains PERF_RECORD_BPF_EVENT records and bpf_prog_info entries.
Verified on perf 6.1.76, 6.6.143 and 7.2-rc1 with a minimal reproducer
(Woken up 1,300,000 -> 3, CPU 100% -> 0%) and an A/B orthogonal test:
keeping the unpatched binary but preventing the target thread from
exiting also makes the storm disappear, confirming the trigger.
Fixes: 59b4412f27 ("libperf: Avoid internal moving of fdarray fds")
Fixes: 657ee55319 ("perf evlist: Introduce side band thread")
Signed-off-by: Jiawei Sun <abyssmystery@gmail.com>
Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
We will use this in perf's evlist class so that it can, at
fdarray__filter() time, to unmap the associated ring buffer.
We may need to have further info associated with each fdarray entry, in
that case we'll make that int array a 'union fdarray_priv' one and put a
pointer there so that users can stash whatever they want there. For now,
an int is enough tho.
v2: Add clarification to the per array entry priv area, as well as make
it a union, which makes usage a bit longer, but if/when we make it
use more space by allowing per entry pointers existing users source
code will not have to be changed, just rebuilt.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jean Pihet <jean.pihet@linaro.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Link: http://lkml.kernel.org/n/tip-0p00bn83quck3fio3kcs9vca@git.kernel.org
The extensible file description array that grew in the perf_evlist class
can be useful for other tools, as it is not something that only evlists
need, so move it to tools/lib/api/fd to ease sharing it.
v2: Don't use {} like in:
libapi_dirs:
$(QUIET_MKDIR)mkdir -p $(OUTPUT){fs,fd}/
in Makefiles, as it will not work in some systems, as in ubuntu13.10.
v3: Add fd/*.[ch] to LIBAPIKFS_SOURCES (Fix from Jiri Olsa)
v4: Leave the fcntl(fd, O_NONBLOCK) in the evlist layer, remains to
be checked if it is really needed there, but has no place in the
fdarray class (Fix from Jiri Olsa)
v5: Remove evlist details from fdarray grow/filter tests. Improve it a
bit doing more tests about expected internal state.
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Corey Ashford <cjashfor@linux.vnet.ibm.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Jean Pihet <jean.pihet@linaro.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
Link: http://lkml.kernel.org/n/tip-kleuni3hckbc3s0lu6yb9x40@git.kernel.org
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Namhyung Kim
Ian Rogers
Jiawei Sun
Alexey Bayduraev
Alexey Budankov
Thomas Gleixner
Wang Nan
Arnaldo Carvalho de Melo