Yan Zhai 387daae8b0 bpf: report RCU QS in cpumap kthread ...

[ Upstream commit 00bf631224 ]

When there are heavy load, cpumap kernel threads can be busy polling
packets from redirect queues and block out RCU tasks from reaching
quiescent states. It is insufficient to just call cond_resched() in such
context. Periodically raise a consolidated RCU QS before cond_resched
fixes the problem.

Fixes: 6710e11269 ("bpf: introduce new bpf cpu map type BPF_MAP_TYPE_CPUMAP")
Reviewed-by: Jesper Dangaard Brouer <hawk@kernel.org>
Signed-off-by: Yan Zhai <yan@cloudflare.com>
Acked-by: Paul E. McKenney <paulmck@kernel.org>
Acked-by: Jesper Dangaard Brouer <hawk@kernel.org>
Link: https://lore.kernel.org/r/c17b9f1517e19d813da3ede5ed33ee18496bb5d8.1710877680.git.yan@cloudflare.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>

2024-03-26 18:21:37 -04:00

..

bpf

bpf: report RCU QS in cpumap kthread

2024-03-26 18:21:37 -04:00

cgroup

cgroup: Remove duplicates in cgroup v1 tasks file

2023-10-19 23:05:37 +02:00

configs

…

debug

kdb: Fix a potential buffer overflow in kdb_local()

2024-01-25 14:52:54 -08:00

dma

dma-mapping: clear dev->dma_mem to NULL after freeing it

2024-01-25 14:52:38 -08:00

entry

entry/rcu: Check TIF_RESCHED _after_ delayed RCU wake-up

2023-03-30 12:47:51 +02:00

events

perf: Fix the nr_addr_filters fix

2024-02-23 08:54:52 +01:00

futex

futex: Don't include process MM in futex key on no-MMU

2023-11-20 11:08:13 +01:00

gcov

gcov: add support for checksum field

2022-12-31 13:14:47 +01:00

irq

genirq/generic_chip: Make irq_remove_generic_chip() irqdomain aware

2023-11-28 16:56:30 +00:00

kcsan

kcsan: Don't expect 64 bits atomic builtins from 32 bits architectures

2023-07-23 13:47:12 +02:00

livepatch

livepatch: Fix missing newline character in klp_resolve_symbols()

2023-11-20 11:08:25 +01:00

locking

lockdep: Fix block chain corruption

2023-12-03 07:31:23 +01:00

power

PM: hibernate: Enforce ordering during image compression/decompression

2024-02-23 08:54:23 +01:00

printk

printk: Consolidate console deferred printing

2023-09-23 11:09:59 +02:00

rcu

rcu-tasks: Provide rcu_trace_implies_rcu_gp()

2024-03-26 18:21:11 -04:00

sched

sched/fair: Take the scheduling domain into account in select_idle_core()

2024-03-26 18:21:15 -04:00

time

timekeeping: Fix cross-timestamp interpolation for non-x86

2024-03-26 18:21:15 -04:00

trace

bpf: Remove trace_printk_lock

2024-03-01 13:21:43 +01:00

.gitignore

…

acct.c

acct: fix potential integer overflow in encode_comp_t()

2022-12-31 13:14:40 +01:00

async.c

async: Introduce async_schedule_dev_nocall()

2024-02-23 08:54:25 +01:00

audit_fsnotify.c

audit: fix potential double free on error path from fsnotify_add_inode_mark

2022-08-31 17:16:33 +02:00

audit_tree.c

…

audit_watch.c

audit: don't WARN_ON_ONCE(!current->mm) in audit_exe_compare()

2023-11-28 16:56:27 +00:00

audit.c

audit: Send netlink ACK before setting connection in auditd_set

2024-02-23 08:54:37 +01:00

audit.h

audit: log AUDIT_TIME_* records only from rules

2022-04-08 14:23:06 +02:00

auditfilter.c

…

auditsc.c

audit: fix possible soft lockup in __audit_inode_child()

2023-09-19 12:22:39 +02:00

backtracetest.c

…

bounds.c

…

capability.c

…

cfi.c

cfi: Fix __cfi_slowpath_diag RCU usage with cpuidle

2022-06-22 14:22:04 +02:00

compat.c

sched_getaffinity: don't assume 'cpumask_size()' is fully initialized

2023-04-05 11:24:53 +02:00

configs.c

…

context_tracking.c

…

cpu_pm.c

…

cpu.c

hrtimers: Push pending hrtimers away from outgoing CPU earlier

2023-12-13 18:36:31 +01:00

crash_core.c

kernel/crash_core: suppress unknown crashkernel parameter warning

2021-12-29 12:28:49 +01:00

crash_dump.c

…

cred.c

cred: switch to using atomic_long_t

2023-12-20 15:17:37 +01:00

delayacct.c

…

dma.c

…

exec_domain.c

…

exit.c

exit: Use READ_ONCE() for all oops/warn limit reads

2023-02-01 08:27:22 +01:00

extable.c

…

fail_function.c

kernel/fail_function: fix memory leak with using debugfs_lookup()

2023-03-11 13:57:38 +01:00

fork.c

kernel/fork: beware of __put_task_struct() calling context

2023-09-23 11:09:55 +02:00

freezer.c

…

gen_kheaders.sh

…

groups.c

…

hung_task.c

…

iomem.c

…

irq_work.c

…

jump_label.c

…

kallsyms.c

kallsyms: Make kallsyms_on_each_symbol generally available

2023-12-13 18:36:45 +01:00

kcmp.c

…

Kconfig.freezer

…

Kconfig.hz

…

Kconfig.locks

…

Kconfig.preempt

…

kcov.c

…

kexec_core.c

kexec: fix a memory leak in crash_shrink_memory()

2023-07-23 13:46:52 +02:00

kexec_elf.c

…

kexec_file.c

kexec: support purgatories with .text.hot sections

2023-06-21 15:59:14 +02:00

kexec_internal.h

panic, kexec: make __crash_kexec() NMI safe

2023-04-20 12:13:57 +02:00

kexec.c

kernel: kexec: copy user-array safely

2023-11-28 16:56:16 +00:00

kheaders.c

kheaders: Use array declaration instead of char

2023-05-11 23:00:17 +09:00

kmod.c

…

kprobes.c

kprobes: Fix to handle forcibly unoptimized kprobes on freeing_list

2024-01-25 14:52:31 -08:00

ksysfs.c

kexec: turn all kexec_mutex acquisitions into trylocks

2023-04-20 12:13:57 +02:00

kthread.c

kthread: add the helper function kthread_run_on_cpu()

2023-03-30 12:47:42 +02:00

latencytop.c

…

Makefile

futex: Move to kernel/futex/

2022-12-31 13:14:04 +01:00

module_signature.c

…

module_signing.c

…

module-internal.h

…

module.c

modules: wait do_free_init correctly

2024-03-26 18:21:31 -04:00

notifier.c

…

nsproxy.c

…

padata.c

crypto: pcrypt - Fix hungtask for PADATA_RESET

2023-11-28 16:56:18 +00:00

panic.c

exit: Use READ_ONCE() for all oops/warn limit reads

2023-02-01 08:27:22 +01:00

params.c

…

pid_namespace.c

rcu-tasks: Fix synchronize_rcu_tasks() VS zap_pid_ns_processes()

2023-03-10 09:39:09 +01:00

pid.c

…

profile.c

profiling: fix shift too large makes kernel panic

2022-08-17 14:24:04 +02:00

ptrace.c

ptrace: Reimplement PTRACE_KILL by always sending SIGKILL

2022-06-09 10:22:29 +02:00

range.c

…

reboot.c

kernel/reboot: emergency_restart: Set correct system_state

2023-11-28 16:56:31 +00:00

regset.c

…

relay.c

relayfs: fix out-of-bounds access in relay_file_read

2023-05-11 23:00:18 +09:00

resource_kunit.c

…

resource.c

dax/kmem: Fix leak of memory-hotplug resources

2023-03-10 09:40:08 +01:00

rseq.c

rseq: Remove broken uapi field layout on 32-bit little endian

2022-04-08 14:23:10 +02:00

scftorture.c

scftorture: Forgive memory-allocation failure if KASAN

2023-09-23 11:09:55 +02:00

scs.c

…

seccomp.c

seccomp: Invalidate seccomp mode to catch death failures

2022-02-16 12:56:38 +01:00

signal.c

signal handling: don't use BUG_ON() for debugging

2022-07-21 21:24:42 +02:00

smp.c

locking/csd_lock: Change csdlock_debug from early_param to __setup

2022-08-17 14:24:24 +02:00

smpboot.c

…

smpboot.h

…

softirq.c

timers/nohz: Last resort update jiffies on nohz_full IRQ entry

2023-08-16 18:22:04 +02:00

stackleak.c

gcc-plugins/stackleak: Use noinstr in favor of notrace

2022-02-23 12:03:07 +01:00

stacktrace.c

stacktrace: move filter_irq_stacks() to kernel/stacktrace.c

2022-04-13 20:59:28 +02:00

static_call_inline.c

static_call: Don't make __static_call_return0 static

2022-04-13 20:59:28 +02:00

static_call.c

static_call: Don't make __static_call_return0 static

2022-04-13 20:59:28 +02:00

stop_machine.c

…

sys_ni.c

kernel/sys_ni: add compat entry for fadvise64_64

2022-08-31 17:16:33 +02:00

sys.c

getrusage: use sig->stats_lock rather than lock_task_sighand()

2024-03-15 10:48:22 -04:00

sysctl-test.c

…

sysctl.c

sched/rt: Disallow writing invalid values to sched_rt_period_us

2024-03-01 13:21:43 +01:00

task_work.c

…

taskstats.c

…

test_kprobes.c

…

torture.c

torture: Fix hang during kthread shutdown phase

2023-08-30 16:18:19 +02:00

tracepoint.c

…

tsacct.c

taskstats: Cleanup the use of task->exit_code

2022-01-27 11:05:35 +01:00

ucount.c

ucounts: Handle wrapping in is_ucounts_overlimit

2022-02-23 12:03:20 +01:00

uid16.c

…

uid16.h

…

umh.c

…

up.c

…

user_namespace.c

ucounts: Fix systemd LimitNPROC with private users regression

2022-03-08 19:12:42 +01:00

user-return-notifier.c

…

user.c

…

usermode_driver.c

…

utsname_sysctl.c

…

utsname.c

…

watch_queue.c

kernel: watch_queue: copy user-array safely

2023-11-28 16:56:16 +00:00

watchdog_hld.c

watchdog/perf: more properly prevent false positives with turbo modes

2023-07-23 13:46:52 +02:00

watchdog.c

watchdog: move softlockup_panic back to early_param

2023-11-28 16:56:28 +00:00

workqueue_internal.h

…

workqueue.c

Revert "workqueue: remove unused cancel_work()"

2023-12-08 08:48:03 +01:00