Create a new source file: req.c and add the request sending entry
functions:
__fuse_simple_request()
fuse_simple_background()
fuse_simple_notify_reply()
Introduce transport layer sending functions that are called by the
respective fs layer function:
fuse_chan_send()
fuse_chan_send_bg()
fuse_chan_send_notify_reply()
Move calculation of request header fields uid, gid and pid from
fuse_get_req() and fuse_force_creads() to a new helper: fuse_fill_creds().
These fileds are now passed to the transport layer via struct fuse_args.
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Only used by FUSE_INIT and CUSE_INIT, these can store the relevant pointer
in their structs derived from fuse_args.
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
- rename fuse_abort_conn() to fuse_chan_abort(), pass fuse_chan pointer
instead of fuse_conn
- pass an abort_with_err argument that tells fuse_dev_(read|write) to
return with ECONNABORTED instead of ENODEV
- move fc->aborted to fch->abort_with_err
- rename fuse_wait_aborted() to fuse_chan_wait_aborted()
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Add helpers (get and set functions mainly) that cleanly separate the
layers.
Remove #include "fuse_dev_i.h" from:
- inode.c
- file.c
- control.c
Remove #include "dev_uring_i.h" from inode.c.
[Li Wang: drop redundant initializer in process_init_limits()]
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
And document which members they protect.
end_polls() is called with both, outer fch->lock is probably unnecessary,
but doesn't hurt for now.
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Move the 'fiq' member from fuse_conn to fuse_chan.
Move iqueue related structure definitions and function declarations from
"fuse_i.h" to "fuse_dev_i.h".
Add a fuse_dev_chan_new() helper, that returns a fuse_chan initialized with
the fuse_dev_fiq_ops.
Add a fuse_chan_release() function, that calls fiq->ops->release().
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
The goal is to separate transport layer stuff out from struct fuse_conn,
leaving just the filesystem related members.
Add a new object referenced from fuse_conn. This patch just implements the
allocation and freeing of this object.
Following patches will move transport related members from fuse_conn to
fuse_chan.
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
This marks the first step in cleanly separating the transport layer from
the filesystem layer.
Add "dev.h", which will contain the interface definition for the transport
layer.
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
When a background request completes via the io_uring path, the
background queue gets flushed to dispatch pending background requests,
but this is done before the connection-level background counters
(fc->num_background, fc->active_background) are properly accounted,
which may reduce effective queue depth to one.
The connection-level counters are decremented in fuse_request_end(), but
flush_bg_queue() flushes the /dev/fuse path queue (fc->bg_queue), not
the io_uring per-queue bg one, which means pending uring background
requests on the queue are never dispatched in this path.
Fix this by accounting the connection-level background counters first
before flushing the queue's background queue. Since
fuse_request_bg_finish() clears FR_BACKGROUND, fuse_request_end() will
skip the background cleanup branch entirely, which avoids any
double-decrements; it will call the wake_up(&req->waitq) branch but this
is effectively a no-op as background requests have no waiters on
req->waitq.
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Fixes: 857b0263f3 ("fuse: Allow to queue bg requests through io-uring")
Cc: stable@vger.kernel.org
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
If device_add() succeeds during CUSE initialization but a subsequent
step (cdev_alloc() or cdev_add()) fails, the error path calls
put_device() without first calling device_del(). This leaks the
devtmpfs entry created by device_add(), leaving a stale /dev/<name>
node that persists until reboot.
Since the cuse_conn is never linked into cuse_conntbl on the failure
path, cuse_channel_release() sees cc->dev == NULL and skips
device_unregister(), so no other code path cleans up the node.
This has several consequences:
- The device name is permanently poisoned: any subsequent attempt to
create a CUSE device with the same name hits the stale sysfs entry,
device_add() fails, and the new device is aborted.
- The collision manifests as ENODEV returned to userspace with no
dmesg diagnostic, making it very difficult to debug.
- The failure is self-perpetuating: once a name is leaked, all future
attempts with that name fail identically.
Fix this by introducing an err_dev label that calls device_del() to
undo device_add() before falling through to err_unlock. The existing
err_unlock path from a device_add() failure correctly skips device_del()
since the device was never added.
Testing instructions can be found at the lore link below.
Link: https://lore.kernel.org/all/20260408-wip-cuse-leak-fix-v1-0-1c028d575e97@redhat.com/
Signed-off-by: Alberto Ruiz <aruiz@redhat.com>
Fixes: 151060ac13 ("CUSE: implement CUSE - Character device in Userspace")
Cc: stable@vger.kernel.org
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Revert the fuse part of commit c9ba789dad ("VFS: introduce
start_creating_noperm() and start_removing_noperm()").
Commit c9ba789dad ("VFS: introduce start_creating_noperm() and
start_removing_noperm()") caused a regression in FUSE_NOTIFY_INVAL_ENTRY,
which failed to invalidate negative dentries.
This manifests in the filesystem returning -ENOENT for operations on an
existing file.
Fixing it properly while still keeping the start_removing* infrastructure
would add much additional complexity.
Instead revert to the original simple implementation.
The start_removing* infrastructure is needed in VFS to abstract the
filesystem locking. However filesystem code can still safely use the raw
locking primitives without affacting other filesystems.
This is part two of the revert.
Reported-by: Артем Лабазов <123321artyom@gmail.com>
Closes: https://lore.kernel.org/all/CAFbF8N7++zopZuEcsKRxBV_sgOGCbzCY0hOyMw1SiGAtuzGhyQ@mail.gmail.com/
Fixes: c9ba789dad ("VFS: introduce start_creating_noperm() and start_removing_noperm()")
Cc: stable@vger.kernel.org # 6.19
Cc: NeilBrown <neilb@ownmail.net>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
This reverts commit cab0123751.
Commit c9ba789dad ("VFS: introduce start_creating_noperm() and
start_removing_noperm()") caused a regression in FUSE_NOTIFY_INVAL_ENTRY,
which failed to invalidate negative dentries.
This manifests in the filesystem returning -ENOENT for operations on an
existing file.
Fixing it properly while still keeping the start_removing* infrastructure
would add much additional complexity.
Instead revert to the original simple implementation.
The start_removing* infrastructure is needed in VFS to abstract the
filesystem locking. However filesystem code can still safely use the raw
locking primitives without affacting other filesystems.
This is part one of the revert.
Reported-by: Артем Лабазов <123321artyom@gmail.com>
Closes: https://lore.kernel.org/all/CAFbF8N7++zopZuEcsKRxBV_sgOGCbzCY0hOyMw1SiGAtuzGhyQ@mail.gmail.com/
Fixes: c9ba789dad ("VFS: introduce start_creating_noperm() and start_removing_noperm()")
Cc: stable@vger.kernel.org # 6.19
Cc: NeilBrown <neilb@ownmail.net>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
FUSE_NOTIFY_PRUNE validates the nodeid payload length with:
size - sizeof(outarg) != outarg.count * sizeof(u64)
On 32-bit kernels, size_t is also 32 bits, so the daemon-controlled
count multiplication can wrap. A prune notification with count
0x20000000 and no nodeid payload passes the check, enters the copy
loop, and asks the device copy path to read nodeids that are not
present in the userspace write buffer. In QEMU this reaches the
fuse_copy_fill() BUG_ON(!err) path.
Validate the payload length with array_size() instead. That accepts
exactly the same valid messages, but avoids wrapping arithmetic before
the copy loop consumes the count.
Assisted-by: Codex:gpt-5.5-cyber-preview
Fixes: 3f29d59e92 ("fuse: add prune notification")
Cc: stable@vger.kernel.org
Signed-off-by: Samuel Moelius <sam.moelius@trailofbits.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
If a copy into the userspace ring buffer fails, a request will be
terminated and fuse_uring_req_end() will set ent->fuse_req to NULL but
it will leave the entry on ent_w_req_queue in FRRS_FUSE_REQ state. This
can lead to a NULL deref if the request expiration logic scans
ent_w_req_queue in the window before the entry is moved off it.
Fix this by taking the entry off ent_w_req_queue and changing its state
from FRRS_FUSE_REQ to FRRS_INVALID before terminating the request.
Fixes: 4fea593e62 ("fuse: optimize over-io-uring request expiration check")
Cc: stable@kernel.org
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
When fuse_resend() moves a request from fpq->processing back to
fiq->pending, it sets FR_PENDING and clears FR_SENT but does not
remove the requests intr_entry from fiq->interrupts. If the
request had FR_INTERRUPTED set from a prior signal, intr_entry
remains dangling on fiq->interrupts. When the requesting task
then receives a fatal signal, fuse_remove_pending_req() sees
FR_PENDING=1, removes the request from fiq->pending and frees it
via the refcount path, also without cleaning intr_entry. The
stale intr_entry causes use-after-free when fuse_read_interrupt()
iterates fiq->interrupts:
- list_del_init(&req->intr_entry) -> UAF write on freed slab
- req->in.h.unique -> UAF read, data leaked to userspace
Remove intr_entry from fiq->interrupts in fuse_resend() for
interrupted requests before they are placed back on fiq->pending.
Add a WARN_ON if the intr_entry is not empty on request destruction.
Fixes: 760eac73f9 ("fuse: Introduce a new notification type for resend pending requests")
Cc: stable@vger.kernel.org # 6.9
Signed-off-by: Ji'an Zhou <eilaimemedsnaimel@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Bad userspace might try to trick us and send commit SQEs request
unique / commit-id of requests that are not even send to
fuse-server (io_uring_cmd_done() not called) yet.
fuse_uring_commit_fetch() ends the fuse request when the ring entry
has a wrong state, but that could have caused a use-after-free
with the memcpy operations in fuse_uring_send_in_task().
In order to avoid such races the call of fuse_uring_add_to_pq()
is moved after the copy operations and just before completing
the io-uring request - malicious userspace cannot find the request
anymore until all prepration work in fuse-client/kernel is completed.
This also moves fuse_uring_add_to_pq() a bit up in the code to
avoid a forward declaration. Also not with a preparation commit,
to make it easier to back port to older kernels.
Reported-by: xlabai <xlabai@tencent.com>
Reported-by: Berkant Koc <me@berkoc.com>
Fixes: c090c8abae ("fuse: Add io-uring sqe commit and fetch support")
Cc: stable@kernel.org # 6.14
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
There are several readers of queue->stopped that check the value
under lock, but fuse_uring_commit_fetch() did not and actually
the value was not set under the lock in fuse_uring_abort_end_requests()
either. Especially in fuse_uring_commit_fetch it is important
to check under a lock, because due to races 'struct fuse_req'
might be freed with fuse_request_end, but another thread/cpu
might already do teardown work.
Cc: stable@kernel.org # 6.14
Fixes: 4a9bfb9b68 ("fuse: {io-uring} Handle teardown of ring entries")
Reported-by: Berkant Koc <me@berkoc.com>
Reported-by: xlabai <xlabai@tencent.com>
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
fuse_uring_async_stop_queues() might run when the last reference
on ring->queue_refs was already dropped.
In order to avoid an early destruction a reference on struct fuse_conn
is now taken before starting fuse_uring_async_stop_queues() and that
reference is only released when that delayed work queue terminates.
Fixes: 4a9bfb9b68 ("fuse: {io-uring} Handle teardown of ring entries")
Cc: stable@kernel.org # 6.14
Reported-by: Berkant Koc <me@berkoc.com>
Signed-off-by: Bernd Schubert <bernd@bsbernd.com>
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
When io_uring delivers task work with tw.cancel set (PF_EXITING,
PF_KTHREAD fallback, or percpu_ref_is_dying on the ring context),
fuse_uring_send_in_task() takes the cancel branch, assigns
-ECANCELED, and falls through to fuse_uring_send(). That path only
flips the entry to FRRS_USERSPACE and completes the io_uring cmd;
it never discharges the ring entry's owning reference to the
fuse_req that fuse_uring_add_req_to_ring_ent() handed it at
dispatch time.
fuse_uring_send_in_task()
tw.cancel == true
err = -ECANCELED
fuse_uring_send(ent, cmd, err, issue_flags)
ent->state = FRRS_USERSPACE
list_move(&ent->list, &queue->ent_in_userspace)
ent->cmd = NULL
io_uring_cmd_done(-ECANCELED)
/* ent->fuse_req still set, req still hashed */
The fuse_req stays linked on fpq->processing[hash] and
fuse_request_end() is never invoked. The originating syscall
thread blocks in D-state in request_wait_answer() until
fuse_abort_conn() runs, which can be the entire connection
lifetime. For FR_BACKGROUND requests fc->num_background is never
decremented either, so repeated cancels inflate the counter until
max_background is hit and all later background ops stall. tw.cancel does
not imply a connection abort (e.g. a single io_uring worker thread exits
while the fuse connection stays up), so this cannot be left for
fuse_abort_conn() to clean up.
Ending the req but still routing the entry through fuse_uring_send()
is not enough: that leaves a req-less entry on ent_in_userspace, and
ent_list_request_expired() dereferences ent->fuse_req unconditionally
on the head of that list, which would then NULL-deref.
Fix the cancel branch to release the entry directly. Remove it from the
queue, complete the io_uring cmd, end the fuse_req, free the entry, and
drop its queue_refs (waking the teardown waiter if it was the last).
Fixes: c2c9af9a0b ("fuse: Allow to queue fg requests through io-uring")
Cc: stable@vger.kernel.org
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Assisted-by: kres (claude-opus-4-7)
Signed-off-by: Chris Mason <clm@meta.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
fuse_uring_cancel() moves entries that are available (these have no reqs
attached) to the ent_in_userspace list. ent_list_request_expired()
checks the first entry on ent_in_userspace and dereferences
ent->fuse_req unconditionally, which will crash on a cancelled entry
that was moved to this list.
Fix this by freeing the entry and dropping queue_refs directly in
fuse_uring_cancel(). This is safe because cancel is the cancel handler
itself - after io_uring_cmd_done(), no more cancels will be dispatched
for this command, and teardown serializes with cancel via queue->lock.
Since cancel now decrements queue_refs, fuse_uring_abort() must no
longer gate fuse_uring_abort_end_requests() on queue_refs > 0, as
cancelled entries may have already dropped queue_refs while requests are
still queued. Remove the gate so abort always flushes requests and stops
queues.
Reported-by: Heechan Kang <gganji11@naver.com>
Tested-by: Heechan Kang <gganji11@naver.com>
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Fixes: 4fea593e62 ("fuse: optimize over-io-uring request expiration check")
Cc: stable@vger.kernel.org
Suggested-by: Jian Huang Li <ali@ddn.com>
Suggested-by: Horst Birthelmer <horst@birthelmer.de>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Check fch->connected under fch->lock in fuse_uring_create() before
attaching a new ring. Without this, a race between fuse_uring_create()
and fuse_chan_abort() can result in the ring, queue, and fpq.processing
table being created after fuse_uring_abort() has already run, leading
to unnecessary allocation and teardown. These are eventually cleaned up
by fuse_uring_destruct() but will linger until the process exits, even
with the connection aborted.
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
This fixes this race:
- thread a: io_uring_enter -> register sqe ->
fuse_uring_create_ring_ent -> allocate ent but doesn't grab queue_ref
yet
- thread b: fuse_conn_destroy() -> fuse_chan_abort() ->
fuse_uring_abort() is a no-op due to queue ref being 0
- thread a: grabs the queue_ref, queue_ref is now 1, rest of
fuse_uring_do_register() logic executes
- thread b: fuse_chan_abort() returns, fuse_chan_wait_aborted() now runs
and calls
"wait_event(ring->stop_waitq, atomic_read(&ring->queue_refs) == 0);"
The abort/unmount thread will hang indefinitely in unkillable state as
nothing will decrement queue_refs or wake stop_waitq, and the ring,
queue, and ent are leaked.
Fix this by checking fch->connected under fch->lock after the created
ent has grabbed a ref count on the queue. This ensures that in the
scenario above, it is guaranteed that we either release the queue ref
and wake up stop_waitq (in case fuse_chan_wait_aborted() is already
waiting) in fuse_uring_do_register() when we detect !fch->connected, or
if the connection is aborted after the check, it is guaranteed that the
async teardown worker will be running in the background cleaning up ents
and decrementing the ent's ref on the queue, which will unblock the
eventual queue and ring teardown.
Fixes: 24fe962c86 ("fuse: {io-uring} Handle SQEs - register commands")
Cc: stable@vger.kernel.org
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Joanne Koong <joannelkoong@gmail.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
On weakly-ordered architectures, the store to fiq->ops can be
reordered past the store to ring->ready, allowing a CPU that sees
ring->ready == true via fuse_uring_ready() to dispatch requests
through a stale fiq->ops pointer. Upgrade the store to
smp_store_release() and the load in fuse_uring_ready() to
smp_load_acquire() so that the preceding WRITE_ONCE(fiq->ops, ...)
is visible to any CPU that observes ring->ready == true.
Additionally, fuse_uring_do_register() publishes ring->ready with
WRITE_ONCE() but the fast-path check reads it with a plain load.
This is a marked-vs-unmarked access that KCSAN will flag. Wrap it in
READ_ONCE() to mark it without adding unnecessary ordering.
Also wrap the fc->ring load in fuse_uring_ready() in READ_ONCE() to
prevent the compiler from reloading it between the NULL check and the
dereference.
Fixes: c2c9af9a0b ("fuse: Allow to queue fg requests through io-uring")
Cc: stable@vger.kernel.org
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Assisted-by: kres (claude-opus-4-7)
Signed-off-by: Chris Mason <clm@meta.com>
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
copy_from_user() returns the number of bytes not copied as an unsigned
residual on failure (1..sizeof(struct fuse_out_header)). fuse_uring_commit
stores that residual in ssize_t err, sets req->out.h.error to -EFAULT,
then jumps to out: with err still holding the positive residual.
err = copy_from_user(&req->out.h, &ent->headers->in_out,
sizeof(req->out.h));
if (err) {
req->out.h.error = -EFAULT;
goto out; /* err is the positive residual */
}
...
out:
fuse_uring_req_end(ent, req, err);
fuse_uring_req_end() then runs
if (error)
req->out.h.error = error;
which overwrites the just-assigned -EFAULT with the positive residual.
FUSE callers such as fuse_simple_request() test err < 0 to detect
failure, so the positive value is interpreted as success and the
caller proceeds with an uninitialised or partial req->out.args.
Fix by assigning err = -EFAULT in the failure branch before jumping
to out, so fuse_uring_req_end() receives a negative errno and sets
req->out.h.error to -EFAULT.
Fixes: c090c8abae ("fuse: Add io-uring sqe commit and fetch support")
Cc: stable@vger.kernel.org
Reviewed-by: Joanne Koong <joannelkoong@gmail.com>
Assisted-by: kres (claude-opus-4-7)
Signed-off-by: Chris Mason <clm@meta.com>
Reviewed-by: Bernd Schubert <bernd@bsbernd.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
Commit dabb903910 ("fuse: increase readdir buffer size") changed
fuse_readdir_uncached() to size its temporary buffer from ctx->count.
This is useful for overlayfs and other in-kernel callers that use
INT_MAX to indicate an unlimited directory read.
The larger buffer is currently supplied as a kvec output argument. For
virtiofs, kvec arguments are copied through req->argbuf, which is
allocated with kmalloc(..., GFP_ATOMIC). A large uncached readdir buffer
can therefore require a multi-megabyte contiguous atomic allocation
before the request is queued.
Avoid the large bounce-buffer allocation by backing uncached readdir
output with pages and setting out_pages. Transports such as virtiofs can
then pass the pages as scatter-gather entries instead of copying the
output through argbuf.
Map the pages with vm_map_ram() only while parsing the returned dirents.
The existing parser can then continue to use a linear kernel mapping.
[SzM: separate allocation of pages into a helper function]
Fixes: dabb903910 ("fuse: increase readdir buffer size")
Cc: stable@vger.kernel.org
Signed-off-by: Matthew R. Ochs <mochs@nvidia.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
iput() called from fuse_release_end() can Oops if the super block has
already been destroyed. Normally this is prevented by waiting for
num_waiting to go down to zero before commencing with super block shutdown.
This only works, however, for the last submount instance, as the wait
counter is per connection, not per superblock.
Revert to using synchronous release requests for the auto_submounts case,
which is virtiofs only at this time.
Reported-by: Aurélien Bombo <abombo@microsoft.com>
Reported-by: Zhihao Cheng <chengzhihao1@huawei.com>
Cc: Greg Kurz <gkurz@redhat.com>
Closes: https://github.com/kata-containers/kata-containers/issues/12589
Fixes: 26e5c67deb ("fuse: fix livelock in synchronous file put from fuseblk workers")
Cc: stable@vger.kernel.org
Reviewed-by: Greg Kurz <gkurz@redhat.com>
Signed-off-by: Miklos Szeredi <mszeredi@redhat.com>
If perf.data files are taken from one machine to another they may
leak virtual addresses and so weaken ASLR on the machine they are
coming from. Add an aslr option for perf inject that remaps all
virtual addresses, or drops data/events, so that the virtual address
information isn't leaked.
This patch introduces the core ASLR remapping tool infrastructure and
implements remapping/tracking for metadata events (MMAP, MMAP2, COMM,
FORK, EXIT, KSYMBOL, TEXT_POKE). Sample events are delegated without
remapping for now.
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
Co-developed-by: Gabriel Marin <gmx@google.com>
Signed-off-by: Gabriel Marin <gmx@google.com>
Tested-by: James Clark <james.clark@linaro.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
During kernel ELF symbol parsing (dso__process_kernel_symbol), proc
kallsyms image loading (dso__load_kernel_sym,
dso__load_guest_kernel_sym), and dynamic kernel memory map alignment
updates (machine__update_kernel_mmap), the loader directly modifies live
virtual address boundary keys fields on map objects.
If these boundaries are mutated while the map pointer actively resides
inside the parent maps cache array list (kmaps) outside of any lock
closure, an unsafe concurrent window is exposed where parallel worker
lookup threads (e.g., inside perf top) can mistakenly assume the cache
remains sorted based on stale parameters, executing binary search
queries (bsearch) across an unsorted range and triggering lookup
failures.
Fix this by introducing maps__mutate_mapping() that explicitly acquires
the parent maps write semaphore lock, executes an incoming mutation
callback block to perform the field updates under lock protection, and
invalidates the sorted tracking flags prior to releasing the write lock.
This guarantees synchronization invariants, closing the concurrent
lookup race window. The adjacent module alignment pass inside
machine__create_kernel_maps() is safely preserved as a high-performance
lockless pass, as its invocation lifecycle bounds remain strictly
single-threaded by contract during session initialization construction.
To safely support this unconditional down_write write lock mutator
without recursive read-to-write self-deadlock upgrades during lazy
symbol loading, we introduce a public maps__load_maps() API.
It copies map pointers under a brief read lock and force-loads all
modules locklessly outside the lock. Callers (such as perf inject) must
pre-load all kernel symbol maps up front at startup using
maps__load_maps(), completely bypassing dynamic runtime mutations.
Fixes: 39b12f7812 ("perf tools: Make it possible to read object code from vmlinux")
Assisted-by: Antigravity:gemini-3.1-pro
Signed-off-by: Ian Rogers <irogers@google.com>
Tested-by: James Clark <james.clark@linaro.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Gabriel Marin <gmx@google.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Memory controller drivers for v7.2, part two
A few improvements for Tegra Memory Controller drivers, including one
fix for UBSAN report for an older commit.
* tag 'memory-controller-drv-7.2-2' of https://git.kernel.org/pub/scm/linux/kernel/git/krzk/linux-mem-ctrl:
memory: tegra234: drop dead NULL check in tegra234_mc_icc_aggregate()
memory: tegra264: drop redundant tegra264_mc_icc_aggregate()
memory: tegra186-emc: stop borrowing MC aggregate hook for EMC
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
With alloc_flags usage in slab, we can replace __GFP_NO_OBJ_EXT with an
alloc flag that prevents kmalloc recursion. For that we need a version
of kmalloc() that takes alloc_flags and use it in places that perform
these potentially recursive kmalloc allocations (of sheaves or obj_ext
arrays).
Add this function, named kmalloc_flags(). Right now it's only useful for
these nested allocations, so it doesn't need to optimize build-time
constant sizes like kmalloc() or kmalloc_buckets.
Since we need it to support both normal and non-spinning
kmalloc_nolock() context through the SLAB_ALLOC_NOLOCK flag, split out
most of the special _kmalloc_nolock_noprof() implementation to
__kmalloc_nolock_noprof() that takes a slab_alloc_context, and make
_kmalloc_nolock_noprof() a simple tail calling wrapper with the proper
context.
kmalloc_flags() can thus determine whether to call
__kmalloc_nolock_noprof() or __do_kmalloc_node(), based on the
given alloc_flags.
Link: https://patch.msgid.link/20260610-slab_alloc_flags-v2-14-7190909db118@kernel.org
Reviewed-by: Hao Li <hao.li@linux.dev>
Reviewed-by: Suren Baghdasaryan <surenb@google.com>
Reviewed-by: Harry Yoo (Oracle) <harry@kernel.org>
Signed-off-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Miklos Szeredi
Joanne Koong
Alberto Ruiz
Samuel Moelius
Ji'an Zhou
Bernd Schubert
Chris Mason
Matthew R. Ochs
Ian Rogers
Arnd Bergmann
Vlastimil Babka (SUSE)