pmu_id() calls filename__read_str() then strips the trailing newline
via str[len - 1] = 0. If the PMU identifier file is empty,
filename__read_str() succeeds with len = 0. len - 1 underflows
size_t to SIZE_MAX, writing a null byte before the heap allocation.
Add a len == 0 check before the newline stripping.
Fixes: 51d5484715 ("perf pmu: Add pmu_id()")
Reported-by: sashiko-bot <sashiko-bot@kernel.org>
Cc: John Garry <john.g.garry@oracle.com>
Assisted-by: Claude:claude-opus-4.6
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
I found this error when I pass EXTRA_CFLAGS=-fsanitize=address on Fedora
44 with GCC 16. Fix it by copying one less byte.
CC util/jitdump.o
util/jitdump.c: In function ‘jit_process’:
util/jitdump.c:237:9: error: ‘__builtin_strncpy’ specified bound 4096 equals destination size
[-Werror=stringop-truncation]
237 | strncpy(jd->dir, name, PATH_MAX);
| ^
cc1: all warnings being treated as errors
make[4]: *** [tools/build/Makefile.build:95: util/jitdump.o] Error 1
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [tools/build/Makefile.build:158: util] Error 2
make[2]: *** [Makefile.perf:578: perf-util-in.o] Error 2
make[1]: *** [Makefile.perf:288: sub-make] Error 2
make: *** [Makefile:76: all] Error 2
Signed-off-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: James Clark <james.clark@linaro.org>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
We already test branch output in perf script mode, but then retest it in
Perf report mode. This is more of a test of Perf itself than Coresight
because Perf uses the same samples to generate both outputs. Also we're
already testing instruction output in Perf report mode.
Remove this test for a speedup. On the systemwide test also remove the
Perf report test because systemwide mode records a lot more data so
running multiple tests on it has a big runtime impact.
Reviewed-by: Leo Yan <leo.yan@arm.com>
Signed-off-by: James Clark <james.clark@linaro.org>
Cc: Amir Ayupov <aaupov@meta.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mike Leach <mike.leach@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paschalis Mpeis <Paschalis.Mpeis@arm.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Like asm_pure_loop, this memcpy test only checks that 10 of each of a
few trace packet types occur after recording a lot of trace, which isn't
more specific than other existing Coresight tests.
Assume it was supposed to be a stress test for dumping and replace it
with one that doesn't require a custom binary and checks for a specific
amount of raw output. Don't bother checking for packets because the
other tests that test decoding will catch issues with malformed data.
This also adds coverage for exit snapshot mode which was missing.
Signed-off-by: James Clark <james.clark@linaro.org>
Tested-by: Leo Yan <leo.yan@arm.com>
Cc: Amir Ayupov <aaupov@meta.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mike Leach <mike.leach@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paschalis Mpeis <Paschalis.Mpeis@arm.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
It's not obvious what this test is for so remove it. It's not a stress
test because it doesn't output lots of data and it's not a functional
test because it only looks for raw trace output. It seems to imply that
a program written in assembly influences whether trace would be
generated by the CPU or not, but the CPU doesn't know what language the
program is written in.
We already have lots of Coresight tests that test the full pipeline
including decoding, and in many more modes of operation than this one,
so if no trace was collected they will already fail leaving this one
redundant.
Reviewed-by: Leo Yan <leo.yan@arm.com>
Signed-off-by: James Clark <james.clark@linaro.org>
Cc: Amir Ayupov <aaupov@meta.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mike Leach <mike.leach@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paschalis Mpeis <Paschalis.Mpeis@arm.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Testing a long sequence without branches seems like it would be better
as a decoder unit test, and this test doesn't test decoding either, so
it's not clear what bugs this is trying to catch.
The new deterministic workload has somewhat long sequences when built
unoptimized, and we can always increase them later if we want to. But
now we test that decoding always gives the same result for the same
sequence of code which we've never had before.
Signed-off-by: James Clark <james.clark@linaro.org>
Tested-by: Leo Yan <leo.yan@arm.com>
Cc: Amir Ayupov <aaupov@meta.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Mike Leach <mike.leach@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paschalis Mpeis <Paschalis.Mpeis@arm.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Add a --workload-ctl=fifo:ctl-fifo[,ack-fifo] option for 'perf test
-w'. When set, run_workload() opens the named FIFO, writes enable before
invoking the builtin workload, writes disable before returning, and
waits for ack responses when an ack FIFO is provided to ensure that the
workload doesn't run until the events are enabled.
This can be used to limit the scope of the recording to only the
workload execution and avoid recording Perf setup and teardown code if
Perf record is started with events disabled (-D 1).
Assisted-by: Codex:GPT-5.5
Signed-off-by: James Clark <james.clark@linaro.org>
Cc: Amir Ayupov <aaupov@meta.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Leo Yan <leo.yan@arm.com>
Cc: Mike Leach <mike.leach@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paschalis Mpeis <Paschalis.Mpeis@arm.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
PE_CONTEXT elements update the context ID and exception level, but the
decoder may still have prior packets cached for frontend processing.
Updating the context immediately in the decoder backend can make those
cached packets get consumed with the wrong thread or EL state.
Add a CS_ETM_CONTEXT packet carrying the TID and EL to the frontend,
this keeps context changes ordered with the rest of the packet stream
and avoids mismatches when synthesizing samples from cached packets.
Separate the memory access function into one for the frontend and one
for decoding. The frontend also needs memory access to attach the
instruction to samples. Because the frontend does memory access for
both previous and current packets, change all the frontend memory access
function signatures to take both a tidq and packet. But backend always
uses the current backend EL and thread from the tidq.
Treat context packets as a boundary for branch sample generation and
remove tidq->prev_packet_thread because it's not possible to branch to a
different thread, so only tracking the current thread is required for
sample generation.
Fixes: e573e978fb ("perf cs-etm: Inject capabilitity for CoreSight traces")
Reported-by: Amir Ayupov <aaupov@meta.com>
Closes: https://lore.kernel.org/linux-perf-users/20260515021135.1729028-1-aaupov@meta.com/
Co-authored-by: James Clark <james.clark@linaro.org>
Signed-off-by: Leo Yan <leo.yan@arm.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: linux-doc@vger.kernel.org
Cc: Mike Leach <mike.leach@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Paschalis Mpeis <Paschalis.Mpeis@arm.com>
Cc: Robert Walker <robert.walker@arm.com>
Cc: Shuah Khan <skhan@linuxfoundation.org>
Cc: Suzuki Poulouse <suzuki.poulose@arm.com>
Signed-off-by: James Clark <james.clark@linaro.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Replace snprintf with scnprintf in buffer offset calculations to
ensure the 'used' count will not exceed the "len".
The current logic in perf_pmu__for_each_event uses an unconditional
+ 1 increment to buf_used to account for null terminators. This can
cause a stack buffer overflow in the subsequent scnprintf call.
When the local stack buffer buf (1024 bytes) is full, buf_used can
reach 1025. This causes the subsequent remaining space calculation
sizeof(buf) - buf_used to underflow.
Use sub_non_neg() to see if space actually existed, and only
increment the offset if remaining space is present.
Changes includes:
- Use sub_non_neg to check if space exists
- Replacing snprintf with scnprintf to ensure the return value
reflects the actual bytes written into the buffer.
- Only increment buf_used by 1 if space exists
- If a parameterized event uses a built-in perf keyword for its
parameter name (eg, config=?), the lexer parses it as a predefined
term token, which sets term->config to NULL. Add check to use
parse_events__term_type_str() if term->config is NULL.
Signed-off-by: Athira Rajeev <atrajeev@linux.ibm.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Hari Bathini <hbathini@linux.vnet.ibm.com>
Cc: Ian Rogers <irogers@google.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Shivani Nittor <shivani@linux.ibm.com>
Cc: Tanushree.Shah@ibm.com
Cc: Tejas.Manhas1@ibm.com
Cc: Thomas Richter <tmricht@linux.ibm.com>
Cc: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
GPIOs 38, 39, 48 and 49 support the SDC2 DATA function, while
GPIO 51 supports SDC2 CMD and GPIO 62 supports SDC2 CLK.
However, the sdc2 pin function is not listed in the corresponding
pingroup definitions, preventing these pins from being muxed for
SDC2 operation.
Add the missing sdc2 function mappings.
Fixes: 6f26989e15 ("pinctrl: qcom: Add Eliza pinctrl driver")
Signed-off-by: Abel Vesa <abel.vesa@oss.qualcomm.com>
Reviewed-by: Konrad Dybcio <konrad.dybcio@oss.qualcomm.com>
Signed-off-by: Linus Walleij <linusw@kernel.org>
While being less compact, using named initializers allows to more easily
see which members of the structs are assigned which value without having
to lookup the declaration of the struct. And it's also more robust
against changes to the struct definition.
The mentioned robustness is relevant for a planned change to struct
i2c_device_id that replaces .driver_data by an anonymous union.
While touching all these arrays, unify usage of whitespace in the list
terminator.
This patch doesn't modify the compiled arrays, only their representation
in source form benefits. The former was confirmed with x86 and arm64
builds.
Signed-off-by: Uwe Kleine-König (The Capable Hub) <u.kleine-koenig@baylibre.com>
Signed-off-by: Linus Walleij <linusw@kernel.org>
i.MX arm64 device tree changes for v7.2 (part2)
- Revert the 32-bit non-prefetchable PCIe window from 3 GiB back to 1 GiB
to prevent overlap between inbound DMA address space and low system RAM.
Such overlap can cause DMA transactions to be routed to a BAR on the same
host bridge instead of system memory.
* tag 'imx-dt64-7.2-part2' of git://git.kernel.org/pub/scm/linux/kernel/git/frank.li/linux:
arm64: dts: lx2160a-rev2: avoid 32-bit pcie window system ram overlap
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
lpi_gpio_write() became unused after the PM clock runtime conversion
switched write paths to register helper calls inside callers that already
hold an active runtime-PM reference.
With -Werror this triggers:
error: 'lpi_gpio_write' defined but not used [-Wunused-function]
Drop the dead wrapper and rename the low-level MMIO helpers from
__lpi_gpio_* to lpi_gpio_*_reg for neutral register-accessor naming.
Fixes: b719ede389 ("pinctrl: qcom: lpass-lpi: Switch to PM clock framework for runtime PM")
Reported-by: Nathan Chancellor <nathan@kernel.org>
Closes: https://lore.kernel.org/all/f03850f6-186d-4988-a450-e6e95f24a551@kernel.org/
Signed-off-by: Ajay Kumar Nandam <ajay.nandam@oss.qualcomm.com>
Signed-off-by: Linus Walleij <linusw@kernel.org>
qat_vf_resume_write() checks filp->f_pos before taking migf->lock, but
copies into the migration-state buffer after taking the lock and
re-reading the shared file position.
Two concurrent writers could therefore pass the bounds check with the
old offset, then have the second writer copy after the first advanced
f_pos, writing past the end of the migration-state buffer.
Take migf->lock before doing the boundary checks.
Fixes: bb208810b1 ("vfio/qat: Add vfio_pci driver for Intel QAT SR-IOV VF devices")
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Signed-off-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Link: https://lore.kernel.org/r/20260608151317.136613-1-giovanni.cabiddu@intel.com
Signed-off-by: Alex Williamson <alex@shazbot.org>
The format_alias() function in util/pmu.c has a check to
detect whether the event has parameterized field ( =? ).
The string alias->terms contains the event and if the event
has user configurable parameter, there will be presence of
sub string "=?" in the alias->terms.
Snippet of code:
/* Paramemterized events have the parameters shown. */
if (strstr(alias->terms, "=?")) {
/* No parameters. */
snprintf(buf, len, "%.*s/%s/", (int)pmu_name_len, pmu->name, alias->name);
if "strstr" contains the substring, it returns a pointer
and hence enters the above check which is not the expected
check. And hence "perf list" doesn't have the parameterized
fields in the result.
Fix this check to use:
if (!strstr(alias->terms, "=?")) {
With this change, perf list shows the events correctly with
the strings showing parameters.
Before the fix:
# ./perf list|grep -w PM_PAU_CYC
hv_24x7/PM_PAU_CYC/ [Kernel PMU event]
With this fix:
# ./perf list|grep -w PM_PAU_CYC
hv_24x7/PM_PAU_CYC,chip=?/ [Kernel PMU event]
Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Athira Rajeev <atrajeev@linux.ibm.com>
Tested-by: Venkat Rao Bagalkote <venkat88@linux.ibm.com>
Acked-by: Namhyung Kim <namhyung@kernel.org>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Hari Bathini <hbathini@linux.vnet.ibm.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Madhavan Srinivasan <maddy@linux.ibm.com>
Cc: Michael Petlan <mpetlan@redhat.com>
Cc: Shivani Nittor <shivani@linux.ibm.com>
Cc: Tanushree.Shah@ibm.com
Cc: Tejas.Manhas1@ibm.com
Cc: Thomas Richter <tmricht@linux.ibm.com>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
On the Surface Pro 10 (Meteor Lake) the touchscreen stops working after a
suspend/resume cycle and only recovers after a reboot. The driver logs
"GET_DEVICE_INFO: recv failed: -11" on resume.
This platform suspends through s2idle: /sys/power/mem_sleep exposes
"[s2idle]" as the only state, there is no "deep"/S3 entry at all. The
touch IC nonetheless loses power across that s2idle suspend, the same
way it does across hibernation. quickspi_resume() only re-selects the
THC port, restores interrupts and DMA and sends a HIDSPI_ON command,
assuming the touch IC kept its power and state. When it has actually
lost power the HIDSPI_ON command is never acknowledged and the
descriptor read fails, leaving the touchscreen dead until the module is
reloaded.
quickspi_restore() already handles this for hibernation by
reconfiguring the THC SPI/LTR settings and running reset_tic() to
re-enumerate the device. Make quickspi_resume() do the same when the
device is not a wake source. A wake-enabled device keeps its power and
state across suspend, so it stays on the light restore path: resetting
it would discard a pending wake touch event and break wake-on-touch.
The non-wake path mirrors the existing quickspi_restore() sequence,
including enabling interrupts before reset_tic(), so it introduces no
new ordering relative to code already in the driver.
This change has been validated on a Surface Pro 10 running the
linux-surface kernel across multiple s2idle suspend/resume cycles; it
has not been tested on a mainline build.
Closes: https://github.com/linux-surface/linux-surface/issues/1799
Signed-off-by: Danny D. <d3z.the.dev@gmail.com>
Reviewed-by: Even Xu <even.xu@intel.com>
Signed-off-by: Jiri Kosina <jkosina@suse.com>
On s390 the Python extension build fails as follows when using a linker
that is configured to treat text relocations (TEXTREL) in shared
libraries as error by default:
GEN python/perf.cpython-314-s390x-linux-gnu.so
/usr/bin/ld.bfd: error: read-only segment has dynamic relocations
This occurrs because util/llvm-c-helpers.o is erroneously built from
util/llvm-c-helpers.cpp without compiler option -fPIC but linked into
the shared library (via libperf-util.a(perf-util-in.o)).
On s390, object files must be compiled as position-indepedent code (PIC)
in order to be linked into shared libraries. Commit a9a3f1d18a ("perf
s390: Always build with -fPIC") added compiler option -fPIC to CFLAGS
for s390, which is used in C compiles. Add -fPIC to CXXFLAGS for s390
as well, so that it is also used in C++ compiles.
Fixes: a9a3f1d18a ("perf s390: Always build with -fPIC")
Reported-by: Thomas Richter <tmricht@linux.ibm.com>
Reviewed-by: Ian Rogers <irogers@google.com>
Reviewed-by: James Clark <james.clark@linaro.org>
Signed-off-by: Jens Remus <jremus@linux.ibm.com>
Cc: Adrian Hunter <adrian.hunter@intel.com>
Cc: Alexander Shishkin <alexander.shishkin@linux.intel.com>
Cc: Bill Wendling <morbo@google.com>
Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Hendrik Brueckner <brueckner@linux.ibm.com>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Jan Polensky <japo@linux.ibm.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Justin Stitt <justinstitt@google.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Cc: Nathan Chancellor <nathan@kernel.org>
Cc: Nick Desaulniers <nick.desaulniers+lkml@gmail.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Currently, host programs are linked with HOSTCC using the toolchain's
default linker. This can result in confusing behavior when using the
LLVM Kbuild variable (which states that the user would like to build
with the LLVM toolchain instead of the GNU one), as clang's default
linker is ld for most platforms, not ld.lld. The documentation mentions
HOSTLD=ld.lld is set but this variable is not used by Kbuild proper,
only within some tools/ projects. Kbuild provides the HOSTLDFLAGS
variable, which allow users to provide the '-fuse-ld' or '--ld-path'
flags to customize what linker is used, but this is not super obvious to
folks not familiar with Kbuild.
If the user has not customized the linker already using one of these
flags, default to ld.lld when using the LLVM variable, which is more in
line with user expectations when using that variable.
Closes: https://github.com/ClangBuiltLinux/linux/issues/2167
Link: https://patch.msgid.link/20260610-kbuild-use-lld-for-linking-hostprogs-v1-1-70396fe42ee3@kernel.org
Signed-off-by: Nathan Chancellor <nathan@kernel.org>
On risc-v, pref probe generates an invalid syntax for a named register in
a kprobe.
$ perf probe --debug verbose --add "n_tty_write tty"
...
Writing event: p:probe/n_tty_write _text+8922528 tty=%"%a0":x64
Failed to write event: Invalid argument
The problem is the combination of
#define REG_DWARFNUM_NAME(reg, idx) [idx] = "%" #reg
and entries such as
REG_DWARFNUM_NAME("%a0", 10)
where #reg will escape the quotes of the first macro parameter.
Update the macro definition to produce the correct syntax for a named
register in a kprobe, i.e. the unquoted register name with only one
leading %.
Fixes: a90c451918 ("perf riscv: Remove dwarf-regs.c and add dwarf-regs-table.h")
Reviewed-by: Ian Rogers <irogers@google.com>
Signed-off-by: Martin Kaiser <martin@kaiser.cx>
Cc: Ian Rogers <irogers@google.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
dwarf-regs.c includes an arch-specific dwarf-regs-table.h for several
architectures. This pulls in different definitions of REG_DWARFNUM_NAME
and causes compiler warnings for W=1 builds.
In file included from util/dwarf-regs.c:23:
.../dwarf-regs-table.h:5: error: "REG_DWARFNUM_NAME" redefined [-Werror]
#define REG_DWARFNUM_NAME(reg, idx) [idx] = reg
Undefine REG_DWARFNUM_NAME before each new definition.
Suggested-by: Arnaldo Carvalho de Melo <acme@redhat.com>
Signed-off-by: Martin Kaiser <martin@kaiser.cx>
Cc: Ian Rogers <irogers@google.com>
Cc: Namhyung Kim <namhyung@kernel.org>
Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
This is the hwmon driver for Microchip EMC1812/13/14/15/33
Multichannel Low-Voltage Remote Diode Sensor Family.
EMC1812 has one external remote temperature monitoring channel.
EMC1813 has two external remote temperature monitoring channels.
EMC1814 has three external remote temperature monitoring channels,
channels 2 and 3 support anti parallel diode.
EMC1815 has four external remote temperature monitoring channels and
channels 1/2 and 3/4 support anti parallel diode.
EMC1833 has two external remote temperature monitoring channels and
channels 1 and 2 support anti parallel diode.
Resistance Error Correction is supported on channels 1/2 and 3/4.
Signed-off-by: Marius Cristea <marius.cristea@microchip.com>
Link: https://lore.kernel.org/r/20260610-hw_mon-emc1812-v11-2-cef809af5c19@microchip.com
Signed-off-by: Guenter Roeck <linux@roeck-us.net>
rpcrdma_prepare_send_sges() gets a sendctx before it maps the SGEs
for the Send WR. If one of the mapping helpers fails, no Send WR
is posted, so no Send completion is guaranteed to advance rb_sc_tail.
Current cleanup clears sc_req so a later completion can sweep over
that slot, but a consecutive run of preparation failures can still
advance rb_sc_head until the ring appears full. At that point
rpcrdma_sendctx_get_locked() returns NULL and no Send can be posted to
produce the completion needed to recover the ring.
The trigger requires CONFIG_SUNRPC_XPRT_RDMA and an NFS/RDMA mount.
Mount setup and reliable DMA-map fault injection require local admin
authority. Unprivileged I/O on an existing mount can exercise the send
path, but a remote peer alone cannot force this local DMA-map failure.
Add rpcrdma_sendctx_unget_locked() for the single-consumer send path
to rewind rb_sc_head when the just-acquired sendctx is canceled before
ib_post_send(). Wake waiters after making the slot available again.
After the rewind, every slot the completion sweep visits belongs to a
posted Send, so rpcrdma_sendctx_put_locked() no longer needs to test
sc_req before unmapping.
Fixes: ae72950abf ("xprtrdma: Add data structure to manage RDMA Send arguments")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
rpcrdma_wc_receive() decrements the transport's Receive count for
every completion before it dispatches a successful Receive to
rpcrdma_reply_handler(). The handler must post a replacement
Receive WR before returning unless ownership of the rep has moved
elsewhere, as on the backchannel path.
Commit 2ae50ad68c ("xprtrdma: Close window between waking RPC
senders and posting Receives") moved the Receive refill out of
rpcrdma_wc_receive(), where it had run ahead of every reply, into
rpcrdma_reply_handler() so that the responder's credit grant could
be parsed before reposting. The bad-version and short-reply exits
never reach that refill: they recycle the rep and return without
calling rpcrdma_post_recvs().
A remote peer can therefore drain the client's posted Receive
queue by sending a sustained stream of replies that are shorter
than the fixed transport header or that carry an unrecognized
RPC/RDMA version. Each such reply consumes one posted Receive
without replacing it. Once the queue empties, the peer's next
Send finds no posted Receive and the transport stalls until
reconnect.
Route both malformed-reply exits through the shared repost tail
after recycling the rep, refilling against buf->rb_credits, the
most recent accepted credit grant. Neither exit updates the
congestion window, so RPCs admitted under the previous grant
remain in flight awaiting replies. A smaller refill target would
let a stream of malformed replies ratchet the posted Receive count
down to the batch floor while the congestion window still admits
rb_credits RPCs; a burst of valid replies to those RPCs could then
overrun the posted Receives, and because the client connects with
rnr_retry_count of zero, a single RNR NAK terminates the
connection. Refilling against rb_credits also restores the target
that applied to malformed replies before commit 2ae50ad68c
("xprtrdma: Close window between waking RPC senders and posting
Receives") when rpcrdma_post_recvs() computed it from rb_credits
internally. rb_credits is at least one from connection
establishment onward, so the repost path always keeps Receives
posted.
Fixes: 2ae50ad68c ("xprtrdma: Close window between waking RPC senders and posting Receives")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
The out_norqst exit in rpcrdma_reply_handler() branches away before
the credit clamp, so a reply that matches no pending request reaches
out_post carrying the raw credit value parsed from the wire.
rpcrdma_post_recvs() does not bound its @needed argument: the refill
loop allocates and chains Receive WRs until the count is satisfied or
allocation fails. A peer that sends a well-formed reply carrying an
unknown XID and an inflated credit grant therefore drives rep
allocation and Receive posting past re_max_requests on every such
reply.
Move the clamp to immediately after the credit field is parsed,
ahead of the first branch that can reach out_post, so every later
consumer sees a sanitized value. The cwnd update stays on the
matched-request path.
Fixes: 704f3f640f ("xprtrdma: Post receive buffers after RPC completion")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
rpcrdma_is_bcall() decodes a reply's first words to decide whether
the frame is a backchannel call. Two issues in that decode path
let a short or malformed reply leak the receive buffer and drain
the Receive queue.
First, the speculative peek
p = xdr_inline_decode(xdr, 0);
/* five p++ reads follow */
asks xdr_inline_decode() for zero bytes, which returns xdr->p
without consulting xdr->end. The five subsequent __be32 reads can
then walk up to 20 bytes past the wire payload into stale regbuf
contents and misclassify the reply as a backchannel call.
Second, after the post-peek
p = xdr_inline_decode(xdr, 3 * sizeof(*p));
if (unlikely(!p))
return true;
the short-header arm returns true without calling
rpcrdma_bc_receive_call(). The contract with the caller is that a
true return transfers ownership of rep to the backchannel path:
rpcrdma_reply_handler()
if (rpcrdma_is_bcall(r_xprt, rep))
return; /* bare return, skips out_post */
...
out_post:
rpcrdma_post_recvs(r_xprt, credits + ...);
Because rpcrdma_bc_receive_call() never ran, no one took rep, but
rpcrdma_reply_handler still bare-returns past rpcrdma_rep_put()
and rpcrdma_post_recvs(). The rep, with its persistently
DMA-mapped receive buffer, is orphaned on rb_all_reps and freed
only at transport teardown. This completion reposts nothing, so
its slot is reclaimed only when a later forward-channel reply
reaches out_post and rpcrdma_post_recvs() allocates a fresh rep to
backfill; absent that traffic the Receive queue drains and the
peer's Sends draw RNR NAKs.
Fix by consulting xdr->end after the zero-length peek so the five
__be32 reads cannot run unless 20 bytes of wire payload remain. A
byte-precise comparison against xdr->end is required because a
non-4-aligned receive rounds the stream's word count up past the
true payload. Also return false from the short-header arm so the
reply falls through the normal out_norqst cleanup chain
(rpcrdma_rep_put() plus rpcrdma_post_recvs()).
Fixes: 41c8f70f5a ("xprtrdma: Harden backchannel call decoding")
Assisted-by: kres:claude-opus-4-7
Signed-off-by: Chris Mason <clm@meta.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Commit 0e13dd9ea8 ("xprtrdma: Remove temp allocation of
rpcrdma_rep objects") made rpcrdma_rep objects survive disconnects.
That is normally fine, but it also means their receive regbufs keep
the size they had when they were first allocated.
Each rep's receive buffer is sized to ep->re_inline_recv when the rep
is created. rpcrdma_ep_create() resets that threshold to the
rdma_max_inline_read ceiling for every new endpoint, and the connect
handshake then shrinks it to the peer's advertised inline send size.
A rep allocated under a smaller negotiated threshold keeps that size:
on disconnect, rpcrdma_xprt_disconnect() drains and DMA-unmaps the
surviving reps but does not free or resize them.
The threshold can come back larger on the next connection. The first
peer may supply no RPC-over-RDMA CM private data, defaulting its send
size to 1024, while the reconnect target is an ordinary server
offering 4096; or, with rdma_max_inline_read raised above its default,
the reconnect target may advertise a larger svcrdma_max_req_size than
the first. rpcrdma_post_recvs() then reposts a surviving rep whose SGE
length is still the old, smaller value, and a larger inline Reply hits
a receive length error and forces another disconnect.
The undersized rep returns to the free list when its failed Receive
flushes, so the following reconnect reposts the same rep and fails the
same way. The transport flaps without making forward progress for as
long as the peer keeps advertising the larger inline size.
This is local/admin-triggerable rather than remote-triggerable: a local
administrator must create and maintain the NFS/RDMA mount, while the
server or reconnect target has to advertise a larger inline send size
and return a reply that uses it.
Fix this by checking each rep before it is reposted. If the receive
regbuf is smaller than the current endpoint's inline receive size,
reallocate it on the current RDMA device's NUMA node and reinitialize
the rep's xdr_buf before DMA-mapping and posting the Receive WR.
Fixes: 0e13dd9ea8 ("xprtrdma: Remove temp allocation of rpcrdma_rep objects")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
frwr_wp_create() creates the singleton Memory Region used to encode
padding for Write chunks whose payload length is not XDR-aligned. Its
failure paths return a negative errno and leave ep->re_write_pad_mr set
to NULL.
rpcrdma_xprt_connect() currently ignores that return value. If
frwr_wp_create() fails after the rest of the connection setup succeeds,
xprt_rdma_connect_worker() treats the connection attempt as successful
and sets XPRT_CONNECTED. A later NFS/RDMA read with a non-4-byte-aligned
receive page length reaches rpcrdma_encode_write_list(), passes the NULL
write-pad MR to encode_rdma_segment(), and dereferences it.
This is locally triggerable on an NFS/RDMA client after a connect or
reconnect hits a local MR allocation, DMA-map, MR-map, or post-send
failure; a remote peer alone cannot force the local MR setup failure.
Check the return value and fail the connect as -ENOTCONN, matching the
adjacent setup failures. This keeps XPRT_CONNECTED clear and lets the
normal reconnect path retry.
Fixes: 21037b8c22 ("xprtrdma: Provide a buffer to pad Write chunks of unaligned length")
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
rpcrdma_create_id() registers ep->re_rn with the rpcrdma ib_client
before returning the new rdma_cm_id to rpcrdma_ep_create(). However
rpcrdma_ep_create() currently stores that pointer in ep->re_id only
after rpcrdma_create_id() returns.
A local administrator can race an NFS/RDMA mount against RDMA device
removal. If rpcrdma_remove_one() observes the just-registered
notification before rpcrdma_ep_create() assigns ep->re_id,
rpcrdma_ep_removal_done() calls trace_xprtrdma_device_removal(NULL).
The tracepoint dereferences id->device->name and copies
id->route.addr.dst_addr, so the callback can crash the kernel with a
NULL pointer dereference.
Store the rdma_cm_id in ep->re_id immediately before publishing
ep->re_rn. The existing error path still destroys the id directly if
registration fails; ep is then freed by the caller without using
ep->re_id. Remove the later duplicate assignment in rpcrdma_ep_create().
Fixes: 3f4eb9ff92 ("xprtrdma: Handle device removal outside of the CM event handler")
Assisted-by: kres:openai-gpt-5
Signed-off-by: Chris Mason <clm@meta.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
rpcrdma_cm_event_handler() falls through to the disconnected: label
on RDMA_CM_EVENT_ADDR_CHANGE and calls rpcrdma_ep_put() with no
matching get when the event arrives before RDMA_CM_EVENT_ESTABLISHED.
The kref then underflows during connect teardown and
rpcrdma_xprt_disconnect() operates on a freed ep.
Reference counts across a normal connection lifecycle:
rpcrdma_ep_create() kref_init ->1
rpcrdma_xprt_connect() ep_get ->2 (before post_recvs)
RDMA_CM_EVENT_ESTABLISHED ep_get ->3
RDMA_CM_EVENT_DISCONNECTED ep_put ->2
rpcrdma_xprt_drain() ep_put ->1
rpcrdma_xprt_disconnect() tail ep_put ->0 (ep_destroy)
The connect-time get in rpcrdma_xprt_connect(), taken just before
rpcrdma_post_recvs() "while there are outstanding Receives," is
balanced by rpcrdma_xprt_drain. ADDR_CHANGE before ESTABLISHED has
no get to consume, so its put drops the count to 1 and the drain
put then frees the ep while rpcrdma_xprt_disconnect() still holds a
pointer to it.
Fix by dispatching on the prior re_connect_status via xchg(): for
prev == 0 (pre-ESTABLISHED) wake the connect waiter and return with
no put; for prev == 1 call rpcrdma_force_disconnect() and return.
The case-1 arm relies on the subsequent RDMA_CM_EVENT_DISCONNECTED
event -- reliably delivered when rdma_disconnect() is called on a
still-connected cm_id -- to balance the ESTABLISHED get;
rpcrdma_xprt_drain() continues to balance only that connect-time
get. Any other prior value means teardown is already in flight.
Fixes: 2acc5cae29 ("xprtrdma: Prevent dereferencing r_xprt->rx_ep after it is freed")
Assisted-by: kres:claude-opus-4-7
Signed-off-by: Chris Mason <clm@meta.com>
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
rpcrdma_buffer_get() and rpcrdma_buffer_put() both take rb_lock to
pop/push from the rb_send_bufs free list. Under high I/O concurrency
(e.g., nconnect=N with small random writes), this spinlock is contended
between the request submission path and the transport completion path.
Replace the list_head with an llist_head. The put side uses
lockless llist_add(), which is safe for concurrent producers. The
get side retains the spinlock to satisfy the llist single-consumer
contract portably; submitters continue to serialize there. Completion
handlers returning buffers no longer contend on rb_lock, eliminating
contention on the return path.
rb_lock remains for the MR free list and the tracking lists used
during setup and teardown. rb_free_reps already uses llist_head, so
the llist idiom is established in this structure. The precedent is the
data structure, not the locking: rb_free_reps serializes its single
consumer through the re_receiving gate in rpcrdma_post_recvs, whereas
rb_send_bufs serializes its consumer with rb_lock. Both satisfy the
llist single-consumer contract.
Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
Signed-off-by: Anna Schumaker <anna.schumaker@hammerspace.com>
Arnaldo Carvalho de Melo
Namhyung Kim
James Clark
Athira Rajeev
Abel Vesa
Uwe Kleine-König (The Capable Hub)
Rosen Penev
Arnd Bergmann
Ajay Kumar Nandam
Giovanni Cabiddu
Geert Uytterhoeven
Danny D.
Tanushree Shah
Jens Remus
Nathan Chancellor
Martin Kaiser
Armin Wolf
Marius Cristea
Chuck Lever
Chris Mason