amdgpu_pci_remove() calls drm_dev_unplug() before invoking the driver
fini routines. This causes drm_dev_enter() in amdgpu_ttm_fini() to
always return false, so iounmap(aper_base_kaddr) never runs on normal
driver unload, leaving an orphaned entry in the x86 PAT interval tree.
On connected_to_cpu hardware, the aperture is mapped write-back (WB) via
ioremap_cache(). On reload, IP discovery calls memremap(..., MEMREMAP_WC)
over the same range. The WC vs WB conflict causes:
ioremap error for 0x..., requested 0x1, got 0x0
amdgpu: discovery failed: -2
Fix by switching to devres-managed mappings so cleanup is guaranteed
regardless of drm_dev_enter() state:
- connected_to_cpu path: devm_memremap(MEMREMAP_WB). For
IORESOURCE_SYSTEM_RAM ranges this takes the try_ram_remap() shortcut,
returning __va(offset) from the existing kernel direct map. No new
ioremap VA or PAT entry is created, so there is nothing to orphan.
- dGPU path: devm_ioremap_wc() registers iounmap() as a devres action,
guaranteeing cleanup at device_del() time.
Also remove iounmap(aper_base_kaddr) from amdgpu_device_unmap_mmio()
since the mapping is now devres-owned.
v2: Remove redundant x86_64 guard (Lijo)
Fixes: 9d0af8b4de ("drm/amdgpu: pre-map device buffer as cached for A+A config")
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit d871e99879)
Cc: stable@vger.kernel.org
The GPU reload test (S3 / mode1 reset / module reload) triggers a
WARN_ON in amdgpu_irq_put() on gfx10 when unloading amdgpu:
WARNING: CPU: 0 PID: 2314 at amd/amdgpu/amdgpu_irq.c:676 amdgpu_irq_put+0xc3/0xe0 [amdgpu]
Call Trace:
gfx_v10_0_hw_fini+0x41/0x150 [amdgpu]
amdgpu_ip_block_hw_fini+0x29/0xc0 [amdgpu]
amdgpu_device_fini_hw+0x315/0x610 [amdgpu]
amdgpu_driver_unload_kms+0x7c/0x90 [amdgpu]
amdgpu_pci_remove+0x51/0x90 [amdgpu]
amdgpu_device_ip_resume_phase2() skips IP blocks whose status.hw is
already set, but amdgpu_device_ip_suspend_phase2() never had the
matching guard, so a block can be suspended twice (e.g. a reset or
recovery issued while the device is already suspended). The second
suspend runs hw_fini again, which now releases the gfx fault IRQs
unconditionally, dropping a refcount that is already zero and tripping
the WARN_ON in amdgpu_irq_put().
The fault/EOP IRQ get/put were balanced through late_init/hw_fini
before, which masked the double-suspend; moving the get into hw_init
made the suspend/resume asymmetry visible as an IRQ refcount underflow.
Honor status.hw in ip_suspend_phase2() so suspend mirrors resume and a
block is only torn down once.
Fixes: 9117d8be85 ("drm/amdgpu/gfx: move fault and EOP IRQ get/put to hw_init/hw_fini")
Fixes: 482f0e5385 ("drm/amdgpu: fix double ucode load by PSP(v3)")
Signed-off-by: Yunxiang Li <Yunxiang.Li@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit f44f2af13c)
The guilty handling tried to establish a second way of signaling problems with
the GPU back to userspace. This caused quite a bunch of issue we had to work
around, especially lifetime issues with the drm_sched_entity.
Just drop the handling altogether and use the dma_fence based approach instead.
v2: fix reversed condition in entity check (Alex)
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add lockdep annotations to teach lockdep the correct lock hierarchy
and catch ordering violations during development. This follows the
pattern established by dma-resv in drivers/dma-buf/dma-resv.c.
Lock ordering hierarchy (outermost to innermost):
1. userq_sch_mutex - Global userq scheduler (enforce_isolation)
2. userq_mutex - Per-context userq (held across queue create/destroy)
3. notifier_lock - MMU notifier synchronization
4. vram_lock - VRAM memory allocator
5. reset_domain->sem - GPU reset synchronization
6. reset_lock - Reset control mutex
7. srbm_mutex - SRBM register access
8. grbm_idx_mutex - GRBM index register access
9. mmio_idx_lock - MMIO index access (spinlock)
The implementation provides:
- Lock ordering training at module init (amdgpu_lockdep_init)
- Lock class association for real driver locks (amdgpu_lockdep_set_class)
Dummy locks are associated with the same class keys as real driver locks
via lockdep_set_class(), ensuring lockdep connects the training ordering
with actual runtime locks.
Testing:
Build the kernel with CONFIG_PROVE_LOCKING=y (enables CONFIG_LOCKDEP):
scripts/config --enable PROVE_LOCKING
scripts/config --enable DEBUG_LOCKDEP
make -j$(nproc)
On boot, dmesg should show:
AMDGPU: Lockdep annotations initialized (9 lock levels)
The companion IGT test (tests/amdgpu/amd_lockdep) exercises lock-heavy
GPU code paths concurrently to trigger lockdep warnings on violations:
sudo ./build/tests/amdgpu/amd_lockdep
sudo dmesg | grep -A 50 "circular locking dependency"
IGT subtests:
concurrent-reset-and-submit - reset_sem vs submission locks
concurrent-mmap-and-evict - mmap_lock vs vram_lock
concurrent-userptr-and-reset - notifier_lock vs reset_sem
stress-all-paths - all of the above simultaneously
A clean dmesg (no "circular locking dependency" or "possible recursive
locking detected" messages) confirms no lock ordering violations.
For CI integration, the test should be run on kernels compiled with
CONFIG_LOCKDEP=y; dmesg is scanned post-run for lockdep splats.
v2: (Christian)
- Move notifier_lock and vram_lock before reset locks in hierarchy.
HMM invalidation holds notifier_lock and can wait for GPU reset
completion, so notifier_lock must be outer to reset_domain->sem.
- Associate dummy locks with lock class keys via lockdep_set_class()
so lockdep connects training with real driver locks.
- Update commit message to list all 9 lock levels.
Requires CONFIG_PROVE_LOCKING=y to activate.
Cc: Christian Konig <christian.koenig@amd.com>
Cc: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Vitaly Prosyak <vitaly.prosyak@amd.com>
Reviewed-by: Christian Konig <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Replace BUG()/BUG_ON() with error logs and safe returns in several
places where they can be triggered by invalid userspace input,
preventing DoS via kernel panic.
Signed-off-by: Ce Sun <cesun102@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
_PR3 detection was changed in commit 134b8c5d86 ("drm/amd: Fix
detection of _PR3 on the PCIe root port") to look at the root port
of the topology containing the GPU. This however was too far because
it ignored whether or not all the intermediary bridges could power
off the device. The original design in commit b10c1c5b3a ("drm/amdgpu:
add check for ACPI power resources") was too narrow because it matched
the switches internal to the GPU.
Use the goldilocks approach and look for the first bridge outside of the
GPU and check for _PR3 on that device.
Fixes: 134b8c5d86 ("drm/amd: Fix detection of _PR3 on the PCIe root port")
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
commit eb53125a7a ("drm/amd: Add dedicated helper for
amdgpu_device_find_parent()") created a dedicated helper to find
the parent device outside of the dGPU but it had a logic error
that caused it to walk all the way up the topology and return
the wrong device.
Break out of the loop when the device is found.
Reviewed-by: Alexander Deucher <alexander.deucher@amd.com>
Fixes: eb53125a7a ("drm/amd: Add dedicated helper for amdgpu_device_find_parent()")
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
There are a few cases that code walks up the topology to find the
link partner of the integrated switch in a dGPU. Split this out
to a helper and call in all places.
This does have a functional change that amdgpu_device_gpu_bandwidth()
doesn't cache the internal link but only the parent.
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
During Mode 1 reset, the ASIC undergoes a reset cycle and becomes temporarily
inaccessible via PCIe. Any attempt to access framebuffer or MMIO registers during
this window can result in uncompleted PCIe transactions, leading to NMI panics or
system hangs.
To prevent this, Unmap all of the applications mappings of the framebuffer
and doorbell BARs before mode1 reset. Also prevent new mappings from coming in
during the reset process.
v2: remove inode in kfd_dev (Christian)
v3: correct unmap offset (Felix), remove prevent new mappings part
to avoid deadlock (Christian)
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Yifan Zhang <yifan1.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
It is illegal to schedule reset work from another reset work!
Fix this by scheduling the userq reset work directly on the work queue
of the reset domain.
Not fully tested, I leave that to the IGT test cases.
Signed-off-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Sunil Khatri <sunil.khatri@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When we do a full adapter reset, if we know the timedout fence
mark the fence with -ETIME rather than -ECANCELED so it
gets properly handled by userspace.
v2: rebase
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Introduce a Peak Tops Limiter (PTL) driver that dynamically caps
engine frequency to ensure delivered TOPS never exceeds a defined
TOPS_limit. This initial implementation provides core data structures
and kernel-space interfaces (set/get, enable/disable) to manage PTL state.
PTL performs a firmware handshake to initialize its state and update
predefined format types. It supports updating these format types at
runtime while user-space tools automatically switch PTL state, and
also allows explicitly switching PTL state via newly added commands.
Signed-off-by: Perry Yuan <perry.yuan@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
drm-misc-next for v7.1-rc1:
UAPI Changes:
- Expose per-client BO memory usage via fdinfo in amdxdna. (Hou)
- Change the default priority of drm scheduler to fair. (Tvrtko)
Cross-subsystem Changes:
- Revert hugetlb support in udmabuf. (Gunthorpe)
- Fix error in udmabuf with CONFIG_DMA_API_DEBUG(/ _SG). (Gavrilov)
- Add Docbook for DRM_IOCTL_SYNCOBJ_EVENTFD, (Ser)
clarify drm_bridge_get/put. (Tvrtko)
- Change signature of drm_connector_attach_hdr_output_metadata_property. (Canal)
- Use IOVA allocations in gpusvm and pagemap APIs. (Brost)
- Fix tracepoints vs dma-fence lifetime. (Tvrtko)
- Convert st-dma*.c tests to use kunit. (Gunthorpe)
Core Changes:
- Deduplicate counter and timestamp retrieval in vblank code. (Ville)
- Parse AMD VSDB v3 in CTA extension blocks, and use it in amdgpu. (Chen)
- Prevent bridge and encoder chain changes at inopportune times. (Ceresoli)
- Map the run queue 1:1 to the drm scheduler. (Tvrtko)
Driver Changes:
- Assorted bugfixes and (documentation) updates to rockchip, bridge/synopsis,
panfrost, tidss, accel/qaic, tilcdc, vc4, ast, imagination, panthor,
renesas, accel/amdxdna, msxfb, bridge/imx8mp, nouveau.
bridge/analogix_dp, bridge/exynos_dp, omap.
- Add support for CSW PNB601LS1-2, LGD LP116WHA-SPB1, panels.
- Add support for a lot of waveshare panels (Baryshkov)
- Support for AIE4 devices in accel/wamdxdna. (Zhang)
- Enable support for GEM shrinking in panthor. (Goel/Brezillon)
- Runtime Power Management is added to v3d. (Canal)
- Allow panel probing and use the panel bridge helper in analogix_dp. (Ding)
- Support XRGB1555 and C8 in mgag and XRGB1555 in ast. (Zimmermann)
From: Maarten Lankhorst <maarten.lankhorst@linux.intel.com>
Link: https://patch.msgid.link/bf31b1a1-951b-4f60-b226-22e8c083697d@linux.intel.com
Signed-off-by: Dave Airlie <airlied@redhat.com>
GFX V11 has GC block as default off IP.
Every time AMDGPU driver sends a request to PMFW
to unload MP1, PMFW will put GC in reset and
power down the voltage.Hence, skipping reset
for APUs with GFX V11 or later to avoid reset
related failures.
Fixes: 34355e6183 ("drm/amdgpu: Fix GFX hang on SteamDeck when amdgpu is reloaded")
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Shubhankar Milind Sardeshpande <Shubhankar.MilindSardeshpande@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
(cherry picked from commit d0a8cadffc)
Cc: stable@vger.kernel.org
GFX V11 has GC block as default off IP.
Every time AMDGPU driver sends a request to PMFW
to unload MP1, PMFW will put GC in reset and
power down the voltage.Hence, skipping reset
for APUs with GFX V11 or later to avoid reset
related failures.
Fixes: 34355e6183 ("drm/amdgpu: Fix GFX hang on SteamDeck when amdgpu is reloaded")
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Shubhankar Milind Sardeshpande <Shubhankar.MilindSardeshpande@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This will allow the use of all of them for clear/fill buffer
operations.
Since drm_sched_entity_init requires a scheduler array, we
store schedulers rather than rings. For the few places that need
access to a ring, we can get it from the sched using container_of.
Since the code is the same for all sdma versions, add a new
helper amdgpu_sdma_set_buffer_funcs_scheds to set buffer_funcs_scheds
based on the number of sdma instances.
Note: the new sched array is identical to the amdgpu_vm_manager one.
These 2 could be merged.
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Acked-by: Felix Kuehling <felix.kuehling@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
For triggering the dpc event with a single device, we still need
to set the in_link_reset flag and the dpc status.
Signed-off-by: Ce Sun <cesun102@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
The correct fix is to tie the global xa entry lifetime to the
queue lifetime: insert in amdgpu_userq_create() and erase in
amdgpu_userq_cleanup(), both at the well-defined doorbell_index key,
making the operation O(1) and resolve the fence driver UAF problem
by binding the userq driver fence to per queue.
v2: clean up the local variables initialization. (Christian)
Signed-off-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This patch continues the effort to refactor workqueue APIs, which has begun
with the changes introducing new workqueues and a new alloc_workqueue flag:
commit 128ea9f6cc ("workqueue: Add system_percpu_wq and system_dfl_wq")
commit 930c2ea566 ("workqueue: Add new WQ_PERCPU flag")
The point of the refactoring is to eventually alter the default behavior of
workqueues to become unbound by default so that their workload placement is
optimized by the scheduler.
Before that to happen after a careful review and conversion of each individual
case, workqueue users must be converted to the better named new workqueues with
no intended behaviour changes:
system_wq -> system_percpu_wq
system_unbound_wq -> system_dfl_wq
This way the old obsolete workqueues (system_wq, system_unbound_wq) can be
removed in the future.
Suggested-by: Tejun Heo <tj@kernel.org>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Marco Crivellari <marco.crivellari@suse.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This patch continues the effort to refactor workqueue APIs, which has begun
with the changes introducing new workqueues and a new alloc_workqueue flag:
commit 128ea9f6cc ("workqueue: Add system_percpu_wq and system_dfl_wq")
commit 930c2ea566 ("workqueue: Add new WQ_PERCPU flag")
The point of the refactoring is to eventually alter the default behavior of
workqueues to become unbound by default so that their workload placement is
optimized by the scheduler.
Before that to happen after a careful review and conversion of each individual
case, workqueue users must be converted to the better named new workqueues with
no intended behaviour changes:
system_wq -> system_percpu_wq
system_unbound_wq -> system_dfl_wq
This way the old obsolete workqueues (system_wq, system_unbound_wq) can be
removed in the future.
Suggested-by: Tejun Heo <tj@kernel.org>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Marco Crivellari <marco.crivellari@suse.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
In amdgpu_device_fini_hw(), deferred coredump formatting work may still
be pending when hardware and IP components are being torn down. Since
the work may access device registers and memory that will be freed or
powered off, it must be completed before proceeding.
Add a flush_work() call for adev->coredump_work, guarded by
CONFIG_DEV_COREDUMP, to ensure any pending coredump work finishes
before the device enters the early IP fini stage.
This avoids potential use-after-free or accessing hardware resources
that are no longer available.
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Suggested-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_device_get_job_timeout_settings() passes a pointer directly
to the global amdgpu_lockup_timeout[] buffer into strsep().
strsep() destructively replaces delimiter characters with '\0'
in-place.
On multi-GPU systems, this function is called once per device.
When a multi-value setting like "0,0,0,-1" is used, the first
GPU's call transforms the global buffer into "0\00\00\0-1". The
second GPU then sees only "0" (terminated at the first '\0'),
parses a single value, hits the single-value fallthrough
(index == 1), and applies timeout=0 to all rings — causing
immediate false job timeouts.
Fix this by copying into a stack-local array before calling
strsep(), so the global module parameter buffer remains intact
across calls. The buffer is AMDGPU_MAX_TIMEOUT_PARAM_LENGTH
(256) bytes, which is safe for the stack.
v2: wrap commit message to 72 columns, add Assisted-by tag.
v3: use stack array with strscpy() instead of kstrdup()/kfree()
to avoid unnecessary heap allocation (Christian).
This patch was developed with assistance from Claude (claude-opus-4-6).
Assisted-by: Claude:claude-opus-4-6
Reviewed-by: Christian König <christian.koenig@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Ruijing Dong <ruijing.dong@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Update the way drm_coredump_printer is used based on its documentation
and Xe's code: the main idea is to generate the final version in one go
and then use memcpy to return the chunks requested by the caller of
amdgpu_devcoredump_read.
The generation is moved to a separate worker thread.
This cuts the time to copy the dump from 40s to ~0s on my machine.
---
v3:
- removed adev->coredump_in_progress and instead use work as
the synchronisation mechanism
- use kvfree instead of kfree
---
Signed-off-by: Pierre-Eric Pelloux-Prayer <pierre-eric.pelloux-prayer@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Acked-by: Christian König <christian.koenig@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
If discovery has failed for any reason (such as no support for a block)
then there is no need to unwind all the IP blocks in fini. In this
condition there can actually be failures during the unwind too.
Reset num_ip_blocks to zero during failure path and skip the unnecessary
cleanup path.
Suggested-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
I found a few more paths that cleanup fails due to a NULL version pointer
on unsupported hardware.
Add NULL checks as applicable.
Fixes: 39fc2bc4da ("drm/amdgpu: Protect GPU register accesses in powergated state in some paths")
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When GPU initialization fails due to an unsupported HW block
IP blocks may have a NULL version pointer. During cleanup in
amdgpu_device_fini_hw, the code calls amdgpu_device_set_pg_state and
amdgpu_device_set_cg_state which iterate over all IP blocks and access
adev->ip_blocks[i].version without NULL checks, leading to a kernel
NULL pointer dereference.
Add NULL checks for adev->ip_blocks[i].version in both
amdgpu_device_set_cg_state and amdgpu_device_set_pg_state to prevent
dereferencing NULL pointers during GPU teardown when initialization has
failed.
Fixes: 39fc2bc4da ("drm/amdgpu: Protect GPU register accesses in powergated state in some paths")
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Mario Limonciello <mario.limonciello@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Mario Limonciello
Kenneth Feng
Asad Kamal
Thadeu Lima de Souza Cascardo
Yunxiang Li
Christian König
Vitaly Prosyak
Ce Sun
Yifan Zhang
Alex Deucher
Perry Yuan
Dave Airlie
Philip Yang
Shubhankar Milind Sardeshpande
Pierre-Eric Pelloux-Prayer
Tvrtko Ursulin
Prike Liang
Marco Crivellari
Jesse Zhang
Ruijing Dong
Hawking Zhang
Lijo Lazar