amdgpu_amdkfd_gpuvm_free_memory_of_gpu() unpinned DOORBELL and MMIO
remap BOs (which are pinned at allocation time) before checking whether
the BO is still mapped to the GPU. When the BO is still mapped, the
function returns -EBUSY and leaves the BO alive, but it has already
been unpinned. The BO is then unpinned again when it is finally freed
during process teardown, triggering a ttm_bo_unpin() underflow warning:
WARNING: CPU: 18 PID: 15066 at ttm/ttm_bo.c:650 amdttm_bo_unpin+0x6d/0x80 [amdttm]
Workqueue: kfd_process_wq kfd_process_wq_release [amdgpu]
RIP: 0010:amdttm_bo_unpin+0x6d/0x80 [amdttm]
Call Trace:
amdgpu_bo_unpin+0x1a/0x90 [amdgpu]
amdgpu_amdkfd_gpuvm_unpin_bo+0x31/0xb0 [amdgpu]
amdgpu_amdkfd_gpuvm_free_memory_of_gpu+0x3bf/0x460 [amdgpu]
kfd_process_free_outstanding_kfd_bos+0xd4/0x170 [amdgpu]
kfd_process_wq_release+0x109/0x1b0 [amdgpu]
process_one_work+0x1e2/0x3b0
worker_thread+0x50/0x3a0
kthread+0xdd/0x100
ret_from_fork+0x29/0x50
Move the unpin after the mapped_to_gpu_memory check so it only happens
once we are committed to freeing the BO.
Fixes: d25e35bc26 ("drm/amdgpu: Pin MMIO/DOORBELL BO's in GTT domain")
Signed-off-by: Yunxiang Li <Yunxiang.Li@amd.com>
Reviewed-by: Felix Kuehling <felix.kuehling@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Compute user bad/hung queue recovery was handled by KFD using
suspend_all_queues_mes, remove_queue(or reset_queue), and
resume_all_queues_mes. Since now those steps are centralized to
amdgpu_gfx_reset_mes_compute function to sync up with KCQ and KGD user
queues, clean up redundant code and rename the function to match its
functionality.
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Host driver determines the bad_page_status, not VF.
VFs do not have access to the EEPROM, and eeprom_init
is skipped. However, check_bad_page_status is called
outside of the eeprom_init sequence without any is_vf checks.
Add a return false in __is_ras_eeprom_supported for VFs, and use
that guard in amdgpu_ras_check_bad_page_status to prevent
incorrect access to un-initialized eeprom_control object.
Signed-off-by: Victor Skvortsov <victor.skvortsov@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
vega10_odn_edit_dpm_table() and smu7_odn_edit_dpm_table() could mutate
the live ODN table for valid triples, then return 0 after detecting a
truncated buffer or out-of-range index. Validate all (index, clock,
voltage) triples first and return -EINVAL on any failure; only then
apply updates.
v2: Use distinct message for different error case, removed unused
input_level from validation loop (Lijo)
v3: Reject negative level indices, input[] is long but was compared only
against unsigned table bounds, so negative values could pass and truncate
when assigned to uint32_t input_level.
Set DPMTABLE_OD_UPDATE_SCLK/MCLK only after validation passes,
so a failed sysfs write does not leave need_update_dpm_table set for a
later commit.
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add helpers to validate custom profile params against
negative/out-of-range values. Use the helpers to validate user passed
params.
Signed-off-by: Lijo Lazar <lijo.lazar@amd.com>
Assisted-by: Claude Sonnet (Cursor AI)
Reviewed-by: Hawking Zhang <Hawking.Zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
amdgpu_regs, amdgpu_regs2, and related debugfs nodes allow
arbitrary MMIO read/write via RREG32/WREG32 without checking
security_locked_down(). On kernel_lockdown=integrity systems
this bypasses the same restrictions as /dev/mem and PCI config
space sysfs.
Check LOCKDOWN_PCI_ACCESS (matching pci-sysfs) at the entry of every
debugfs handler that performs direct register access.
v2: Use consistent check as per previous check to use
LOCKDOWN_DEBUGFS(Lijo)
v3: Do not create any entry from amdgpu_debugfs_regs_init() if
LOCKDOWN_PCI_ACCESS is active and log once. (Lijo)
Signed-off-by: Asad Kamal <asad.kamal@amd.com>
Reviewed-by: Lijo Lazar <lijo.lazar@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
suspend_all, resume_all, and remove bad queue has been integrated to a
centralized function, amdgpu_gfx_reset_mes_compute. Remove remove_queue
and resume_all in KFD and pass the known bad queue information required
for remove_queue to amdgpu_gfx_reset_mes_compute.
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Acked-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
MES in newer versions on gfx11 and gfx12 can support queue/pipe reset via
MES.
v2: update the fw version check (Jesse)
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
When driver already knows a bad queue but MES suspend_all is successful
and MES hung queue detection doesn't detect it, remove this queue refore
resume_all.
Signed-off-by: Amber Lin <Amber.Lin@amd.com>
Reviewed-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Separate settings for gfx (ME) and compute (MEC).
Use this rather than explicitly specifying it.
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
And while we are at it remove the reset parameter as it's
no longer needed.
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
And while we are at it remove the reset parameter as it's
no longer needed.
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Call MES with detect only to get the list of hung queues rather
than detecting an resetting. Then loop over the bad queues
and reset them individually and finally remove them. Skip
queues not owned by KFD.
v2: always call resume_all after queue reset
Reviewed-by: Amber Lin <Amber.Lin@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Update the parameter in SET_HW_RESOURCES API 1. Align with the setting
of enable_lr_compute_wa 2. Add enable_compute_pipe_reset to enable
pipe reset when compute queue reset failes
v2: add driver flags to track when we enable it
Signed-off-by: Shaoyun Liu <shaoyun.liu@amd.com>
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This makes the code flows cleaner and it's only supported on
the use_mmio path.
v2: fix typo
v3: fix typo
v4: directly clear ACTIVE and DEQUEUE_REQUEST (Shaoyun Liu)
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
This makes the code flows cleaner and it's only supported on
the use_mmio path.
v2: fix typo
v3: fix typo
v4: directly clear ACTIVE and DEQUEUE_REQUEST (Shaoyun Liu)
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Refactor gfx_v12_0_reset_compute_pipe() to accept explicit me, pipe, and
queue parameters instead of deriving them from the ring structure. This
enables the function to be used in generic pipe reset flows.
Introduce gfx_v12_0_clear_hqds_on_mec_pipe() to properly clear
CP_HQD_ACTIVE and CP_HQD_DEQUEUE_REQUEST for all queues on a given MEC
pipe while the pipe reset is asserted, ensuring the HQDs are torn down
correctly before deasserting reset.
Switch the KCQ reset path to use the common MEC pipe reset helper
amdgpu_gfx_mec_pipe_reset_run(), which coordinates the reset sequence
including KFD suspend/resume to avoid conflicts with user mode queues.
v2: just update the sequence (Alex)
v3: directly clear ACTIVE and DEQUEUE_REQUEST (Shaoyun Liu)
Suggested-by: Manu Rastogi <manu.rastogi@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <jesse.zhang@amd.com>
Reviewed-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Shaoyun Liu <shaoyun.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Refactor gfx_v11_0_reset_compute_pipe() to accept explicit me, pipe, and
queue parameters instead of deriving them from the ring structure. This
enables the function to be used in generic pipe reset flows.
Introduce gfx_v11_0_clear_hqds_on_mec_pipe() to properly clear
CP_HQD_ACTIVE and CP_HQD_DEQUEUE_REQUEST for all queues on a given MEC
pipe while the pipe reset is asserted, ensuring the HQDs are torn down
correctly before deasserting reset.
Switch the KCQ reset path to use the common MEC pipe reset helper
amdgpu_gfx_mec_pipe_reset_run(), which coordinates the reset sequence
including KFD suspend/resume to avoid conflicts with user mode queues.
v2: just update the sequence (Alex)
v3: directly clear ACTIVE and DEQUEUE_REQUEST (Shaoyun Liu)
Suggested-by: Manu Rastogi <manu.rastogi@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Signed-off-by: Jesse Zhang <jesse.zhang@amd.com>
Reviewed-by: Prike Liang <Prike.Liang@amd.com>
Reviewed-by: Shaoyun Liu <shaoyun.liu@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
when suspend_all_gangs is issued to pipe0 MES during system suspend or runtime PM, pipe0 can only suspend and resume queues it has tracked.
KCQs registered with a non-zero pipe slot may not be correctly handled, leaving them in an inconsistent state after resume.
v3: fix the schedule pipe issue
Suggested-by: Michael Chen <michael.chen@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Suggested-by: Shaoyun Liu <shaoyun.liu@amd.com>
Signed-off-by: Jesse Zhang <jesse.zhang@amd.com>
Reviewed-by: Prike Liang <Prike.Liang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
when suspend_all_gangs is issued to pipe0 MES during system suspend or runtime PM, pipe0 can only suspend and resume queues it has tracked.
KCQs registered with a non-zero pipe slot may not be correctly handled, leaving them in an inconsistent state after resume.
v3: fix the schedule pipe issue
v4: use schedule pipe for KQ resets
Reviewed-by: Michael Chen <michael.chen@amd.com>
Suggested-by: Michael Chen <michael.chen@amd.com>
Suggested-by: Alex Deucher <alexander.deucher@amd.com>
Suggested-by: Shaoyun Liu <shaoyun.liu@amd.com>
Signed-off-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
s/soc21_grbm_select/soc24_grbm_select/
No functional difference as the register offsets are
the same.
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
All the guilty fence parameter to be NULL. Will be needed for
future functionality.
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Add a helper to extract the first amdgpu_fence which has
not yet signalled and is thus guilty or at least collateral
damage.
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
If we've already seen a fence, don't backup the
ring contents since presumably either the previous reset
was not successful or there was something wrong with the
data.
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Return early in amdgpu_ring_set_fence_errors_and_reemit()
if ring_backup_entries_to_copy is 0. That means that either
the ring is idle and there is nothing to reemit, or there
some reason why we should reemit, so return early and
signal the fences (if applicable).
Reviewed-by: Jesse Zhang <jesse.zhang@amd.com>
Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
Yunxiang Li
Amber Lin
Victor Skvortsov
Asad Kamal
Lijo Lazar
Yifan Zhang
Alex Deucher
Shaoyun Liu
Jesse Zhang
Dave Airlie