mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2026-08-09 06:14:34 +02:00
Jinjie RuanandWill Deacon
f9a82544c7
On arm64, when booting with `maxcpus` greater than the number of present
CPUs (e.g., QEMU -smp cpus=4,maxcpus=8), some CPUs are marked as 'present'
but have not yet been registered via register_cpu(). Consequently,
the per-cpu device objects for these CPUs are not yet initialized.
In cpuhp_smt_enable(), the code iterates over all present CPUs. Calling
_cpu_up() for these unregistered CPUs eventually leads to
sysfs_create_group() being called with a NULL kobject (or a kobject
without a directory), triggering the following warning in
fs/sysfs/group.c:
WARNING: fs/sysfs/group.c:137 at internal_create_group+0x41c/0x4bc, CPU#2: sh/181
[...]
Call trace:
internal_create_group+0x41c/0x4bc (P)
sysfs_create_group+0x18/0x24
topology_add_dev+0x1c/0x28
cpuhp_invoke_callback+0x104/0x20c
__cpuhp_invoke_callback_range+0x94/0x11c
_cpu_up+0x200/0x37c
When booting with ACPI, arm64 smp_prepare_cpus() currently sets all
enumerated CPUs as "present" regardless of their status in the MADT. This
causes issues with SMT hotplug control. For instance, with QEMU's
"-smp 4,maxcpus=8" configuration, the MADT GICC entries are populated as
follows:
1. The first four CPUs: `Enabled` set but `Online Capable` not set.
2. The remaining four CPUs: `Online Capable` set but `Enabled` not set
to support potential hot-plugging.
Fix this by:
1. When booting with ACPI, checking the ACPI_MADT_ENABLED flag in the GICC
entry before calling set_cpu_present() during SMP initialization.
2. Properly managing the present mask in acpi_map_cpu() and
acpi_unmap_cpu() to support actual CPU hotplug events, This aligns with
other architectures like x86 and LoongArch.
3. Update the arm64 CPU hotplug documentation to no longer state that all
online-capable vCPUs are marked as present by the kernel at boot time.
This ensures that only physically available or explicitly enabled CPUs
are in the present mask, keeping the SMT control logic consistent with
the actual hardware state.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Jonathan Cameron <jic23@kernel.org>
Cc: James Morse <james.morse@arm.com>
Cc: Yicong Yang <yangyicong@hisilicon.com>
Cc: stable@vger.kernel.org
Link: https://uefi.org/specs/ACPI/6.5/05_ACPI_Software_Programming_Model.html#gic-cpu-interface-gicc-structure
Fixes: eed4583bcf ("arm64: Kconfig: Enable HOTPLUG_SMT")
Reviewed-by: Catalin Marinas <catalin.marinas@arm.com>
Suggested-by: Catalin Marinas <catalin.marinas@arm.com>
Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
Signed-off-by: Will Deacon <will@kernel.org>
84 lines
3.9 KiB
ReStructuredText
84 lines
3.9 KiB
ReStructuredText
.. SPDX-License-Identifier: GPL-2.0
|
|
.. _cpuhp_index:
|
|
|
|
====================
|
|
CPU Hotplug and ACPI
|
|
====================
|
|
|
|
CPU hotplug in the arm64 world is commonly used to describe the kernel taking
|
|
CPUs online/offline using PSCI. This document is about ACPI firmware allowing
|
|
CPUs that were not available during boot to be added to the system later.
|
|
|
|
``possible`` and ``present`` refer to the state of the CPU as seen by linux.
|
|
|
|
|
|
CPU Hotplug on physical systems - CPUs not present at boot
|
|
----------------------------------------------------------
|
|
|
|
Physical systems need to mark a CPU that is ``possible`` but not ``present`` as
|
|
being ``present``. An example would be a dual socket machine, where the package
|
|
in one of the sockets can be replaced while the system is running.
|
|
|
|
This is not supported.
|
|
|
|
In the arm64 world CPUs are not a single device but a slice of the system.
|
|
There are no systems that support the physical addition (or removal) of CPUs
|
|
while the system is running, and ACPI is not able to sufficiently describe
|
|
them.
|
|
|
|
e.g. New CPUs come with new caches, but the platform's cache topology is
|
|
described in a static table, the PPTT. How caches are shared between CPUs is
|
|
not discoverable, and must be described by firmware.
|
|
|
|
e.g. The GIC redistributor for each CPU must be accessed by the driver during
|
|
boot to discover the system wide supported features. ACPI's MADT GICC
|
|
structures can describe a redistributor associated with a disabled CPU, but
|
|
can't describe whether the redistributor is accessible, only that it is not
|
|
'always on'.
|
|
|
|
arm64's ACPI tables assume that everything described is ``present``.
|
|
|
|
|
|
CPU Hotplug on virtual systems - CPUs not enabled at boot
|
|
---------------------------------------------------------
|
|
|
|
Virtual systems have the advantage that all the properties the system will
|
|
ever have can be described at boot. There are no power-domain considerations
|
|
as such devices are emulated.
|
|
|
|
CPU Hotplug on virtual systems is supported. It is distinct from physical
|
|
CPU Hotplug as all vCPU resources are statically described in the firmware
|
|
configuration tables (e.g. MADT), meaning their maximum possible count is
|
|
known at boot. However, vCPUs that are not enabled at boot are not marked
|
|
as ``present`` by the kernel until they are hotplugged. An example is where
|
|
a virtual machine boots with a single CPU, and additional CPUs are added
|
|
once a cloud orchestrator deploys the workload.
|
|
|
|
For a virtual machine, the VMM (e.g. Qemu) plays the part of firmware.
|
|
|
|
Virtual hotplug is implemented as a firmware policy affecting which CPUs can be
|
|
brought online. Firmware can enforce its policy via PSCI's return codes. e.g.
|
|
``DENIED``.
|
|
|
|
The ACPI tables must describe all the resources of the virtual machine. CPUs
|
|
that are hot-pluggable must have the ``online capable`` bit set and the
|
|
``enabled`` bit cleared in the MADT GICC structures to indicate they can be
|
|
enabled later. The boot CPU must be marked as ``enabled`` with its
|
|
``online capable`` bit cleared. The 'always on' GICR structure must be used
|
|
to describe the redistributors.
|
|
|
|
CPUs described as ``online capable`` but not ``enabled`` can be set to enabled
|
|
by the DSDT's Processor object's _STA method. On virtual systems the _STA method
|
|
must always set the ``ACPI_STA_DEVICE_PRESENT`` bit, while toggling the
|
|
``ACPI_STA_DEVICE_ENABLED`` bit to reflect its plug status. The kernel will
|
|
then dynamically mark the vCPU as ``present`` within the OS when the
|
|
``ACPI_STA_DEVICE_ENABLED`` bit becomes set during hot-add. Changes to the
|
|
firmware policy can be notified to the OS via device-check or eject-request.
|
|
|
|
CPUs described as ``enabled`` in the static table, should not have their _STA
|
|
modified dynamically by firmware. Soft-restart features such as kexec will
|
|
re-read the static properties of the system from these static tables, and
|
|
may malfunction if these no longer describe the running system. Linux will
|
|
re-discover the dynamic properties of the system from the _STA method later
|
|
during boot.
|