mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2026-08-09 06:14:34 +02:00
Pull kvm updates from Paolo Bonzini:
"arm64:
This is a bit of an odd merge window on the KVM/arm64 front. There
is absolutely no new feature in the pull request. It is purely
fixes, because it is simply becoming too hard to review new stuff
when so many AI-fuelled fixes hit the list.
- Significant cleanup of the vgic-v5 PPI support which was merged in
7.1. This makes the code more maintainable, and squashes a couple
of bugs in the meantime
- Set of fixes for the handling of the MMU in an NV context,
particularly VNCR-triggered faults. S1POE support is fixed as well
- Large set of pKVM fixes, mostly addressing recurring issues around
hypervisor tracking of donated pages in obscure cases where the
donation could fail and leave things in a bizarre state
- Fixes for the so-called "lazy vgic init", which resulted in
sleeping operations in non-preemptible sections. This turned out to
be far more invasive than initially expected..
- Reduce the overhead of L1/L2 context switch by not touching the FP
registers
- Fix the way non-implemented page sizes are dealt with when a guest
insist on using them for S2 translation
- The usual set of low-impact fixes and cleanups all over the map
Loongarch:
- On a request for lazy FPU load, load all FPU state that the VM
supports instead of enabling only the part (FPU, LSX or LASX) that
caused the FPU load request
- Some enhancements about interrupt injection
- Some bug fixes and other small changes
RISC-V:
- Batch G-stage TLB flushes for GPA range based page table updates
- Convert HGEI line management to fully per-HART
- Fix missing CSR dirty marking when FWFT state updated via ONE_REG
- Fix stale FWFT feature exposure to Guest/VM
- Speed up dirty logging write faults using MMU rwlock and atomic PTE
updates using cmpxchg() for permission-only changes
- Use flexible array for APLIC IRQ state
- Use kvm_slot_dirty_track_enabled() for logging enable check on a
memslot
- Avoid skipping valid pages in kvm_riscv_gstage_wp_range()
- Avoid skipping valid pages in kvm_riscv_gstage_unmap_range()
- Use endian-specific __lelong for NACL shared memory
S390:
- KVM_PRE_FAULT_MEMORY support
- Support for 2G hugepages
- Support for the ASTFLEIE 2 facility
- Support for fast inject using kvm_arch_set_irq_inatomic
- Fix potential leak of uninitialized bytes
- A few more misc gmap fixes
x86:
- Generic support for the more granular permissions allowed by EPT,
namely "read" (which was previously usurping the U bit) and
separate execution bits for kernel and userspace
- Do not assume that all page tables start with U=1/W=1/NX=0 at the
root, as AMD GMET needs to have U=0 at the root
- Introduce common assembly macros for use within Intel and AMD
vendor-specific vmentry code. This touches the SPEC_CTRL handling,
which is now entirely done in assembly for Intel (by reusing the
AMD code that already existed), and register save/restore which
uses some macro magic to compute the offsets in the struct. Both of
these are preparatory changes for upcoming APX support
- Clean up KVM's register tracking and storage, primarily to prepare
for APX support, which expands the maximum number of GPRs from 16
to 32
- Keep a single copy of the PDPTRs rather than two, since
architecturally there is just one
- Handle EXIT_FASTPATH_EXIT_USERSPACE in vendor code to ensure vendor
code gets a chance to handle things like reaping the PML buffer
- Update KVM's view of PV async enabling if and only if the MSR write
fully succeeds
- Fix a variety of issues where the emulator doesn't honor
guest-debug state, and clean up related code along the way
- Synthesize EPT Violation and #NPF "error code" bits when injecting
faults into L1 that didn't originate in hardware (in which case the
VMCS/VMCB doesn't hold relevant information)
- Add support for virtualizing (well, emulating) AMD's flavor of
CPL>0 CPUID faulting
- Clean up the GPR APIs so that KVM's use of "raw" is consistent, and
fix a variety of minor bugs along the way
- Fix an OOB memory access due to not checking the VP ID when
handling a Hyper-V PV TLB flush for L2
- Fix a bug in the mediated PMU's handling of fixed counters that
allowed the guest to bypass the PMU event filter
- Allow userspace to return EAGAIN when handling SNP and TDX
hypercalls, so the KVM can forward a "retry" status code to the
guest, and reserve all unused error codes for future usage
- Overhaul the TDP MMU => S-EPT code to move as much S-EPT specific
logic as possible into the TDX code, and to funnel (almost) all
S-EPT updates into a single chokepoint. The motivation is largely
to prepare for upcoming Dynamic PAMT support, but the cleanups are
nice to have on their own
- Plug a hole in shadow page table handling, where KVM fails to
recursively zap nested EPT/NPT shadow page tables when the nested
hypervisor tears down its own EPT/NPT page tables from the bottom
up
x86 (Intel):
- Support for nested MBEC (Mode-Based Execute Control), see above in
the generic section; also run with MBEC enabled even for non-nested
mode
- Use the kernel's "enum pg_level" in the TDX APIs instead of the
TDX-Module's level definitions (which are 0-based)
- Rework the TDX memory APIs to not require/assume that guest memory
is backed by "struct page" (in prepartion for guest_memfd hugepage
support)
- Fix a largely benign bug where KVM TDX would incorrectly state it
could emulate several x2APIC MSRs
- Use the "safe" WRMSR API when proxying LBR MSR writes as the
to-be-written value is guest controlled and completely unvalidated
x86 (AMD):
- Support for nested GMET (Guest Mode Execution Trap), see above in
the generic section; also run with GMET enabled even for non-nested
mode
- Fixes and minor cleanups to GHCB handling, on top of the earlier
work already merged into 7.1-rc
- Ensure KVM's copy of CR0 and CR3 are up-to-date prior to invoking
fastpath handlers
- Add support for virtualizing gPAT (KVM previously just used L1's
PAT when running L2)
- Fix goofs where KVM mishandles side effects (e.g. single-step and
PMC updates) when emulating VMRUN
- Fix a variety of bugs in AVIC's handling of x2APIC MSR
interception, most notably where KVM didn't disable interception of
IRR, ISR, and TMR regs
- Add support for virtualizing Host-Only/Guest-Only bits in the
mediated PMU
- Don't advertise support for unusable VM types, and account for VM
types that are disabled by firmware, e.g. to mitigate security
vulnerabilities
- Rewrite the SEV {en,de}crypt debug ioctls as they were riddle with
bugs and unnecessarily complicated, and add comprehensive tests
- Clean up and deduplicate the SEV page pinning code
- Fix minor goofs related to writing back CPUID information after
firmware rejects a CPUID page for an SNP vCPU
Generic:
- Rename invalidate_begin() to invalidate_start() throughout KVM to
follow the kernel's nomenclature, e.g. for mmu_notifiers
- Use guard() to cleanup up various KVM+VFIO flows
- Minor cleanups
guest_memfd:
- Return -EEXIST instead of -EINVAL if userspace attempts to bind a
gmem range to multiple memslots, and fix the test that was supposed
to ensure KVM returns -EEXIST
- Treat memslot binding offsets and sizes as unsigned values to fix a
bug where KVM interprets a large "offset + size" as a negative
value and allows a nonsensical offset
- Use the inode number instead of the page offset for the NUMA
interleaving index to fix a bug where the effective index would
jump by two for consecutive pages (the caller also adds in the page
offset)
Selftests:
- Randomize the dirty log test's delay when reaping the bitmap on the
first pass, as always waiting only 1ms hid a KVM RISC-V bug as the
test reaped the bitmap before KVM could build up enough state to
hit the bug
- A pile of one-off fixes and cleanups"
* tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: (326 commits)
KVM: x86/mmu: Ensure hugepage is in by slot before checking max mapping level
KVM: x86: Fix shadow paging use-after-free due to unexpected role
KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject
KVM: s390: Enable adapter_indicators_set to use mapped pages
KVM: s390: Add map/unmap ioctl and clean mappings post-guest
riscv: kvm: Use endian-specific __lelong for NACL shared memory
KVM: selftests: access_tracking_perf_test: bump number of NUMA nodes to 32
KVM: s390: vsie: Implement ASTFLEIE facility 2
KVM: s390: vsie: Refactor handle_stfle
s390/sclp: Detect ASTFLEIE 2 facility
KVM: s390: Minor refactor of base/ext facility lists
KVM: x86/mmu: move pdptrs out of the MMU
KVM: x86: check that kvm_handle_invpcid is only invoked with shadow paging
KVM: nSVM: invalidate cached PDPTRs across nested NPT transitions
KVM: nVMX: remove unnecessary code in prepare_vmcs02_rare
KVM: x86: remove nested_mmu from mmu_is_nested()
KVM: arm64: vgic-its: Make ABI commit helpers return void
KVM: s390: Initialize KVM_S390_GET_CMMA_BITS memory
LoongArch: KVM: Add missing slots_lock for device register/unregister
LoongArch: KVM: Validate irqchip index in irqfd routing
...
189 lines
5.8 KiB
C
189 lines
5.8 KiB
C
/* SPDX-License-Identifier: GPL-2.0 */
|
|
/* Copyright (C) 2021-2022 Intel Corporation */
|
|
#ifndef _ASM_X86_TDX_H
|
|
#define _ASM_X86_TDX_H
|
|
|
|
#include <linux/init.h>
|
|
#include <linux/bits.h>
|
|
#include <linux/mmzone.h>
|
|
#include <linux/kvm_types.h>
|
|
|
|
#include <asm/errno.h>
|
|
#include <asm/ptrace.h>
|
|
#include <asm/trapnr.h>
|
|
#include <asm/shared/tdx.h>
|
|
|
|
/*
|
|
* SW-defined error codes.
|
|
*
|
|
* Bits 47:40 == 0xFF indicate Reserved status code class that never used by
|
|
* TDX module.
|
|
*/
|
|
#define TDX_ERROR _BITUL(63)
|
|
#define TDX_NON_RECOVERABLE _BITUL(62)
|
|
#define TDX_SW_ERROR (TDX_ERROR | GENMASK_ULL(47, 40))
|
|
#define TDX_SEAMCALL_VMFAILINVALID (TDX_SW_ERROR | _UL(0xFFFF0000))
|
|
|
|
#define TDX_SEAMCALL_GP (TDX_SW_ERROR | X86_TRAP_GP)
|
|
#define TDX_SEAMCALL_UD (TDX_SW_ERROR | X86_TRAP_UD)
|
|
|
|
/*
|
|
* TDX module SEAMCALL leaf function error codes
|
|
*/
|
|
#define TDX_SUCCESS 0ULL
|
|
#define TDX_RND_NO_ENTROPY 0x8000020300000000ULL
|
|
|
|
/* Bit definitions of TDX_FEATURES0 metadata field */
|
|
#define TDX_FEATURES0_TD_PRESERVING BIT_ULL(1)
|
|
#define TDX_FEATURES0_NO_RBP_MOD BIT_ULL(18)
|
|
|
|
#ifndef __ASSEMBLER__
|
|
|
|
#include <uapi/asm/mce.h>
|
|
#include <asm/tdx_global_metadata.h>
|
|
#include <linux/pgtable.h>
|
|
|
|
/*
|
|
* TDX module and P-SEAMLDR version convention: "major.minor.update"
|
|
* (e.g., "1.5.08") with zero-padded two-digit update field.
|
|
*/
|
|
#define TDX_VERSION_FMT "%u.%u.%02u"
|
|
|
|
/*
|
|
* Used by the #VE exception handler to gather the #VE exception
|
|
* info from the TDX module. This is a software only structure
|
|
* and not part of the TDX module/VMM ABI.
|
|
*/
|
|
struct ve_info {
|
|
u64 exit_reason;
|
|
u64 exit_qual;
|
|
/* Guest Linear (virtual) Address */
|
|
u64 gla;
|
|
/* Guest Physical Address */
|
|
u64 gpa;
|
|
u32 instr_len;
|
|
u32 instr_info;
|
|
};
|
|
|
|
#ifdef CONFIG_INTEL_TDX_GUEST
|
|
|
|
void __init tdx_early_init(void);
|
|
|
|
void tdx_get_ve_info(struct ve_info *ve);
|
|
|
|
bool tdx_handle_virt_exception(struct pt_regs *regs, struct ve_info *ve);
|
|
|
|
void tdx_halt(void);
|
|
|
|
bool tdx_early_handle_ve(struct pt_regs *regs);
|
|
|
|
int tdx_mcall_get_report0(u8 *reportdata, u8 *tdreport);
|
|
|
|
int tdx_mcall_extend_rtmr(u8 index, u8 *data);
|
|
|
|
u64 tdx_hcall_get_quote(u8 *buf, size_t size);
|
|
|
|
void __init tdx_dump_attributes(u64 td_attr);
|
|
void __init tdx_dump_td_ctls(u64 td_ctls);
|
|
|
|
#else
|
|
|
|
static inline void tdx_early_init(void) { };
|
|
static inline void tdx_halt(void) { };
|
|
|
|
static inline bool tdx_early_handle_ve(struct pt_regs *regs) { return false; }
|
|
|
|
#endif /* CONFIG_INTEL_TDX_GUEST */
|
|
|
|
#if defined(CONFIG_KVM_GUEST) && defined(CONFIG_INTEL_TDX_GUEST)
|
|
long tdx_kvm_hypercall(unsigned int nr, unsigned long p1, unsigned long p2,
|
|
unsigned long p3, unsigned long p4);
|
|
#else
|
|
static inline long tdx_kvm_hypercall(unsigned int nr, unsigned long p1,
|
|
unsigned long p2, unsigned long p3,
|
|
unsigned long p4)
|
|
{
|
|
return -ENODEV;
|
|
}
|
|
#endif /* CONFIG_INTEL_TDX_GUEST && CONFIG_KVM_GUEST */
|
|
|
|
#ifdef CONFIG_INTEL_TDX_HOST
|
|
void tdx_init(void);
|
|
int tdx_cpu_enable(void);
|
|
const char *tdx_dump_mce_info(struct mce *m);
|
|
const struct tdx_sys_info *tdx_get_sysinfo(void);
|
|
|
|
static inline bool tdx_supports_runtime_update(const struct tdx_sys_info *sysinfo)
|
|
{
|
|
return sysinfo->features.tdx_features0 & TDX_FEATURES0_TD_PRESERVING;
|
|
}
|
|
|
|
int tdx_guest_keyid_alloc(void);
|
|
u32 tdx_get_nr_guest_keyids(void);
|
|
void tdx_guest_keyid_free(unsigned int keyid);
|
|
|
|
void tdx_quirk_reset_paddr(unsigned long base, unsigned long size);
|
|
|
|
struct tdx_td {
|
|
/* TD root structure: */
|
|
struct page *tdr_page;
|
|
|
|
int tdcs_nr_pages;
|
|
/* TD control structure: */
|
|
struct page **tdcs_pages;
|
|
|
|
/* Size of `tdcx_pages` in struct tdx_vp */
|
|
int tdcx_nr_pages;
|
|
};
|
|
|
|
struct tdx_vp {
|
|
/* TDVP root page */
|
|
struct page *tdvpr_page;
|
|
/* precalculated page_to_phys(tdvpr_page) for use in noinstr code */
|
|
phys_addr_t tdvpr_pa;
|
|
|
|
/* TD vCPU control structure: */
|
|
struct page **tdcx_pages;
|
|
};
|
|
|
|
void tdx_sys_disable(void);
|
|
|
|
u64 tdh_vp_enter(struct tdx_vp *vp, struct tdx_module_args *args);
|
|
u64 tdh_mng_addcx(struct tdx_td *td, struct page *tdcs_page);
|
|
u64 tdh_mem_page_add(struct tdx_td *td, u64 gpa, kvm_pfn_t pfn, struct page *source,
|
|
u64 *ext_err1, u64 *ext_err2);
|
|
u64 tdh_mem_sept_add(struct tdx_td *td, u64 gpa, enum pg_level level, struct page *page, u64 *ext_err1, u64 *ext_err2);
|
|
u64 tdh_vp_addcx(struct tdx_vp *vp, struct page *tdcx_page);
|
|
u64 tdh_mem_page_aug(struct tdx_td *td, u64 gpa, enum pg_level level, kvm_pfn_t pfn,
|
|
u64 *ext_err1, u64 *ext_err2);
|
|
u64 tdh_mem_range_block(struct tdx_td *td, u64 gpa, enum pg_level level, u64 *ext_err1, u64 *ext_err2);
|
|
u64 tdh_mng_key_config(struct tdx_td *td);
|
|
u64 tdh_mng_create(struct tdx_td *td, u16 hkid);
|
|
u64 tdh_vp_create(struct tdx_td *td, struct tdx_vp *vp);
|
|
u64 tdh_mng_rd(struct tdx_td *td, u64 field, u64 *data);
|
|
u64 tdh_mr_extend(struct tdx_td *td, u64 gpa, u64 *ext_err1, u64 *ext_err2);
|
|
u64 tdh_mr_finalize(struct tdx_td *td);
|
|
u64 tdh_vp_flush(struct tdx_vp *vp);
|
|
u64 tdh_mng_vpflushdone(struct tdx_td *td);
|
|
u64 tdh_mng_key_freeid(struct tdx_td *td);
|
|
u64 tdh_mng_init(struct tdx_td *td, u64 td_params, u64 *extended_err);
|
|
u64 tdh_vp_init(struct tdx_vp *vp, u64 initial_rcx, u32 x2apicid);
|
|
u64 tdh_vp_rd(struct tdx_vp *vp, u64 field, u64 *data);
|
|
u64 tdh_vp_wr(struct tdx_vp *vp, u64 field, u64 data, u64 mask);
|
|
u64 tdh_phymem_page_reclaim(struct page *page, u64 *tdx_pt, u64 *tdx_owner, u64 *tdx_size);
|
|
u64 tdh_mem_track(struct tdx_td *tdr);
|
|
u64 tdh_mem_page_remove(struct tdx_td *td, u64 gpa, enum pg_level level, u64 *ext_err1, u64 *ext_err2);
|
|
u64 tdh_phymem_cache_wb(bool resume);
|
|
u64 tdh_phymem_page_wbinvd_tdr(struct tdx_td *td);
|
|
u64 tdh_phymem_page_wbinvd_hkid(u64 hkid, kvm_pfn_t pfn);
|
|
#else
|
|
static inline void tdx_init(void) { }
|
|
static inline u32 tdx_get_nr_guest_keyids(void) { return 0; }
|
|
static inline const char *tdx_dump_mce_info(struct mce *m) { return NULL; }
|
|
static inline const struct tdx_sys_info *tdx_get_sysinfo(void) { return NULL; }
|
|
static inline void tdx_sys_disable(void) { }
|
|
#endif /* CONFIG_INTEL_TDX_HOST */
|
|
|
|
#endif /* !__ASSEMBLER__ */
|
|
#endif /* _ASM_X86_TDX_H */
|