Can Guo <can.guo@oss.qualcomm.com> says:
Hi,
The UFS v5.0 and UFSHCI v5.0 standards have published, introducing support
for HS-G6 (46.6 Gbps per lane) through the new UniPro V3.0 interconnect
layer and M-PHY V6.0 physical layer specifications. To achieve reliable
operation at these higher speeds, UniPro V3.0 introduces TX Equalization
and Pre-Coding mechanisms that are essential for signal integrity.
This patch series implements TX Equalization support in the UFS core
driver as specified in UFSHCI v5.0, along with the necessary vendor
operations and a reference implementation for Qualcomm UFS host
controllers.
Background
==========
TX Equalization is a signal conditioning technique that compensates for
channel impairments at high data rates (HS-G4 through HS-G6). It works
by adjusting two key parameters:
- PreShoot: Pre-emphasis applied before the main signal transition
- DeEmphasis: De-emphasis applied after the main signal transition
UniPro V3.0 defines TX Equalization Training (EQTR) procedure to
automatically discover optimal TX Equalization settings. The EQTR
procedure:
1. Starts from the most reliable link state (HS-G1)
2. Iterates through all possible PreShoot and DeEmphasis combinations
3. Evaluates signal quality using Figure of Merit (FOM) measurements
4. Selects the best settings for both host and device TX lanes
For HS-G6, Pre-Coding is also introduced to further improve signal
quality. Pre-Coding must be enabled on both transmitter and receiver
when the RX_FOM indicates it is required.
Implementation Overview
=======================
The implementation follows the UFSHCI v5.0 specification and consists of:
Core Infrastructure (Patches 1-6):
- New vops callback negotiate_pwr_mode() to allow vendors to negotiate
power mode parameters before applying TX Equalization settings
- Support for HS-G6 gear enumeration
- Complete TX EQTR procedure implementation in ufs-txeq.c
- Debugfs interface for TX Equalization parameter inspection and manual
retraining
- Module parameters for adaptive TX Equalization control
Qualcomm Implementation (Patches 7-11):
- PHY-specific configurations for TX EQTR procedure
- Vendor-specific FOM measurement support
- TX Equalization settings application
- Enable TX Equalization for HW version 0x7 and onwards
The implementation is designed to be vendor-agnostic, with platform-
specific details handled through the vops callbacks. Other vendors can
add support by implementing the three new vops:
- tx_eqtr_notify(): Called before/after TX EQTR for vendor setup
- apply_tx_eqtr_settings(): Apply vendor-specific PHY configurations
- get_rx_fom(): Retrieve vendor-specific FOM measurements if needed
Module Parameters
=================
The implementation provides several module parameters for flexibility:
- use_adaptive_txeq: Enable/disable adaptive TX Equalization (default: false)
- adaptive_txeq_gear: Minimum gear for adaptive TX EQ (default: HS-G6)
- use_txeq_presets: Use only the 8 standaird presets (default: false)
- txeq_presets_selected[]: Select specific presets for EQTR
Testing
=======
This patch series has been tested on Qualcomm platforms with UFS 5.0
devices, validating:
- Successful TX EQTR completion for HS-G6
- Proper FOM evaluation and optimal settings selection
- Pre-Coding enablement for HS-G6
- Power mode changes with TX Equalization settings applied
- Report of TX Equalization settings via debugfs entries
- Report of TX EQTR histories via debug entries (see next section)
- Re-training TX Equalization via debugfs entry
Example of TX EQTR history
==========================
Device TX EQTR record summary -
Target Power Mode: HS-G6, Rate-B
Most recent record index: 2
Most recent record timestamp: 219573378 us
TX Lane 0 FOM - PreShoot\DeEmphasis
\ 0 1 2 3 4 5 6 7
0 50 70 65 - - - - x
1 x x x x x x x x
2 100 90 70 - - - - x
3 x x x x x x x x
4 95 90 - - - - - x
5 - - - - - - - x
6 x x x x x x x x
7 x x x x x x x x
TX Lane 1 FOM - PreShoot\DeEmphasis
\ 0 1 2 3 4 5 6 7
0 50 70 60 - - - - x
1 x x x x x x x x
2 100 80 65 - - - - x
3 x x x x x x x x
4 95 85 - - - - - x
5 - - - - - - - x
6 x x x x x x x x
7 x x x x x x x x
Patch Structure
===============
Patches 1-3: Preparatory changes for power mode negotiation and HS-G6
Patch 4: Core TX Equalization and EQTR implementation
Patches 5-7: Debugfs support for TX Equalization
Patches 8-12: Qualcomm vendor implementation
Next
====
One more series has been developed to enhance TX Equalization support,
which will be submitted for review after this series is accepted:
- Provide board specific (static) TX Equalization settings from DTS
- Parse static TX Equalization settings from DTS if provided
- Apply static TX Equalization settings if use_adaptive_txeq is disabled
- Add support for UFS v5.0 attributes qTxEQGnSettings & wTxEQGnSettingsExt
- Enable persistent storage and retrieval of optimal TX Equalization settings
Link: https://patch.msgid.link/20260325152154.1604082-1-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
On some platforms, HW does not support triggering TX EQTR from the most
reliable High-Speed (HS) Gear (HS Gear1), but only allows to trigger TX
EQTR for the target HS Gear from the same HS Gear. To work around the HW
limitation, implement vops tx_eqtr_notify() to change Power Mode to the
target TX EQTR HS Gear prior to TX EQTR procedure and change Power Mode
back to HS Gear1 (the most reliable gear) post TX EQTR procedure.
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-10-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
If HS-G6 Power Mode change handshake is successful and outbound data Lanes
are expected to transmit ADAPT, M-TX Lanes shall be configured as
if (Adapt Type == REFRESH)
TX_HS_ADAPT_LENGTH_L0_L1_L2_L3 = PA_PeerRxHsG6AdaptRefreshL0L1L2L3.
else if (Adapt Type == INITIAL)
TX_HS_ADAPT_LENGTH_L0_L1_L2_L3 = PA_PeerRxHsG6AdaptInitialL0L1L2L3.
On some platforms, the ADAPT_L0_L1_L2_L3 duration on Host TX Lanes is only
a half of theoretical ADAPT_L0_L1_L2_L3 duration TADAPT_L0_L1_L2_L3 (in
PAM-4 UI) calculated from TX_HS_ADAPT_LENGTH_L0_L1_L2_L3.
For such platforms, the workaround is to double the ADAPT_L0_L1_L2_L3
duration by uplifting TX_HS_ADAPT_LENGTH_L0_L1_L2_L3. UniPro initializes
TX_HS_ADAPT_LENGTH_L0_L1_L2_L3 during HS-G6 Power Mode change handshake, it
would be too late for SW to update TX_HS_ADAPT_LENGTH_L0_L1_L2_L3 post
HS-G6 Power Mode change. Update PA_PeerRxHsG6AdaptRefreshL0L1L2L3 and
PA_PeerRxHsG6AdaptInitialL0L1L2L3 post Link Startup and before HS-G6 Power
Mode change, so that the UniPro would use the updated value during HS-G6
Power Mode change handshake.
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-9-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Drastic environmental changes, such as significant temperature shifts, can
impact link signal integrity. In such cases, retraining TX Equalization is
necessary to compensate for these environmental changes.
Add a debugfs entry, 'tx_eq_ctrl', to allow userspace to manually trigger
the TX Equalization training (EQTR) procedure and apply the identified
optimal settings on the fly. These entries are created on a per-gear basis
for High Speed Gear 4 (HS-G4) and above, as TX EQTR is not supported for
lower gears.
The 'tx_eq_ctrl' entry currently accepts the 'retrain' command to initiate
the procedure. The interface is designed to be scalable to support
additional commands in the future.
Reading the 'tx_eq_ctrl' entry provides a usage hint to the user, ensuring
the interface is self-documenting.
The ufshcd's debugfs folder structure will look like below:
/sys/kernel/debug/ufshcd/*ufs*/
|--tx_eq_hs_gear1/
| |--device_tx_eq_params
| |--host_tx_eq_params
|--tx_eq_hs_gear2/
|--tx_eq_hs_gear3/
|--tx_eq_hs_gear4/
|--tx_eq_hs_gear5/
|--tx_eq_hs_gear6/
|--device_tx_eq_params
|--device_tx_eqtr_record
|--host_tx_eq_params
|--host_tx_eqtr_record
|--tx_eq_ctrl
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-8-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
In preparation for supporting TX Equalization refreshing, introduce helper
functions to safely pause and resume command processing.
ufshcd_pause_command_processing() ensures the host is in a quiescent state
by stopping the block layer tagset, acquiring the necessary locks
(scan_mutex and clk_scaling_lock), and waiting for any in-flight commands
to complete within a specified timeout.
ufshcd_resume_command_processing() restores the host to its previous
operational state by reversing these steps in the correct order.
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-7-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add debugfs support for UFS TX Equalization and UFS TX Equalization
Training (EQTR) to facilitate runtime inspection of link quality. These
entries allow developers to monitor and optimize TX Equalization parameters
and EQTR records during live operation.
The debugfs entries are organized on a per-gear basis under the HBA's
debugfs root. Since TX EQTR is only defined for High Speed Gear 4 (HS-G4)
and above, EQTR-related entries are explicitly excluded for HS-G1 through
HS-G3 to avoid exposing unsupported attributes.
The ufshcd's debugfs folder structure will look like below:
/sys/kernel/debug/ufshcd/*ufs*/
|--tx_eq_hs_gear1/
| |--device_tx_eq_params
| |--host_tx_eq_params
|--tx_eq_hs_gear2/
|--tx_eq_hs_gear3/
|--tx_eq_hs_gear4/
|--tx_eq_hs_gear5/
|--tx_eq_hs_gear6/
|--device_tx_eq_params
|--device_tx_eqtr_record
|--host_tx_eq_params
|--host_tx_eqtr_record
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Bean Huo <beanhuo@micron.com>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-6-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
MIPI Unipro3.0 introduced PA_TxEQGnSetting and PA_PreCodeEn attributes for
TX Equalization and Pre-Coding. It is Host Software's responsibility to
configure these attributes for both host and device before initiating Power
Mode Change to High-Speed Gears.
MIPI Unipro3.0 also introduced TX Equalization Training (EQTR) to identify
optimal TX Equalization settings for use by both Host's and Device's
UniPro. TX EQTR shall be initiated from the most reliable High-Speed Gear
(HS-G1) targeting High-Speed Gears (HS-G4 to HS-G6).
Implement TX Equalization configuration and TX EQTR procedure as defined
in UFSHCI v5.0 specification. The TX EQTR procedure determines the optimal
TX Equalization settings by iterating through all possible PreShoot and
DeEmphasis combinations and selecting the best combinations for both Host
and Device based on Figure of Merit (FOM) evaluation.
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-5-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Most vendor specific implemenations of vops pwr_change_notify(PRE_CHANGE)
are fulfilling two things at once:
- Vendor specific target power mode negotiation
- Vendor specific power mode change preparation
When TX Equalization is added into consideration, before power mode change
to a target power mode, TX Equalization Training (EQTR) needs be done for
that target power mode. In addition, UFSHCI spec requires to start TX EQTR
from HS-G1 (the most reliable High Speed Gear).
Adding TX EQTR before pwr_change_notify(PRE_CHANGE) is not applicable
because we don't know the negotiated power mode yet.
Adding TX EQTR post pwr_change_notify(PRE_CHANGE) is inappropriate because
pwr_change_notify(PRE_CHANGE) has finished preparation for a power mode
change to negotiated power mode, yet we are changing power mode to HS-G1
for TX EQTR.
Add a new vops negotiate_pwr_mode() so that vendor specific power mode
negotiation can be fulfilled in its vendor specific implementations. Later
on, TX EQTR can be added post vops negotiate_pwr_mode() and before vops
pwr_change_notify(PRE_CHANGE).
Reviewed-by: Bean Huo <beanhuo@micron.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260325152154.1604082-2-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Enhance the IOC shutdown process to handle transient failures during
controller cleanup. Add retry logic with configurable maximum retry
count (MPI3MR_MAX_SHUTDOWN_RETRY_COUNT) and proper timeout management
that resets on each retry attempt. This ensures shutdown can recover
from temporary issues without failing completely.
Signed-off-by: Ranjan Kumar <ranjan.kumar@broadcom.com>
Link: https://patch.msgid.link/20260320090326.47544-4-ranjan.kumar@broadcom.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The function lpfc_freenode() was renamed to lpfc_cleanup_node() by
commit 685f0bf7af ("[SCSI] lpfc 8.1.12 : Collapse discovery lists to a
single node list"), and commit a70e63eee1 ("scsi: lpfc: Fix NPIV
Fabric Node reference counting") later removed the lpfc_unreg_rpi() call
from lpfc_cleanup_node().
Remove the now-inaccurate "called from lpfc_freenode()" sentence and
reflow the remaining comment text for lpfc_unreg_rpi().
Assisted-by: unnamed:deepseek-v3.2 coccinelle
Signed-off-by: Kexin Sun <kexinsun@smail.nju.edu.cn>
Link: https://patch.msgid.link/20260321105909.7804-1-kexinsun@smail.nju.edu.cn
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The last step of virtscsi_handle_event() is to call
virtscsi_kick_event(), which calls INIT_WORK on its own work
item. INIT_WORK resets the work item's data bits to 0.
If this occurs while the work item is being flushed by
cancel_work_sync(), then kernel/workqueue.c/work_offqd_enable triggers a
kernel warning, as it expects the "disable" bit to be 1:
[ 21.450115] workqueue: work disable count underflowed
[ 21.450117] WARNING: CPU: 1 PID: 56 at kernel/workqueue.c:4328 enable_work+0x10a/0x120
...
[ 21.450171] Call Trace:
[ 21.450173] [<000003db2e5bdc3e>] enable_work+0x10e/0x120
[ 21.450176] ([<000003db2e5bdc3a>] enable_work+0x10a/0x120)
[ 21.450178] [<000003db2e5bdd86>] cancel_work_sync+0x86/0xa0
[ 21.450181] [<000003daae97d9e4>] virtscsi_remove+0xb4/0xd0 [virtio_scsi]
[ 21.450184] [<000003db2ef3b5ca>] virtio_dev_remove+0x6a/0xd0
[ 21.450186] [<000003db2ef9106c>] device_release_driver_internal+0x1ac/0x260
[ 21.450190] [<000003db2ef8edc8>] bus_remove_device+0xf8/0x190
[ 21.450192] [<000003db2ef88d72>] device_del+0x142/0x340
[ 21.450194] [<000003db2ef88fa0>] device_unregister+0x30/0xa0
[ 21.450196] [<000003db2ef3b2fa>] unregister_virtio_device+0x2a/0x40
This warning may occur if a controller is detached immediately following
a disk detach.
Move the INIT_WORK call to prevent this. Don't re-init event list work
items in virtscsi_kick_event(), init them only once in virtscsi_probe()
instead.
Signed-off-by: Joshua Daley <jdaley@linux.ibm.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Link: https://patch.msgid.link/20260325180857.3675854-2-jdaley@linux.ibm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Replace the deprecated[1] strncpy() with strscpy() in
transport_dump_vpd_proto_id(), transport_dump_vpd_assoc(),
transport_dump_vpd_ident_type(), and transport_dump_vpd_ident().
All four functions follow the same pattern: a local
buf[VPD_TMP_BUF_SIZE] (254 bytes) is zeroed with memset(), populated
via sprintf()/snprintf() (always NUL-terminated), then conditionally
copied to p_buf with strncpy(). The p_buf destination is used as a
C string by all callers in target_core_configfs.c: strlen(buf) and
sprintf(page+len, "%s", buf) to build sysfs output.
NUL-padding is not required: callers in target_core_configfs.c
pre-zero p_buf with memset() or initializer before calling these
functions, and consume p_buf only as a NUL-terminated C string via
strlen() and "%s", never exposing trailing bytes.
No behavioral change: the source buf is always NUL-terminated and
shorter than VPD_TMP_BUF_SIZE, so strscpy() produces identical output.
Link: https://github.com/KSPP/linux/issues/90 [1]
Signed-off-by: Kees Cook <kees@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Link: https://patch.msgid.link/20260323171311.work.101-kees@kernel.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
clang-22 started warning about functions that take printf format
strings:
drivers/scsi/esas2r/esas2r_log.c:160:50: error: diagnostic behavior may be improved by adding the 'format(printf, 3, 0)' attribute to the declaration of 'esas2r_log_master' [-Werror,-Wmissing-format-attribute]
121 | retval = vsnprintf(buffer, buflen, format, args);
| ^
drivers/scsi/esas2r/esas2r_log.c:121:12: note: 'esas2r_log_master' declared here
121 | static int esas2r_log_master(const long level,
| ^
The warning already got silenced for gcc but not clang in the past.
Rather than modify that hack to turn it off for both, just add the
attribute as suggested and remove the pragma again.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260323100027.1975646-1-arnd@kernel.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
lpfc_cgn_calc_crc32(data, size, crc) is really just an open-coded version
of ~crc32c(bitrev32(crc), data, size). However, all callers pass crc ==
~0, so it can be simplified even further to just ~crc32c(~0, data, size).
Remove the crc argument and implement it that way.
While we're at it, also use proper types in the function prototype.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Link: https://patch.msgid.link/20260316223631.72361-1-ebiggers@kernel.org
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add support for handling aggregation-based interrupts when operating in MCQ
mode.
In legacy interrupt mode, an IE.IAGES is triggered when the counter or
timer threshold is reached. To manage this, the handler now resets the
aggregation counter and timer by writing to the MCQIACRy.CTR register.
Since the register layout of MCQIACRy is identical to the existing UTRIACR
register, this implementation reuses the previously defined bitfield masks
to maintain consistency and reduce code duplication.
Extend ufshcd_handle_mcq_cq_events() with a boolean iag parameter. If set,
the handler resets the MCQ IAG counter and timer.
Define MCQ_IAG_EVENT_STATUS (0x200000) and include it in
UFSHCD_ENABLE_MCQ_INTRS to ensure the interrupt is unmasked during
initialization.
Signed-off-by: Vamshi Gajjela <vamshigajjela@google.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260310190308.2474956-1-vamshigajjela@google.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add the mphy reset property to the devicetree bindings for the Rockchip
RK3576 UFS host controller. The mphy reset signal is used to reset the
physical adapter. Resetting other components while leaving the mphy unreset
may occasionally prevent the UFS controller from successfully linking up
with the device.
This addresses an intermittent hardware bug where the UFS link fails to
establish under specific timing conditions with certain chips. While
difficult to reproduce initially, this issue was consistently observed in
downstream testing and requires explicit mphy reset control for full
stability.
Although this change increases the maxItems for resets and adds a new entry
(which technically alters the binding ABI), it does not break compatibility
for existing Linux systems. The driver uses
devm_reset_control_array_get_exclusive() to manage resets, allowing it to
function correctly with both older Device Trees (without the mphy entry)
and newer ones.
Fixes: d90e920237 ("scsi: ufs: dt-bindings: Document Rockchip UFS host controller")
Signed-off-by: Shawn Lin <shawn.lin@rock-chips.com>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Link: https://patch.msgid.link/1773368467-109650-1-git-send-email-shawn.lin@rock-chips.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
There were two copy and paste type of errors where I swapped 'complete'
with 'submit' names. Use the correct variables and definitions.
This problem was found by Dmitry Bogdanov and this patch builds on top of
his patch to fix a second instance that was found.
Fixes: 06933066d8 ("scsi: target: Add support for completing commands from backend context")
Signed-off-by: Dmitry Bogdanov <d.bogdanov@yadro.com>
[mnc: Fixed second instance]
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://patch.msgid.link/20260307220630.131008-1-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The UniPro stack manages to repair many potential Link problems without the
need to notify the Application Layer. Repair mechanisms of the stack
include L2 re-transmission and successful handling of PA_INIT.req.
Nevertheless, any successful repair sequence requires Link bandwidth that
is no longer vailable for the Application. Therefore, it may be useful for
an Application to understand how often such repair attempts are made.
The DME implements Quality of Service monitoring using a simple counting
scheme, counting error events and comparing them against the number of
correctly received or transmitted bytes. When the error counter exceeds a
programmed threshold before the byte counter overflows, a DME_QoS.ind is
issued to the Application and both counters are reset. When the byte
counter overflows before the error counter has reached the programmed
threshold, both counters are reset without triggering a DME_QoS.ind.
The DME provides Link quality monitoring for the following purposes:
1. Detection of re-occurring repaired fatal error conditions on the Link
(PA_INIT loop). This kind of detection is useful if capabilities
exchanged between local and peer permit a potential operation at a
higher M-PHY Gear, but the physical interconnect between local and peer
Device does not, or, after Line quality degradation, no longer satisfies
channel characteristics.
2. Detection of degraded inbound or outbound Link quality, to allow an
Application to issue an ADAPT sequence for a Link running in HS-G4 or
higher HS Gears. This kind of detection is used to monitor a slowly
degrading Link quality, e.g., one being affected by temperature and
voltage variations, against the expected M-PHY bit error rate.
Userspace can configure and enable UniPro QoS via UniPro QoS Attributes
(via UFS BSG) and get notified by dme_qos_notification without polling
UniPro QoS Status attribute. The dme_qos_notification attribute is a
bitfield with the following bit assignments:
Bit Description
=== ======================================
0 DME QoS Monitor has been reset by host
1 QoS from TX is detected
2 QoS from RX is detected
3 QoS from PA_INIT is detected
Signed-off-by: Can Guo <can.guo@oss.qualcomm.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Peter Wang <peter.wang@mediatek.com>
Link: https://patch.msgid.link/20260305110856.959211-2-can.guo@oss.qualcomm.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Previously, host_lock was used to prevent bit-set conflicts in async_scan,
but this approach introduced naked reads in some code paths.
Convert async_scan from a bitfield to a bool type to eliminate bit-level
conflicts entirely. Use __guarded_by(&scan_mutex) to indicate that the
async_scan variable is protected by scan_mutex.
Signed-off-by: Chaohai Chen <wdhh6@aliyun.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: John Garry <john.g.garry@oracle.com>
Link: https://patch.msgid.link/20260305025125.3649517-1-wdhh6@aliyun.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The following patches made over Linus's current tree allow users to
tell the target layer to perform direct completions instead of always
deferring to LIO's completion workqueue. When the frontend driver
already has multiple worker threads (or you are doing a LUN per target
with a single thread per target) then bypassing the LIO workqueue can
increase performance 20-30%.
Link: https://patch.msgid.link/20260222232946.7637-1-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This has vhost-scsi report that it supports direct completions.
When using a worker task per queue or group of queues with fast backends
then enabling direct completion and submissions increases performance
20-30% with workloads like:
fio --filename=/dev/sdb --direct=1 --rw=randrw --bs=8K \
--ioengine=libaio --iodepth=128 --numjobs=$jobs
As jobs matches and passes the number of vCPUs in the VM then the
benefit increases.
However, when using a single worker then queueing completions and
submissions is best as the worker is busy handling mapping data and
setting/tearing down commands.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://patch.msgid.link/20260222232946.7637-5-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
This allows userspace to request if we complete in the backend context
or the frontend driver. It works the same as submission where you can
write 0 to 2 to complete_type:
0 - Use the fabric driver's preference.
1 - Complete from the backend calling context if the fabric supports
it.
2 - Queue the completion to LIO's completion workqueue.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://patch.msgid.link/20260222232946.7637-4-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
To complete a command several drivers just drop their reference and add
it to list to be processed by a driver specific thread. So there's no
need to go from backend context to the LIO thread then to the driver's
thread. When avoiding the LIO thread, IOPS can increase from 20-30% for
workloads like:
fio --filename=/dev/sdb --direct=1 --rw=randrw --bs=8K \
--ioengine=libaio --iodepth=128 --numjobs=$jobs
where increasing jobs increases the performance improvement (this is
using NVMe drives with LIO's submit_type=1 to directly submit).
Add the infrastructure so drivers and userspace can control how to
complete a command like is done for the submission path. In this commit
there is no behavior change and we continue to defer to the LIO
workqueue thread. In the subsequent commits we will allow drivers to
report what they support and allow userspace to control the behavior.
Signed-off-by: Mike Christie <michael.christie@oracle.com>
Link: https://patch.msgid.link/20260222232946.7637-2-michael.christie@oracle.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The WRITE SAME(16) and WRITE SAME(10) SCSI commands use a page from a
dedicated mempool (sd_page_pool) for their payload. This pool was
initialized to allocate single pages, which was sufficient as long as the
device sector size did not exceed the PAGE_SIZE.
Given that block layer now supports block size upto 64KB, i.e. beyond
PAGE_SIZE, initialize a large page pool in sd_probe() if a higher sector
device is attached, ensuring atomicity. Adapt sd_set_special_bvec() to use
large page pool when a higher sector size device is attached. Hence enable
sector sizes > PAGE_SIZE in SCSI sd driver.
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Signed-off-by: Swarna Prabhu <s.prabhu@samsung.com>
Co-developed-by: Pankaj Raghav <p.raghav@samsung.com>
Signed-off-by: Pankaj Raghav <p.raghav@samsung.com>
Link: https://patch.msgid.link/20260219043741.276729-2-sw.prabhu6@gmail.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
The parameter def_reserved_size defines the default buffer size reserved
for each Sg_fd and should be restricted to a range between 0 and 1,048,576
(see https://tldp.org/HOWTO/SCSI-Generic-HOWTO/proc.html). Although the
function sg_proc_write_dressz enforces this limit, it is possible to bypass
it by directly modifying the module parameter as shown below, which then
causes a soft lockup:
echo -1 > /sys/module/sg/parameters/def_reserved_size
exec 4<> /dev/sg0
watchdog: BUG: soft lockup - CPU#5 stuck for 26 seconds! [bash:537]
Modules loaded:
CPU: 5 UID: 0 PID: 537 Command: bash, kernel version 6.19.0-rc3+ #134,
PREEMPT disabled
Hardware: QEMU Standard PC (i440FX + PIIX, 1996), BIOS version
1.16.1-2.fc37 dated 04/01/2014
...
Call Trace:
sg_build_reserve+0x5c/0xa0
sg_add_sfp+0x168/0x270
sg_open+0x16e/0x340
chrdev_open+0xbe/0x230
do_dentry_open+0x175/0x480
vfs_open+0x34/0xf0
do_open+0x265/0x3d0
path_openat+0x110/0x290
do_filp_open+0xc3/0x170
do_sys_openat2+0x71/0xe0
__x64_sys_openat+0x6d/0xa0
do_syscall_64+0x62/0x310
entry_SYSCALL_64_after_hwframe+0x76/0x7e
The fix is to use module_param_cb to validate and reject invalid values
assigned to def_reserved_size.
Fixes: 6460e75a10 ("[SCSI] sg: fixes for large page_size")
Signed-off-by: Yang Erkun <yangerkun@huawei.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Link: https://patch.msgid.link/20260127062044.3034148-3-yangerkun@huawei.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Add a 'serial' sysfs attribute for SCSI and SATA devices. This attribute
exposes the Unit Serial Number, which is derived from the Device
Identification Vital Product Data (VPD) page 0x80.
Whitespace is stripped from the retrieved serial number to handle the
different alignment (right-aligned for SCSI, potentially left-aligned for
SATA). As noted in SAT-5 10.5.3, "Although SPC-5 defines the PRODUCT SERIAL
NUMBER field as right-aligned, ACS-5 does not require its SERIAL NUMBER
field to be right-aligned. Therefore, right-alignment of the PRODUCT SERIAL
NUMBER field for the translation is not assured."
This attribute is used by tools such as lsblk to display the serial number
of block devices.
[mkp: length adjustment]
Signed-off-by: Igor Pylypiv <ipylypiv@google.com>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Link: https://patch.msgid.link/20260209212151.342151-1-ipylypiv@google.com
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Martin K. Petersen
Can Guo
Ranjan Kumar
Kexin Sun
Joshua Daley
Dave Marquardt
Kees Cook
Arnd Bergmann
Andy Shevchenko
Eric Biggers
vamshi gajjela
Shawn Lin
Abel Vesa
Junrui Luo
Ed Tsai
Adrian Hunter
Mike Christie
Nilesh Javali
Yihang Li
Peter Wang
Chaohai Chen
Swarna Prabhu
Yang Erkun
Igor Pylypiv