mirror of
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
synced 2026-04-03 12:05:13 +02:00
commitb41c1d8d07upstream. Make fscrypt no longer use Crypto API drivers for non-inline crypto engines, even when the Crypto API prioritizes them over CPU-based code (which unfortunately it often does). These drivers tend to be really problematic, especially for fscrypt's workload. This commit has no effect on inline crypto engines, which are different and do work well. Specifically, exclude drivers that have CRYPTO_ALG_KERN_DRIVER_ONLY or CRYPTO_ALG_ALLOCATES_MEMORY set. (Later, CRYPTO_ALG_ASYNC should be excluded too. That's omitted for now to keep this commit backportable, since until recently some CPU-based code had CRYPTO_ALG_ASYNC set.) There are two major issues with these drivers: bugs and performance. First, these drivers tend to be buggy. They're fundamentally much more error-prone and harder to test than the CPU-based code. They often don't get tested before kernel releases, and even if they do, the crypto self-tests don't properly test these drivers. Released drivers have en/decrypted or hashed data incorrectly. These bugs cause issues for fscrypt users who often didn't even want to use these drivers, e.g.: - https://github.com/google/fscryptctl/issues/32 - https://github.com/google/fscryptctl/issues/9 - https://lore.kernel.org/r/PH0PR02MB731916ECDB6C613665863B6CFFAA2@PH0PR02MB7319.namprd02.prod.outlook.com These drivers have also similarly caused issues for dm-crypt users, including data corruption and deadlocks. Since Linux v5.10, dm-crypt has disabled most of them by excluding CRYPTO_ALG_ALLOCATES_MEMORY. Second, these drivers tend to be *much* slower than the CPU-based code. This may seem counterintuitive, but benchmarks clearly show it. There's a *lot* of overhead associated with going to a hardware driver, off the CPU, and back again. To prove this, I gathered as many systems with this type of crypto engine as I could, and I measured synchronous encryption of 4096-byte messages (which matches fscrypt's workload): Intel Emerald Rapids server: AES-256-XTS: xts-aes-vaes-avx512 16171 MB/s [CPU-based, Vector AES] qat_aes_xts 289 MB/s [Offload, Intel QuickAssist] Qualcomm SM8650 HDK: AES-256-XTS: xts-aes-ce 4301 MB/s [CPU-based, ARMv8 Crypto Extensions] xts-aes-qce 73 MB/s [Offload, Qualcomm Crypto Engine] i.MX 8M Nano LPDDR4 EVK: AES-256-XTS: xts-aes-ce 647 MB/s [CPU-based, ARMv8 Crypto Extensions] xts(ecb-aes-caam) 20 MB/s [Offload, CAAM] AES-128-CBC-ESSIV: essiv(cbc-aes-caam,sha256-lib) 23 MB/s [Offload, CAAM] STM32MP157F-DK2: AES-256-XTS: xts-aes-neonbs 13.2 MB/s [CPU-based, ARM NEON] xts(stm32-ecb-aes) 3.1 MB/s [Offload, STM32 crypto engine] AES-128-CBC-ESSIV: essiv(cbc-aes-neonbs,sha256-lib) 14.7 MB/s [CPU-based, ARM NEON] essiv(stm32-cbc-aes,sha256-lib) 3.2 MB/s [Offload, STM32 crypto engine] Adiantum: adiantum(xchacha12-arm,aes-arm,nhpoly1305-neon) 52.8 MB/s [CPU-based, ARM scalar + NEON] So, there was no case in which the crypto engine was even *close* to being faster. On the first three, which have AES instructions in the CPU, the CPU was 30 to 55 times faster (!). Even on STM32MP157F-DK2 which has a Cortex-A7 CPU that doesn't have AES instructions, AES was over 4 times faster on the CPU. And Adiantum encryption, which is what actually should be used on CPUs like that, was over 17 times faster. Other justifications that have been given for these non-inline crypto engines (almost always coming from the hardware vendors, not actual users) don't seem very plausible either: - The crypto engine throughput could be improved by processing multiple requests concurrently. Currently irrelevant to fscrypt, since it doesn't do that. This would also be complex, and unhelpful in many cases. 2 of the 4 engines I tested even had only one queue. - Some of the engines, e.g. STM32, support hardware keys. Also currently irrelevant to fscrypt, since it doesn't support these. Interestingly, the STM32 driver itself doesn't support this either. - Free up CPU for other tasks and/or reduce energy usage. Not very plausible considering the "short" message length, driver overhead, and scheduling overhead. There's just very little time for the CPU to do something else like run another task or enter low-power state, before the message finishes and it's time to process the next one. - Some of these engines resist power analysis and electromagnetic attacks, while the CPU-based crypto generally does not. In theory, this sounds great. In practice, if this benefit requires the use of an off-CPU offload that massively regresses performance and has a low-quality, buggy driver, the price for this hardening (which is not relevant to most fscrypt users, and tends to be incomplete) is just too high. Inline crypto engines are much more promising here, as are on-CPU solutions like RISC-V High Assurance Cryptography. Fixes:b30ab0e034("ext4 crypto: add ext4 encryption facilities") Cc: stable@vger.kernel.org Acked-by: Ard Biesheuvel <ardb@kernel.org> Link: https://lore.kernel.org/r/20250704070322.20692-1-ebiggers@kernel.org Signed-off-by: Eric Biggers <ebiggers@kernel.org> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
739 lines
22 KiB
C
739 lines
22 KiB
C
/* SPDX-License-Identifier: GPL-2.0 */
|
|
/*
|
|
* fscrypt_private.h
|
|
*
|
|
* Copyright (C) 2015, Google, Inc.
|
|
*
|
|
* Originally written by Michael Halcrow, Ildar Muslukhov, and Uday Savagaonkar.
|
|
* Heavily modified since then.
|
|
*/
|
|
|
|
#ifndef _FSCRYPT_PRIVATE_H
|
|
#define _FSCRYPT_PRIVATE_H
|
|
|
|
#include <linux/fscrypt.h>
|
|
#include <linux/siphash.h>
|
|
#include <crypto/hash.h>
|
|
#include <linux/blk-crypto.h>
|
|
|
|
#define CONST_STRLEN(str) (sizeof(str) - 1)
|
|
|
|
#define FSCRYPT_FILE_NONCE_SIZE 16
|
|
|
|
/*
|
|
* Minimum size of an fscrypt master key. Note: a longer key will be required
|
|
* if ciphers with a 256-bit security strength are used. This is just the
|
|
* absolute minimum, which applies when only 128-bit encryption is used.
|
|
*/
|
|
#define FSCRYPT_MIN_KEY_SIZE 16
|
|
|
|
/*
|
|
* This mask is passed as the third argument to the crypto_alloc_*() functions
|
|
* to prevent fscrypt from using the Crypto API drivers for non-inline crypto
|
|
* engines. Those drivers have been problematic for fscrypt. fscrypt users
|
|
* have reported hangs and even incorrect en/decryption with these drivers.
|
|
* Since going to the driver, off CPU, and back again is really slow, such
|
|
* drivers can be over 50 times slower than the CPU-based code for fscrypt's
|
|
* workload. Even on platforms that lack AES instructions on the CPU, using the
|
|
* offloads has been shown to be slower, even staying with AES. (Of course,
|
|
* Adiantum is faster still, and is the recommended option on such platforms...)
|
|
*
|
|
* Note that fscrypt also supports inline crypto engines. Those don't use the
|
|
* Crypto API and work much better than the old-style (non-inline) engines.
|
|
*/
|
|
#define FSCRYPT_CRYPTOAPI_MASK \
|
|
(CRYPTO_ALG_ALLOCATES_MEMORY | CRYPTO_ALG_KERN_DRIVER_ONLY)
|
|
|
|
#define FSCRYPT_CONTEXT_V1 1
|
|
#define FSCRYPT_CONTEXT_V2 2
|
|
|
|
/* Keep this in sync with include/uapi/linux/fscrypt.h */
|
|
#define FSCRYPT_MODE_MAX FSCRYPT_MODE_AES_256_HCTR2
|
|
|
|
struct fscrypt_context_v1 {
|
|
u8 version; /* FSCRYPT_CONTEXT_V1 */
|
|
u8 contents_encryption_mode;
|
|
u8 filenames_encryption_mode;
|
|
u8 flags;
|
|
u8 master_key_descriptor[FSCRYPT_KEY_DESCRIPTOR_SIZE];
|
|
u8 nonce[FSCRYPT_FILE_NONCE_SIZE];
|
|
};
|
|
|
|
struct fscrypt_context_v2 {
|
|
u8 version; /* FSCRYPT_CONTEXT_V2 */
|
|
u8 contents_encryption_mode;
|
|
u8 filenames_encryption_mode;
|
|
u8 flags;
|
|
u8 log2_data_unit_size;
|
|
u8 __reserved[3];
|
|
u8 master_key_identifier[FSCRYPT_KEY_IDENTIFIER_SIZE];
|
|
u8 nonce[FSCRYPT_FILE_NONCE_SIZE];
|
|
};
|
|
|
|
/*
|
|
* fscrypt_context - the encryption context of an inode
|
|
*
|
|
* This is the on-disk equivalent of an fscrypt_policy, stored alongside each
|
|
* encrypted file usually in a hidden extended attribute. It contains the
|
|
* fields from the fscrypt_policy, in order to identify the encryption algorithm
|
|
* and key with which the file is encrypted. It also contains a nonce that was
|
|
* randomly generated by fscrypt itself; this is used as KDF input or as a tweak
|
|
* to cause different files to be encrypted differently.
|
|
*/
|
|
union fscrypt_context {
|
|
u8 version;
|
|
struct fscrypt_context_v1 v1;
|
|
struct fscrypt_context_v2 v2;
|
|
};
|
|
|
|
/*
|
|
* Return the size expected for the given fscrypt_context based on its version
|
|
* number, or 0 if the context version is unrecognized.
|
|
*/
|
|
static inline int fscrypt_context_size(const union fscrypt_context *ctx)
|
|
{
|
|
switch (ctx->version) {
|
|
case FSCRYPT_CONTEXT_V1:
|
|
BUILD_BUG_ON(sizeof(ctx->v1) != 28);
|
|
return sizeof(ctx->v1);
|
|
case FSCRYPT_CONTEXT_V2:
|
|
BUILD_BUG_ON(sizeof(ctx->v2) != 40);
|
|
return sizeof(ctx->v2);
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
/* Check whether an fscrypt_context has a recognized version number and size */
|
|
static inline bool fscrypt_context_is_valid(const union fscrypt_context *ctx,
|
|
int ctx_size)
|
|
{
|
|
return ctx_size >= 1 && ctx_size == fscrypt_context_size(ctx);
|
|
}
|
|
|
|
/* Retrieve the context's nonce, assuming the context was already validated */
|
|
static inline const u8 *fscrypt_context_nonce(const union fscrypt_context *ctx)
|
|
{
|
|
switch (ctx->version) {
|
|
case FSCRYPT_CONTEXT_V1:
|
|
return ctx->v1.nonce;
|
|
case FSCRYPT_CONTEXT_V2:
|
|
return ctx->v2.nonce;
|
|
}
|
|
WARN_ON_ONCE(1);
|
|
return NULL;
|
|
}
|
|
|
|
union fscrypt_policy {
|
|
u8 version;
|
|
struct fscrypt_policy_v1 v1;
|
|
struct fscrypt_policy_v2 v2;
|
|
};
|
|
|
|
/*
|
|
* Return the size expected for the given fscrypt_policy based on its version
|
|
* number, or 0 if the policy version is unrecognized.
|
|
*/
|
|
static inline int fscrypt_policy_size(const union fscrypt_policy *policy)
|
|
{
|
|
switch (policy->version) {
|
|
case FSCRYPT_POLICY_V1:
|
|
return sizeof(policy->v1);
|
|
case FSCRYPT_POLICY_V2:
|
|
return sizeof(policy->v2);
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
/* Return the contents encryption mode of a valid encryption policy */
|
|
static inline u8
|
|
fscrypt_policy_contents_mode(const union fscrypt_policy *policy)
|
|
{
|
|
switch (policy->version) {
|
|
case FSCRYPT_POLICY_V1:
|
|
return policy->v1.contents_encryption_mode;
|
|
case FSCRYPT_POLICY_V2:
|
|
return policy->v2.contents_encryption_mode;
|
|
}
|
|
BUG();
|
|
}
|
|
|
|
/* Return the filenames encryption mode of a valid encryption policy */
|
|
static inline u8
|
|
fscrypt_policy_fnames_mode(const union fscrypt_policy *policy)
|
|
{
|
|
switch (policy->version) {
|
|
case FSCRYPT_POLICY_V1:
|
|
return policy->v1.filenames_encryption_mode;
|
|
case FSCRYPT_POLICY_V2:
|
|
return policy->v2.filenames_encryption_mode;
|
|
}
|
|
BUG();
|
|
}
|
|
|
|
/* Return the flags (FSCRYPT_POLICY_FLAG*) of a valid encryption policy */
|
|
static inline u8
|
|
fscrypt_policy_flags(const union fscrypt_policy *policy)
|
|
{
|
|
switch (policy->version) {
|
|
case FSCRYPT_POLICY_V1:
|
|
return policy->v1.flags;
|
|
case FSCRYPT_POLICY_V2:
|
|
return policy->v2.flags;
|
|
}
|
|
BUG();
|
|
}
|
|
|
|
static inline int
|
|
fscrypt_policy_v2_du_bits(const struct fscrypt_policy_v2 *policy,
|
|
const struct inode *inode)
|
|
{
|
|
return policy->log2_data_unit_size ?: inode->i_blkbits;
|
|
}
|
|
|
|
static inline int
|
|
fscrypt_policy_du_bits(const union fscrypt_policy *policy,
|
|
const struct inode *inode)
|
|
{
|
|
switch (policy->version) {
|
|
case FSCRYPT_POLICY_V1:
|
|
return inode->i_blkbits;
|
|
case FSCRYPT_POLICY_V2:
|
|
return fscrypt_policy_v2_du_bits(&policy->v2, inode);
|
|
}
|
|
BUG();
|
|
}
|
|
|
|
/*
|
|
* For encrypted symlinks, the ciphertext length is stored at the beginning
|
|
* of the string in little-endian format.
|
|
*/
|
|
struct fscrypt_symlink_data {
|
|
__le16 len;
|
|
char encrypted_path[];
|
|
} __packed;
|
|
|
|
/**
|
|
* struct fscrypt_prepared_key - a key prepared for actual encryption/decryption
|
|
* @tfm: crypto API transform object
|
|
* @blk_key: key for blk-crypto
|
|
*
|
|
* Normally only one of the fields will be non-NULL.
|
|
*/
|
|
struct fscrypt_prepared_key {
|
|
struct crypto_skcipher *tfm;
|
|
#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
|
|
struct blk_crypto_key *blk_key;
|
|
#endif
|
|
};
|
|
|
|
/*
|
|
* fscrypt_inode_info - the "encryption key" for an inode
|
|
*
|
|
* When an encrypted file's key is made available, an instance of this struct is
|
|
* allocated and stored in ->i_crypt_info. Once created, it remains until the
|
|
* inode is evicted.
|
|
*/
|
|
struct fscrypt_inode_info {
|
|
|
|
/* The key in a form prepared for actual encryption/decryption */
|
|
struct fscrypt_prepared_key ci_enc_key;
|
|
|
|
/* True if ci_enc_key should be freed when this struct is freed */
|
|
u8 ci_owns_key : 1;
|
|
|
|
#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
|
|
/*
|
|
* True if this inode will use inline encryption (blk-crypto) instead of
|
|
* the traditional filesystem-layer encryption.
|
|
*/
|
|
u8 ci_inlinecrypt : 1;
|
|
#endif
|
|
|
|
/* True if ci_dirhash_key is initialized */
|
|
u8 ci_dirhash_key_initialized : 1;
|
|
|
|
/*
|
|
* log2 of the data unit size (granularity of contents encryption) of
|
|
* this file. This is computable from ci_policy and ci_inode but is
|
|
* cached here for efficiency. Only used for regular files.
|
|
*/
|
|
u8 ci_data_unit_bits;
|
|
|
|
/* Cached value: log2 of number of data units per FS block */
|
|
u8 ci_data_units_per_block_bits;
|
|
|
|
/* Hashed inode number. Only set for IV_INO_LBLK_32 */
|
|
u32 ci_hashed_ino;
|
|
|
|
/*
|
|
* Encryption mode used for this inode. It corresponds to either the
|
|
* contents or filenames encryption mode, depending on the inode type.
|
|
*/
|
|
struct fscrypt_mode *ci_mode;
|
|
|
|
/* Back-pointer to the inode */
|
|
struct inode *ci_inode;
|
|
|
|
/*
|
|
* The master key with which this inode was unlocked (decrypted). This
|
|
* will be NULL if the master key was found in a process-subscribed
|
|
* keyring rather than in the filesystem-level keyring.
|
|
*/
|
|
struct fscrypt_master_key *ci_master_key;
|
|
|
|
/*
|
|
* Link in list of inodes that were unlocked with the master key.
|
|
* Only used when ->ci_master_key is set.
|
|
*/
|
|
struct list_head ci_master_key_link;
|
|
|
|
/*
|
|
* If non-NULL, then encryption is done using the master key directly
|
|
* and ci_enc_key will equal ci_direct_key->dk_key.
|
|
*/
|
|
struct fscrypt_direct_key *ci_direct_key;
|
|
|
|
/*
|
|
* This inode's hash key for filenames. This is a 128-bit SipHash-2-4
|
|
* key. This is only set for directories that use a keyed dirhash over
|
|
* the plaintext filenames -- currently just casefolded directories.
|
|
*/
|
|
siphash_key_t ci_dirhash_key;
|
|
|
|
/* The encryption policy used by this inode */
|
|
union fscrypt_policy ci_policy;
|
|
|
|
/* This inode's nonce, copied from the fscrypt_context */
|
|
u8 ci_nonce[FSCRYPT_FILE_NONCE_SIZE];
|
|
};
|
|
|
|
typedef enum {
|
|
FS_DECRYPT = 0,
|
|
FS_ENCRYPT,
|
|
} fscrypt_direction_t;
|
|
|
|
/* crypto.c */
|
|
extern struct kmem_cache *fscrypt_inode_info_cachep;
|
|
int fscrypt_initialize(struct super_block *sb);
|
|
int fscrypt_crypt_data_unit(const struct fscrypt_inode_info *ci,
|
|
fscrypt_direction_t rw, u64 index,
|
|
struct page *src_page, struct page *dest_page,
|
|
unsigned int len, unsigned int offs,
|
|
gfp_t gfp_flags);
|
|
struct page *fscrypt_alloc_bounce_page(gfp_t gfp_flags);
|
|
|
|
void __printf(3, 4) __cold
|
|
fscrypt_msg(const struct inode *inode, const char *level, const char *fmt, ...);
|
|
|
|
#define fscrypt_warn(inode, fmt, ...) \
|
|
fscrypt_msg((inode), KERN_WARNING, fmt, ##__VA_ARGS__)
|
|
#define fscrypt_err(inode, fmt, ...) \
|
|
fscrypt_msg((inode), KERN_ERR, fmt, ##__VA_ARGS__)
|
|
|
|
#define FSCRYPT_MAX_IV_SIZE 32
|
|
|
|
union fscrypt_iv {
|
|
struct {
|
|
/* zero-based index of data unit within the file */
|
|
__le64 index;
|
|
|
|
/* per-file nonce; only set in DIRECT_KEY mode */
|
|
u8 nonce[FSCRYPT_FILE_NONCE_SIZE];
|
|
};
|
|
u8 raw[FSCRYPT_MAX_IV_SIZE];
|
|
__le64 dun[FSCRYPT_MAX_IV_SIZE / sizeof(__le64)];
|
|
};
|
|
|
|
void fscrypt_generate_iv(union fscrypt_iv *iv, u64 index,
|
|
const struct fscrypt_inode_info *ci);
|
|
|
|
/*
|
|
* Return the number of bits used by the maximum file data unit index that is
|
|
* possible on the given filesystem, using the given log2 data unit size.
|
|
*/
|
|
static inline int
|
|
fscrypt_max_file_dun_bits(const struct super_block *sb, int du_bits)
|
|
{
|
|
return fls64(sb->s_maxbytes - 1) - du_bits;
|
|
}
|
|
|
|
/* fname.c */
|
|
bool __fscrypt_fname_encrypted_size(const union fscrypt_policy *policy,
|
|
u32 orig_len, u32 max_len,
|
|
u32 *encrypted_len_ret);
|
|
|
|
/* hkdf.c */
|
|
struct fscrypt_hkdf {
|
|
struct crypto_shash *hmac_tfm;
|
|
};
|
|
|
|
int fscrypt_init_hkdf(struct fscrypt_hkdf *hkdf, const u8 *master_key,
|
|
unsigned int master_key_size);
|
|
|
|
/*
|
|
* The list of contexts in which fscrypt uses HKDF. These values are used as
|
|
* the first byte of the HKDF application-specific info string to guarantee that
|
|
* info strings are never repeated between contexts. This ensures that all HKDF
|
|
* outputs are unique and cryptographically isolated, i.e. knowledge of one
|
|
* output doesn't reveal another.
|
|
*/
|
|
#define HKDF_CONTEXT_KEY_IDENTIFIER 1 /* info=<empty> */
|
|
#define HKDF_CONTEXT_PER_FILE_ENC_KEY 2 /* info=file_nonce */
|
|
#define HKDF_CONTEXT_DIRECT_KEY 3 /* info=mode_num */
|
|
#define HKDF_CONTEXT_IV_INO_LBLK_64_KEY 4 /* info=mode_num||fs_uuid */
|
|
#define HKDF_CONTEXT_DIRHASH_KEY 5 /* info=file_nonce */
|
|
#define HKDF_CONTEXT_IV_INO_LBLK_32_KEY 6 /* info=mode_num||fs_uuid */
|
|
#define HKDF_CONTEXT_INODE_HASH_KEY 7 /* info=<empty> */
|
|
|
|
int fscrypt_hkdf_expand(const struct fscrypt_hkdf *hkdf, u8 context,
|
|
const u8 *info, unsigned int infolen,
|
|
u8 *okm, unsigned int okmlen);
|
|
|
|
void fscrypt_destroy_hkdf(struct fscrypt_hkdf *hkdf);
|
|
|
|
/* inline_crypt.c */
|
|
#ifdef CONFIG_FS_ENCRYPTION_INLINE_CRYPT
|
|
int fscrypt_select_encryption_impl(struct fscrypt_inode_info *ci);
|
|
|
|
static inline bool
|
|
fscrypt_using_inline_encryption(const struct fscrypt_inode_info *ci)
|
|
{
|
|
return ci->ci_inlinecrypt;
|
|
}
|
|
|
|
int fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key,
|
|
const u8 *raw_key,
|
|
const struct fscrypt_inode_info *ci);
|
|
|
|
void fscrypt_destroy_inline_crypt_key(struct super_block *sb,
|
|
struct fscrypt_prepared_key *prep_key);
|
|
|
|
/*
|
|
* Check whether the crypto transform or blk-crypto key has been allocated in
|
|
* @prep_key, depending on which encryption implementation the file will use.
|
|
*/
|
|
static inline bool
|
|
fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key,
|
|
const struct fscrypt_inode_info *ci)
|
|
{
|
|
/*
|
|
* The two smp_load_acquire()'s here pair with the smp_store_release()'s
|
|
* in fscrypt_prepare_inline_crypt_key() and fscrypt_prepare_key().
|
|
* I.e., in some cases (namely, if this prep_key is a per-mode
|
|
* encryption key) another task can publish blk_key or tfm concurrently,
|
|
* executing a RELEASE barrier. We need to use smp_load_acquire() here
|
|
* to safely ACQUIRE the memory the other task published.
|
|
*/
|
|
if (fscrypt_using_inline_encryption(ci))
|
|
return smp_load_acquire(&prep_key->blk_key) != NULL;
|
|
return smp_load_acquire(&prep_key->tfm) != NULL;
|
|
}
|
|
|
|
#else /* CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
|
|
|
|
static inline int fscrypt_select_encryption_impl(struct fscrypt_inode_info *ci)
|
|
{
|
|
return 0;
|
|
}
|
|
|
|
static inline bool
|
|
fscrypt_using_inline_encryption(const struct fscrypt_inode_info *ci)
|
|
{
|
|
return false;
|
|
}
|
|
|
|
static inline int
|
|
fscrypt_prepare_inline_crypt_key(struct fscrypt_prepared_key *prep_key,
|
|
const u8 *raw_key,
|
|
const struct fscrypt_inode_info *ci)
|
|
{
|
|
WARN_ON_ONCE(1);
|
|
return -EOPNOTSUPP;
|
|
}
|
|
|
|
static inline void
|
|
fscrypt_destroy_inline_crypt_key(struct super_block *sb,
|
|
struct fscrypt_prepared_key *prep_key)
|
|
{
|
|
}
|
|
|
|
static inline bool
|
|
fscrypt_is_key_prepared(struct fscrypt_prepared_key *prep_key,
|
|
const struct fscrypt_inode_info *ci)
|
|
{
|
|
return smp_load_acquire(&prep_key->tfm) != NULL;
|
|
}
|
|
#endif /* !CONFIG_FS_ENCRYPTION_INLINE_CRYPT */
|
|
|
|
/* keyring.c */
|
|
|
|
/*
|
|
* fscrypt_master_key_secret - secret key material of an in-use master key
|
|
*/
|
|
struct fscrypt_master_key_secret {
|
|
|
|
/*
|
|
* For v2 policy keys: HKDF context keyed by this master key.
|
|
* For v1 policy keys: not set (hkdf.hmac_tfm == NULL).
|
|
*/
|
|
struct fscrypt_hkdf hkdf;
|
|
|
|
/*
|
|
* Size of the raw key in bytes. This remains set even if ->raw was
|
|
* zeroized due to no longer being needed. I.e. we still remember the
|
|
* size of the key even if we don't need to remember the key itself.
|
|
*/
|
|
u32 size;
|
|
|
|
/* For v1 policy keys: the raw key. Wiped for v2 policy keys. */
|
|
u8 raw[FSCRYPT_MAX_KEY_SIZE];
|
|
|
|
} __randomize_layout;
|
|
|
|
/*
|
|
* fscrypt_master_key - an in-use master key
|
|
*
|
|
* This represents a master encryption key which has been added to the
|
|
* filesystem. There are three high-level states that a key can be in:
|
|
*
|
|
* FSCRYPT_KEY_STATUS_PRESENT
|
|
* Key is fully usable; it can be used to unlock inodes that are encrypted
|
|
* with it (this includes being able to create new inodes). ->mk_present
|
|
* indicates whether the key is in this state. ->mk_secret exists, the key
|
|
* is in the keyring, and ->mk_active_refs > 0 due to ->mk_present.
|
|
*
|
|
* FSCRYPT_KEY_STATUS_INCOMPLETELY_REMOVED
|
|
* Removal of this key has been initiated, but some inodes that were
|
|
* unlocked with it are still in-use. Like ABSENT, ->mk_secret is wiped,
|
|
* and the key can no longer be used to unlock inodes. Unlike ABSENT, the
|
|
* key is still in the keyring; ->mk_decrypted_inodes is nonempty; and
|
|
* ->mk_active_refs > 0, being equal to the size of ->mk_decrypted_inodes.
|
|
*
|
|
* This state transitions to ABSENT if ->mk_decrypted_inodes becomes empty,
|
|
* or to PRESENT if FS_IOC_ADD_ENCRYPTION_KEY is called again for this key.
|
|
*
|
|
* FSCRYPT_KEY_STATUS_ABSENT
|
|
* Key is fully removed. The key is no longer in the keyring,
|
|
* ->mk_decrypted_inodes is empty, ->mk_active_refs == 0, ->mk_secret is
|
|
* wiped, and the key can no longer be used to unlock inodes.
|
|
*/
|
|
struct fscrypt_master_key {
|
|
|
|
/*
|
|
* Link in ->s_master_keys->key_hashtable.
|
|
* Only valid if ->mk_active_refs > 0.
|
|
*/
|
|
struct hlist_node mk_node;
|
|
|
|
/* Semaphore that protects ->mk_secret, ->mk_users, and ->mk_present */
|
|
struct rw_semaphore mk_sem;
|
|
|
|
/*
|
|
* Active and structural reference counts. An active ref guarantees
|
|
* that the struct continues to exist, continues to be in the keyring
|
|
* ->s_master_keys, and that any embedded subkeys (e.g.
|
|
* ->mk_direct_keys) that have been prepared continue to exist.
|
|
* A structural ref only guarantees that the struct continues to exist.
|
|
*
|
|
* There is one active ref associated with ->mk_present being true, and
|
|
* one active ref for each inode in ->mk_decrypted_inodes.
|
|
*
|
|
* There is one structural ref associated with the active refcount being
|
|
* nonzero. Finding a key in the keyring also takes a structural ref,
|
|
* which is then held temporarily while the key is operated on.
|
|
*/
|
|
refcount_t mk_active_refs;
|
|
refcount_t mk_struct_refs;
|
|
|
|
struct rcu_head mk_rcu_head;
|
|
|
|
/*
|
|
* The secret key material. Wiped as soon as it is no longer needed;
|
|
* for details, see the fscrypt_master_key struct comment.
|
|
*
|
|
* Locking: protected by ->mk_sem.
|
|
*/
|
|
struct fscrypt_master_key_secret mk_secret;
|
|
|
|
/*
|
|
* For v1 policy keys: an arbitrary key descriptor which was assigned by
|
|
* userspace (->descriptor).
|
|
*
|
|
* For v2 policy keys: a cryptographic hash of this key (->identifier).
|
|
*/
|
|
struct fscrypt_key_specifier mk_spec;
|
|
|
|
/*
|
|
* Keyring which contains a key of type 'key_type_fscrypt_user' for each
|
|
* user who has added this key. Normally each key will be added by just
|
|
* one user, but it's possible that multiple users share a key, and in
|
|
* that case we need to keep track of those users so that one user can't
|
|
* remove the key before the others want it removed too.
|
|
*
|
|
* This is NULL for v1 policy keys; those can only be added by root.
|
|
*
|
|
* Locking: protected by ->mk_sem. (We don't just rely on the keyrings
|
|
* subsystem semaphore ->mk_users->sem, as we need support for atomic
|
|
* search+insert along with proper synchronization with other fields.)
|
|
*/
|
|
struct key *mk_users;
|
|
|
|
/*
|
|
* List of inodes that were unlocked using this key. This allows the
|
|
* inodes to be evicted efficiently if the key is removed.
|
|
*/
|
|
struct list_head mk_decrypted_inodes;
|
|
spinlock_t mk_decrypted_inodes_lock;
|
|
|
|
/*
|
|
* Per-mode encryption keys for the various types of encryption policies
|
|
* that use them. Allocated and derived on-demand.
|
|
*/
|
|
struct fscrypt_prepared_key mk_direct_keys[FSCRYPT_MODE_MAX + 1];
|
|
struct fscrypt_prepared_key mk_iv_ino_lblk_64_keys[FSCRYPT_MODE_MAX + 1];
|
|
struct fscrypt_prepared_key mk_iv_ino_lblk_32_keys[FSCRYPT_MODE_MAX + 1];
|
|
|
|
/* Hash key for inode numbers. Initialized only when needed. */
|
|
siphash_key_t mk_ino_hash_key;
|
|
bool mk_ino_hash_key_initialized;
|
|
|
|
/*
|
|
* Whether this key is in the "present" state, i.e. fully usable. For
|
|
* details, see the fscrypt_master_key struct comment.
|
|
*
|
|
* Locking: protected by ->mk_sem, but can be read locklessly using
|
|
* READ_ONCE(). Writers must use WRITE_ONCE() when concurrent readers
|
|
* are possible.
|
|
*/
|
|
bool mk_present;
|
|
|
|
} __randomize_layout;
|
|
|
|
static inline const char *master_key_spec_type(
|
|
const struct fscrypt_key_specifier *spec)
|
|
{
|
|
switch (spec->type) {
|
|
case FSCRYPT_KEY_SPEC_TYPE_DESCRIPTOR:
|
|
return "descriptor";
|
|
case FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER:
|
|
return "identifier";
|
|
}
|
|
return "[unknown]";
|
|
}
|
|
|
|
static inline int master_key_spec_len(const struct fscrypt_key_specifier *spec)
|
|
{
|
|
switch (spec->type) {
|
|
case FSCRYPT_KEY_SPEC_TYPE_DESCRIPTOR:
|
|
return FSCRYPT_KEY_DESCRIPTOR_SIZE;
|
|
case FSCRYPT_KEY_SPEC_TYPE_IDENTIFIER:
|
|
return FSCRYPT_KEY_IDENTIFIER_SIZE;
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
void fscrypt_put_master_key(struct fscrypt_master_key *mk);
|
|
|
|
void fscrypt_put_master_key_activeref(struct super_block *sb,
|
|
struct fscrypt_master_key *mk);
|
|
|
|
struct fscrypt_master_key *
|
|
fscrypt_find_master_key(struct super_block *sb,
|
|
const struct fscrypt_key_specifier *mk_spec);
|
|
|
|
int fscrypt_get_test_dummy_key_identifier(
|
|
u8 key_identifier[FSCRYPT_KEY_IDENTIFIER_SIZE]);
|
|
|
|
int fscrypt_add_test_dummy_key(struct super_block *sb,
|
|
struct fscrypt_key_specifier *key_spec);
|
|
|
|
int fscrypt_verify_key_added(struct super_block *sb,
|
|
const u8 identifier[FSCRYPT_KEY_IDENTIFIER_SIZE]);
|
|
|
|
int __init fscrypt_init_keyring(void);
|
|
|
|
/* keysetup.c */
|
|
|
|
struct fscrypt_mode {
|
|
const char *friendly_name;
|
|
const char *cipher_str;
|
|
int keysize; /* key size in bytes */
|
|
int security_strength; /* security strength in bytes */
|
|
int ivsize; /* IV size in bytes */
|
|
int logged_cryptoapi_impl;
|
|
int logged_blk_crypto_native;
|
|
int logged_blk_crypto_fallback;
|
|
enum blk_crypto_mode_num blk_crypto_mode;
|
|
};
|
|
|
|
extern struct fscrypt_mode fscrypt_modes[];
|
|
|
|
int fscrypt_prepare_key(struct fscrypt_prepared_key *prep_key,
|
|
const u8 *raw_key, const struct fscrypt_inode_info *ci);
|
|
|
|
void fscrypt_destroy_prepared_key(struct super_block *sb,
|
|
struct fscrypt_prepared_key *prep_key);
|
|
|
|
int fscrypt_set_per_file_enc_key(struct fscrypt_inode_info *ci,
|
|
const u8 *raw_key);
|
|
|
|
int fscrypt_derive_dirhash_key(struct fscrypt_inode_info *ci,
|
|
const struct fscrypt_master_key *mk);
|
|
|
|
void fscrypt_hash_inode_number(struct fscrypt_inode_info *ci,
|
|
const struct fscrypt_master_key *mk);
|
|
|
|
int fscrypt_get_encryption_info(struct inode *inode, bool allow_unsupported);
|
|
|
|
/**
|
|
* fscrypt_require_key() - require an inode's encryption key
|
|
* @inode: the inode we need the key for
|
|
*
|
|
* If the inode is encrypted, set up its encryption key if not already done.
|
|
* Then require that the key be present and return -ENOKEY otherwise.
|
|
*
|
|
* No locks are needed, and the key will live as long as the struct inode --- so
|
|
* it won't go away from under you.
|
|
*
|
|
* Return: 0 on success, -ENOKEY if the key is missing, or another -errno code
|
|
* if a problem occurred while setting up the encryption key.
|
|
*/
|
|
static inline int fscrypt_require_key(struct inode *inode)
|
|
{
|
|
if (IS_ENCRYPTED(inode)) {
|
|
int err = fscrypt_get_encryption_info(inode, false);
|
|
|
|
if (err)
|
|
return err;
|
|
if (!fscrypt_has_encryption_key(inode))
|
|
return -ENOKEY;
|
|
}
|
|
return 0;
|
|
}
|
|
|
|
/* keysetup_v1.c */
|
|
|
|
void fscrypt_put_direct_key(struct fscrypt_direct_key *dk);
|
|
|
|
int fscrypt_setup_v1_file_key(struct fscrypt_inode_info *ci,
|
|
const u8 *raw_master_key);
|
|
|
|
int fscrypt_setup_v1_file_key_via_subscribed_keyrings(
|
|
struct fscrypt_inode_info *ci);
|
|
|
|
/* policy.c */
|
|
|
|
bool fscrypt_policies_equal(const union fscrypt_policy *policy1,
|
|
const union fscrypt_policy *policy2);
|
|
int fscrypt_policy_to_key_spec(const union fscrypt_policy *policy,
|
|
struct fscrypt_key_specifier *key_spec);
|
|
const union fscrypt_policy *fscrypt_get_dummy_policy(struct super_block *sb);
|
|
bool fscrypt_supported_policy(const union fscrypt_policy *policy_u,
|
|
const struct inode *inode);
|
|
int fscrypt_policy_from_context(union fscrypt_policy *policy_u,
|
|
const union fscrypt_context *ctx_u,
|
|
int ctx_size);
|
|
const union fscrypt_policy *fscrypt_policy_to_inherit(struct inode *dir);
|
|
|
|
#endif /* _FSCRYPT_PRIVATE_H */
|