rhashtable: Add rhashtable_next_key() API

Introduce a simpler iteration mechanism for rhashtable that lets
the caller continue from an arbitrary position by supplying the
previous key, without the per-iterator state of the
rhashtable_walk_* API.

  void *rhashtable_next_key(struct rhashtable *ht,
                            const void *prev_key);

Caller holds RCU; passes NULL prev_key for the first element or
the previously returned key to advance. Walks tbl->future_tbl
chain so in-flight rehashes are observed.

Best-effort: in case of concurrent resize, provides no guarantees:
 - may produce duplicate elements
 - may skip any amount of elements
 - termination of the loop is not guaranteed in case of
 sustained rehash. Callers are advised to bound loop externally
 or avoid inserting new elements during such loop.

Returns ERR_PTR(-ENOENT) if prev_key is not found.
Behavior on tables with duplicate keys is undefined.
rhltable is not supported — returns ERR_PTR(-EOPNOTSUPP).

Signed-off-by: Mykyta Yatsenko <yatsenko@meta.com>
Acked-by: Herbert Xu <herbert@gondor.apana.org.au>
Link: https://lore.kernel.org/r/20260605-rhash-v7-1-5b8e05f8630d@meta.com
Signed-off-by: Alexei Starovoitov <ast@kernel.org>
This commit is contained in:
Mykyta Yatsenko
2026-06-05 04:41:18 -07:00
committed by Alexei Starovoitov
parent bf29346fc3
commit 8f4fa9f89b
2 changed files with 109 additions and 0 deletions
+40
View File
@@ -650,6 +650,46 @@ restart:
return NULL;
}
/**
* rhashtable_next_key - return next element after a given key
* @ht: hash table
* @prev_key: pointer to previous key, or NULL for the first element
*
* WARNING: this walk is highly unstable. Unlike rhashtable_walk_*(),
* it cannot detect a concurrent resize or rehash, so a full iteration
* is NOT guaranteed to terminate under adversarial or sustained
* rehashing. Callers MUST tolerate skipped and duplicated elements and
* SHOULD bound their loop externally.
*
* Returns the next element in best-effort iteration order, walking the
* @tbl chain (including any future_tbl in flight). Caller must hold RCU.
*
* Pass @prev_key == NULL to obtain the first element. To iterate, set
* @prev_key to the key of the previously returned element on each call,
* and stop when NULL is returned.
*
* Best-effort semantics:
* - Across the tbl->future_tbl chain, an element being migrated may
* transiently appear in both tables and be observed twice.
* - Concurrent inserts may or may not be observed.
* - Termination of a full iteration loop is NOT guaranteed under
* adversarial continuous rehash; callers MUST tolerate skips and
* repeats and SHOULD bound their loop externally.
* - Behavior on tables that contain duplicate keys is undefined:
* duplicates may be skipped, repeated, or trap the walk in a
* cycle. Callers requiring duplicate-key iteration must use
* rhashtable_walk_*() instead.
* - rhltable instances are not supported and return
* ERR_PTR(-EOPNOTSUPP).
* - If prev_key was concurrently deleted and is not present in any
* in-flight table, returns ERR_PTR(-ENOENT).
*
* Returns entry of the next element, or NULL when iteration is exhausted,
* or ERR_PTR(-ENOENT) if prev_key is not found, or
* ERR_PTR(-EOPNOTSUPP) if @ht is an rhltable.
*/
void *rhashtable_next_key(struct rhashtable *ht, const void *prev_key);
/**
* rhashtable_lookup - search hash table
* @ht: hash table
+69
View File
@@ -687,6 +687,75 @@ void *rhashtable_insert_slow(struct rhashtable *ht, const void *key,
}
EXPORT_SYMBOL_GPL(rhashtable_insert_slow);
/* Scan one element forward from prev_key's position in @tbl.
* Returns first rhash_head whose bucket > prev_key's bucket, or the
* element immediately after prev_key inside prev_key's bucket.
* Returns the first element if prev_key is NULL, NULL when @tbl is
* exhausted, or ERR_PTR(-ENOENT) if prev_key is not found in @tbl.
*/
static struct rhash_head *__rhashtable_next_in_table(
struct rhashtable *ht, struct bucket_table *tbl,
const void *prev_key)
{
struct rhashtable_compare_arg arg = { .ht = ht, .key = prev_key };
const struct rhashtable_params params = ht->p;
struct rhash_head *he;
unsigned int b = 0;
bool found = false;
if (prev_key) {
b = rht_key_hashfn(ht, tbl, prev_key, params);
rht_for_each_rcu(he, tbl, b) {
bool match = params.obj_cmpfn
? !params.obj_cmpfn(&arg, rht_obj(ht, he))
: !rhashtable_compare(&arg, rht_obj(ht, he));
if (found) {
if (match)
continue;
return he;
}
if (match)
found = true;
}
if (!found)
return ERR_PTR(-ENOENT);
b++;
}
for (; b < tbl->size; b++)
rht_for_each_rcu(he, tbl, b)
return he;
return NULL;
}
/**
* rhashtable_next_key - return next element after a given key
*
* See include/linux/rhashtable.h for the full contract.
*/
void *rhashtable_next_key(struct rhashtable *ht, const void *prev_key)
{
struct bucket_table *tbl;
struct rhash_head *he;
if (unlikely(ht->rhlist))
return ERR_PTR(-EOPNOTSUPP);
tbl = rht_dereference_rcu(ht->tbl, ht);
do {
he = __rhashtable_next_in_table(ht, tbl, prev_key);
if (!IS_ERR_OR_NULL(he))
return rht_obj(ht, he);
if (!he)
prev_key = NULL;
/* See any new future_tbl attached during a rehash. */
smp_rmb();
tbl = rht_dereference_rcu(tbl->future_tbl, ht);
} while (tbl);
return he; /* NULL or -ENOENT */
}
EXPORT_SYMBOL_GPL(rhashtable_next_key);
/**
* rhashtable_walk_enter - Initialise an iterator
* @ht: Table to walk over