mirror of
https://github.com/git/git.git
synced 2025-12-12 20:36:24 +01:00
The reference updates performed as a part of 'git-receive-pack(1)', take
place one at a time. For each reference update, a new transaction is
created and committed. This is necessary to ensure we can allow
individual updates to fail without failing the entire command. The
command also supports an 'atomic' mode, which uses a single transaction
to update all of the references. But this mode has an all-or-nothing
approach, where if a single update fails, all updates would fail.
In 23fc8e4f61 (refs: implement batch reference update support,
2025-04-08), we introduced a new mechanism to batch reference updates.
Under the hood, this uses a single transaction to perform a batch of
reference updates, while allowing only individual updates to fail.
Utilize this newly introduced batch update mechanism in
'git-receive-pack(1)'. This provides a significant bump in performance,
especially when dealing with repositories with large number of
references.
With the reftable backend there is a 18x performance improvement, when
performing receive-pack with 10000 refs:
Benchmark 1: receive: many refs (refformat = reftable, refcount = 10000, revision = master)
Time (mean ± σ): 4.276 s ± 0.078 s [User: 0.796 s, System: 3.318 s]
Range (min … max): 4.185 s … 4.430 s 10 runs
Benchmark 2: receive: many refs (refformat = reftable, refcount = 10000, revision = HEAD)
Time (mean ± σ): 235.4 ms ± 6.9 ms [User: 75.4 ms, System: 157.3 ms]
Range (min … max): 228.5 ms … 254.2 ms 11 runs
Summary
receive: many refs (refformat = reftable, refcount = 10000, revision = HEAD) ran
18.16 ± 0.63 times faster than receive: many refs (refformat = reftable, refcount = 10000, revision = master)
In similar conditions, the files backend sees a 1.21x performance
improvement:
Benchmark 1: receive: many refs (refformat = files, refcount = 10000, revision = master)
Time (mean ± σ): 1.121 s ± 0.021 s [User: 0.128 s, System: 0.975 s]
Range (min … max): 1.097 s … 1.156 s 10 runs
Benchmark 2: receive: many refs (refformat = files, refcount = 10000, revision = HEAD)
Time (mean ± σ): 927.9 ms ± 22.6 ms [User: 99.0 ms, System: 815.2 ms]
Range (min … max): 903.1 ms … 978.0 ms 10 runs
Summary
receive: many refs (refformat = files, refcount = 10000, revision = HEAD) ran
1.21 ± 0.04 times faster than receive: many refs (refformat = files, refcount = 10000, revision = master)
As using batched updates requires the error handling to be moved to the
end of the flow, create and use a 'struct strset' to track the failed
refs and attribute the correct errors to them.
This change also uncovers an issue when a client provides multiple
updates to the same reference. For example:
$ git send-pack remote.git A:foo B:foo
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 20 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 226 bytes | 226.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)
remote: error: cannot lock ref 'refs/heads/foo': reference already exists
To remote.git
! [remote rejected] A -> foo (failed to update ref)
! [remote failure] B -> foo (remote failed to report status)
As you can see, the remote runs into an error because it cannot lock the
target reference for the second update. Furthermore, the remote complains
that the first update has been rejected whereas the second update didn't
receive any status update because we failed to lock it. Reading this status
message alone a user would probably expect that `foo` has not been updated
at all. But that's not the case: while we claim that the ref wasn't updated,
it surprisingly points to `A` now.
One could argue that this is merely an error in how we report the result of
this push. But ultimately, the user's request itself is already broken and
doesn't make any sense in the first place and cannot ever lead to a sensible
outcome that honors the full request.
The conversion to batched transactions fixes the issue because we now try to
queue both updates in the same transaction. As such, the transaction itself
will notice this conflict and refuse the update altogether before we commit
any of the values.
Note that this requires changes to a couple of tests in t5408 that happened
to exercise this behaviour. Given that the generated output is misleading
and given that the user request cannot ever be fully honored this really
feels more like a bug than properly designed behaviour. As such, changing
the behaviour feels like the right thing to do.
Since now reference updates are batched, the 'reference-transaction'
hook will be invoked with all updates together. Currently git will 'die'
when the hook returns with a non-zero exit status in the 'prepared'
stage. For 'git-receive-pack(1)', this allowed users to reject an
individual reference update, git would have applied previous updates but
immediately abort further execution. This is definitely an incorrect
usage of this hook, since the right place to do this would be the
'update' hook. This patch retains the latter behavior, but
'reference-transaction' hook now changes to a all-or-nothing behavior
when a non-zero exit status is returned in the 'prepared' stage, since
batch updates use a transaction under the hood. This explains the change
in 't1416'.
Helped-by: Jeff King <peff@peff.net>
Helped-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
210 lines
4.8 KiB
Bash
Executable File
210 lines
4.8 KiB
Bash
Executable File
#!/bin/sh
|
|
|
|
test_description='reference transaction hooks'
|
|
|
|
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
|
|
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
|
|
|
|
. ./test-lib.sh
|
|
|
|
test_expect_success setup '
|
|
test_commit PRE &&
|
|
PRE_OID=$(git rev-parse PRE) &&
|
|
test_commit POST &&
|
|
POST_OID=$(git rev-parse POST)
|
|
'
|
|
|
|
test_expect_success 'hook allows updating ref if successful' '
|
|
git reset --hard PRE &&
|
|
test_hook reference-transaction <<-\EOF &&
|
|
echo "$*" >>actual
|
|
EOF
|
|
cat >expect <<-EOF &&
|
|
prepared
|
|
committed
|
|
EOF
|
|
git update-ref HEAD POST &&
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_expect_success 'hook aborts updating ref in prepared state' '
|
|
git reset --hard PRE &&
|
|
test_hook reference-transaction <<-\EOF &&
|
|
if test "$1" = prepared
|
|
then
|
|
exit 1
|
|
fi
|
|
EOF
|
|
test_must_fail git update-ref HEAD POST 2>err &&
|
|
test_grep "ref updates aborted by hook" err
|
|
'
|
|
|
|
test_expect_success 'hook gets all queued updates in prepared state' '
|
|
test_when_finished "rm actual" &&
|
|
git reset --hard PRE &&
|
|
test_hook reference-transaction <<-\EOF &&
|
|
if test "$1" = prepared
|
|
then
|
|
while read -r line
|
|
do
|
|
printf "%s\n" "$line"
|
|
done >actual
|
|
fi
|
|
EOF
|
|
cat >expect <<-EOF &&
|
|
$ZERO_OID $POST_OID refs/heads/main
|
|
EOF
|
|
git update-ref HEAD POST <<-EOF &&
|
|
update HEAD $ZERO_OID $POST_OID
|
|
update refs/heads/main $ZERO_OID $POST_OID
|
|
EOF
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_expect_success 'hook gets all queued updates in committed state' '
|
|
test_when_finished "rm actual" &&
|
|
git reset --hard PRE &&
|
|
test_hook reference-transaction <<-\EOF &&
|
|
if test "$1" = committed
|
|
then
|
|
while read -r line
|
|
do
|
|
printf "%s\n" "$line"
|
|
done >actual
|
|
fi
|
|
EOF
|
|
cat >expect <<-EOF &&
|
|
$ZERO_OID $POST_OID refs/heads/main
|
|
EOF
|
|
git update-ref HEAD POST &&
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_expect_success 'hook gets all queued updates in aborted state' '
|
|
test_when_finished "rm actual" &&
|
|
git reset --hard PRE &&
|
|
test_hook reference-transaction <<-\EOF &&
|
|
if test "$1" = aborted
|
|
then
|
|
while read -r line
|
|
do
|
|
printf "%s\n" "$line"
|
|
done >actual
|
|
fi
|
|
EOF
|
|
cat >expect <<-EOF &&
|
|
$ZERO_OID $POST_OID HEAD
|
|
$ZERO_OID $POST_OID refs/heads/main
|
|
EOF
|
|
git update-ref --stdin <<-EOF &&
|
|
start
|
|
update HEAD POST $ZERO_OID
|
|
update refs/heads/main POST $ZERO_OID
|
|
abort
|
|
EOF
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_expect_success 'interleaving hook calls succeed' '
|
|
test_when_finished "rm -r target-repo.git" &&
|
|
|
|
git init --bare target-repo.git &&
|
|
|
|
test_hook -C target-repo.git reference-transaction <<-\EOF &&
|
|
echo $0 "$@" >>actual
|
|
EOF
|
|
|
|
test_hook -C target-repo.git update <<-\EOF &&
|
|
echo $0 "$@" >>actual
|
|
EOF
|
|
|
|
cat >expect <<-EOF &&
|
|
hooks/update refs/tags/PRE $ZERO_OID $PRE_OID
|
|
hooks/update refs/tags/POST $ZERO_OID $POST_OID
|
|
hooks/reference-transaction prepared
|
|
hooks/reference-transaction committed
|
|
EOF
|
|
|
|
git push ./target-repo.git PRE POST &&
|
|
test_cmp expect target-repo.git/actual
|
|
'
|
|
|
|
test_expect_success 'hook captures git-symbolic-ref updates' '
|
|
test_when_finished "rm actual" &&
|
|
|
|
test_hook reference-transaction <<-\EOF &&
|
|
echo "$*" >>actual
|
|
while read -r line
|
|
do
|
|
printf "%s\n" "$line"
|
|
done >>actual
|
|
EOF
|
|
|
|
git symbolic-ref refs/heads/symref refs/heads/main &&
|
|
|
|
cat >expect <<-EOF &&
|
|
prepared
|
|
$ZERO_OID ref:refs/heads/main refs/heads/symref
|
|
committed
|
|
$ZERO_OID ref:refs/heads/main refs/heads/symref
|
|
EOF
|
|
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_expect_success 'hook gets all queued symref updates' '
|
|
test_when_finished "rm actual" &&
|
|
|
|
git update-ref refs/heads/branch $POST_OID &&
|
|
git symbolic-ref refs/heads/symref refs/heads/main &&
|
|
git symbolic-ref refs/heads/symrefd refs/heads/main &&
|
|
git symbolic-ref refs/heads/symrefu refs/heads/main &&
|
|
|
|
test_hook reference-transaction <<-\EOF &&
|
|
echo "$*" >>actual
|
|
while read -r line
|
|
do
|
|
printf "%s\n" "$line"
|
|
done >>actual
|
|
EOF
|
|
|
|
# In the files backend, "delete" also triggers an additional transaction
|
|
# update on the packed-refs backend, which constitutes additional reflog
|
|
# entries.
|
|
if test_have_prereq REFFILES
|
|
then
|
|
cat >expect <<-EOF
|
|
aborted
|
|
$ZERO_OID $ZERO_OID refs/heads/symrefd
|
|
EOF
|
|
else
|
|
>expect
|
|
fi &&
|
|
|
|
cat >>expect <<-EOF &&
|
|
prepared
|
|
ref:refs/heads/main $ZERO_OID refs/heads/symref
|
|
ref:refs/heads/main $ZERO_OID refs/heads/symrefd
|
|
$ZERO_OID ref:refs/heads/main refs/heads/symrefc
|
|
ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
|
|
committed
|
|
ref:refs/heads/main $ZERO_OID refs/heads/symref
|
|
ref:refs/heads/main $ZERO_OID refs/heads/symrefd
|
|
$ZERO_OID ref:refs/heads/main refs/heads/symrefc
|
|
ref:refs/heads/main ref:refs/heads/branch refs/heads/symrefu
|
|
EOF
|
|
|
|
git update-ref --no-deref --stdin <<-EOF &&
|
|
start
|
|
symref-verify refs/heads/symref refs/heads/main
|
|
symref-delete refs/heads/symrefd refs/heads/main
|
|
symref-create refs/heads/symrefc refs/heads/main
|
|
symref-update refs/heads/symrefu refs/heads/branch ref refs/heads/main
|
|
prepare
|
|
commit
|
|
EOF
|
|
test_cmp expect actual
|
|
'
|
|
|
|
test_done
|