mirror of
https://github.com/git/git.git
synced 2026-10-06 23:36:05 +02:00
Patrick Steinhardt
4f44ef12de
Originally, when designing pluggable object databases the goal was that
the object database can have multiple sources, and every source attached
to it could use a different backend. This would have allowed for quite a
lot of flexibility, as you could trivially mix and match different kinds
of object storages in whatever way you like.
But while well-intentioned, this design led to a bunch of conceptual
problems:
- We're now trying to read objects in source order, whereas we
previously tried to read objects via packfiles before trying to read
them via loose objects. This led to a performance regression when
using alternates or when using a quarantine directory.
- Some data structures are supposed to only ever exist once, like for
example bitmaps and commit graphs. At the same time, those data
structures also span across the union of all objects, so they may
cross sources.
- It is unclear how we can extend GIT_OBJECT_DIRECTORY or
GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
backwards-compatible way. In general, introducing an object storage
extension into the current status quo where alternates may have to
be extended to become generic was proving to be painful.
- Some mechanisms of alternates assume way too much about how exactly
their backends work. Alternate refs for example assume that the
alternate is backed by a filesystem path, and that this filesystem
path may also allow us to read references. This is not a given
though, as backends may not even have local data at all.
In short, there are a bunch of conceptual mismatches when we have
alternates and pluggable object databases coexist. So while the original
idea was nice, it does not result in a system that is easy to reason
about.
Correct course by moving alternates into the "files" source itself so
that it becomes an implementation detail thereof so that we can avoid
all of these shortcomings. While it's unfortunate that we cannot easily
mix and match sources now, that ability doesn't go away. It's still very
much feasible to introduce a new backend that allows for exactly that
use case, and such a backend may also be a lot more flexible as we can
now add new logic to determine which objects should be stored where. So
the original motivation for having per-source backends can still be
realized with the new architecture.
Note that as part of this move, we also handle the GIT_OBJECT_DIRECTORY
and GIT_ALTERNATE_OBJECT_DIRECTORIES environment variables in the
"files" backend. This may be surprising at first, but object directories
are very much a concept of that backend, too. So these variables would
have bad interactions with other backends, and they create a bit of a
mismatch with the eventual object storage extension that we plan to
introduce.
Note that this commit is way larger than I'd like it to be. I'm sorry, I
couldn't find a way to split it up further. That being said, most of the
changes are straight-forward conversions that go from iterating over
sources to iterating over object directories. The more involved changes
are moving the infrastructure to track individual sources from "odb.c"
and moving them into "odb/source-files.c".
This also serves to show a bit of a who's-who of commands that don't
work properly with pluggable object databases. Almost all of these
commands are related to housekeeping though -- some of them will be
converted eventually, like for example commit graphs or MIDXs. But many
of them will stay incompatible going forward as they are simply too
specific to the "files" backend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
76 lines
1.6 KiB
C
76 lines
1.6 KiB
C
#include "git-compat-util.h"
|
|
#include "gettext.h"
|
|
#include "odb/source.h"
|
|
#include "odb/transaction.h"
|
|
|
|
int odb_transaction_begin(struct object_database *odb,
|
|
struct odb_transaction **out,
|
|
enum odb_transaction_flags flags)
|
|
{
|
|
int ret;
|
|
|
|
if (odb->transaction)
|
|
return error(_("object database transaction already pending"));
|
|
|
|
ret = odb_source_begin_transaction(odb->source, out, flags);
|
|
if (!ret)
|
|
odb->transaction = *out;
|
|
|
|
return ret;
|
|
}
|
|
|
|
int odb_transaction_commit(struct odb_transaction *transaction)
|
|
{
|
|
int ret;
|
|
|
|
if (!transaction)
|
|
return 0;
|
|
|
|
/*
|
|
* Ensure the transaction ending matches the pending transaction.
|
|
*/
|
|
ASSERT(transaction == transaction->source->odb->transaction);
|
|
|
|
ret = transaction->commit(transaction);
|
|
transaction->source->odb->transaction = NULL;
|
|
|
|
return ret;
|
|
}
|
|
|
|
int odb_transaction_finalize(struct odb_transaction *transaction)
|
|
{
|
|
int ret = 0;
|
|
|
|
if (!transaction)
|
|
return 0;
|
|
|
|
if (transaction->finalize)
|
|
ret = transaction->finalize(transaction);
|
|
|
|
free(transaction);
|
|
|
|
return ret;
|
|
}
|
|
|
|
int odb_transaction_write_object_stream(struct odb_transaction *transaction,
|
|
struct odb_stream *stream,
|
|
struct object_id *oid)
|
|
{
|
|
return transaction->write_object_stream(transaction, stream, oid);
|
|
}
|
|
|
|
int odb_transaction_write_pack(struct odb_transaction *transaction, int pack_fd,
|
|
struct strbuf *err_msg,
|
|
const struct odb_transaction_write_pack_opts *opts)
|
|
{
|
|
return transaction->write_pack(transaction, pack_fd, err_msg, opts);
|
|
}
|
|
|
|
int odb_transaction_env(struct odb_transaction *transaction, struct strvec *env)
|
|
{
|
|
if (!transaction)
|
|
return 0;
|
|
|
|
return transaction->env(transaction, env);
|
|
}
|