This seems to more than fix a performance regression that we
detected on a metadata-allocation microbenchmark.
A few months ago, I improved the metadata cache representation
and changed the metadata allocation scheme to primarily use malloc.
Previously, we'd been using malloc in the concurrent tree data
structure but a per-cache slab allocator for the metadata itself.
At the time, I was concerned about the overhead of per-cache
allocators, since many metadata patterns see only a small number
of instantiations. That's still an important factor, so in the
new scheme we're using a global allocator; but instead of using
malloc for individual allocations, we're using a slab allocator,
which should have better peak, single-thread performance, at the
cost of not easily supporting deallocation. Deallocation is
only used for metadata when there's contention on the cache, and
specifically only when there's contention for the same key, so
leaking a little isn't the worst thing in the world.
The initial slab is a 64K globally-allocated buffer.
Successive slabs are 16K and allocated with malloc.
rdar://28189496
Combine the "next" and "next-upstream" checkout schemes since the new
approach for maintaining master-next is to building against the
"upstream-with-swift" Clang/LLVM branches. We do not plan to use the
"stable-next" branches anymore. (We can delete those branches after
this transition has had time to settle out.) I have also merged the
"upstream" scheme with "next-upstream" because building the combination
of branches previously in the scheme is not something we expect to work
now.
Some cmark CMake stuff changed recently and the default rules
we have in the Python build script code doesn't behave correctly
anymore, likely because it was relying on incorrect settings.
Now, by default, if the cmark build type isn't specified, it will
follow Swift's. If we don't do this, Xcode builds are broken, and
building with Xcode is important.
Reworks the update_all_repositories() and update_single_repository()
functions to benefit from more per-repo parallelism.
* Branch-scheme search loop is relocated from update_repository_to_scheme()
to update_all_repositories().
* Functions update_repository_to_tag() and update_repository_to_scheme()
are removed.
* Per-repo work of these functions is shifted to update_single_repository().
* Repeated work (repeated pushd's, etc.) has been eliminated.
* As much work as possible is performed within update_single_repository()'s
outer try clause to catch more errors.
* Arguments passed to update_single_repository() have been changed
appropriately.
* New helper functions confirm_tag_in_repo() and get_branch_for_repo() added
to keep update_single_repository()'s length under control.
* Unnecessary setting of the cross_repo flag by the --tag argument has been
removed, so repos which lack the tag are now properly rebased when updated.
* utils/update-checkout: Fix for mishandling detached HEAD in SR-3854
In update_single_repository, before doing the rebase step, check whether
the current HEAD is a "detached HEAD", normally the result of checking
out a tag by previously invoking update-checkout with the --tag option.
If HEAD is a detached HEAD, skip the rebase (and print an explanatory
message), because rebasing would do the wrong thing and make a mess.
* utils/update-checkout: more robust error handling for SR-3854 fix
We now examine the exception raised by the "git symbolic-ref -q HEAD"
command to check whether the exception was indeed due to a detached HEAD.
If there was an actual error in the command, we re-raise the exception
so that it can be handled like errors from other commands.
This should help speed up people trying to compile the standard library and do
SILGen work. *NOTE* This will not necessarily result in a type checker that is
as fast as a release build since most likely the type checker will use some
link_once odr functions that are debug. But it should still be significantly
faster otherwise.
This makes getting to SILGen take 16 seconds on my machine instead of forever
when compiling with everything else in the compiler in debug mode.
utils/update-checkout: Add a call to ``git rebase --abort`` when using the ``--clean`` option.
Calling ``git fetch`` updates .git/FETCH_HEAD and marks all branches
besides the currently checked out branch as not-for-merge. So tot
ensure that the branch we want to merge is available for merging, we
must do a checkout of that branch before updating the FETCH_HEAD file.
We also must ensure that we have fetched before optionally resetting to remote.
Fixes https://bugs.swift.org/browse/SR-3800.
The --clean option is meant to restore your repository to a pristine
condition, but there are cases where you can ``reset --hard`` and still be
in the middle of a rebase. This is annoying across 10+ repositories, so
if the user wants to --clean, then abort all rebases in progress.
This seems to more than fix a performance regression that we
detected on a metadata-allocation microbenchmark.
A few months ago, I improved the metadata cache representation
and changed the metadata allocation scheme to primarily use malloc.
Previously, we'd been using malloc in the concurrent tree data
structure but a per-cache slab allocator for the metadata itself.
At the time, I was concerned about the overhead of per-cache
allocators, since many metadata patterns see only a small number
of instantiations. That's still an important factor, so in the
new scheme we're using a global allocator; but instead of using
malloc for individual allocations, we're using a slab allocator,
which should have better peak, single-thread performance, at the
cost of not easily supporting deallocation. Deallocation is
only used for metadata when there's contention on the cache, and
specifically only when there's contention for the same key, so
leaking a little isn't the worst thing in the world.
The initial slab is a 64K globally-allocated buffer.
Successive slabs are 16K and allocated with malloc.
rdar://28189496
Adds a three second delay when the `-c` or `--clean` options are
given to utils/build-script, providing a chance to cancel the build
before the existing build folder is removed.
