The `-Winvalid-offsetof` warning is valid in this case. `offsetof` is
being applied to types with a non-standard layout. The layout of this
type is undefined by the specification. There is no guarantee that the
type layout is uniform across all ABIs. It is not possible to portably
compute the offset statically, especially efficiently.
Sink this check into a unit test to avoid performing this test at
runtime. In order to do this in the standard library, we would need to
do this check through a global constructor.
The descriptor and arguments for prespecialized metadata will always be in the shared cache. Skip creating the mangling for any lookup involving pointers outside the shared cache, as an optimization.
Create a version of the metadata specialization code which is abstracted so that it can work in different contexts, such as building specialized metadata from dylibs on disk rather than from inside a running process.
The GenericMetadataBuilder class is templatized on a ReaderWriter. The ReaderWriter abstracts out everything that's different between in-process and external construction of this data. Instead of reading and writing pointers directly, the builder calls the ReaderWriter to resolve and write pointers. The ReaderWriter also handles symbol lookups and looking up other Swift types by name.
This is accompanied by a simple implementation of the ReaderWriter which works in-process. The abstracted calls to resolve and write pointers are implemented using standard pointer dereferencing.
A new SWIFT_DEBUG_VALIDATE_EXTERNAL_GENERIC_METADATA_BUILDER environment variable uses the in-process ReaderWriter to validate the builder by running it in parallel with the existing metadata builder code in the runtime. When enabled, the GenericMetadataBuilder is used to build a second copy of metadata built by the runtime, and the two are compared to ensure that they match. When this environment variable is not set, the new builder code is inactive.
The builder is incomplete, and this initial version only works on structs. Any unsupported type produces an error, and skips the validation.
rdar://116592420
Experiment shows that the vast majority (~99.99%) of keys with equal hashes have equal bytes. Fast-path that case by doing a memcmp before doing the deeper comparison.
rdar://110478978
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
ReflectionContext::allocationMetadataPointer() was reading the
metadata pointer from a wrong offset because of the out-of-sync struct
layouts and dump-generic-metadata was not working correctly.
This change resync's the layouts and adds a static_assert to verify
that the offsets match between GenericMetadataCacheEntry and
GenericCacheEntry.
This replaces a number of `#include`-s like this:
```
#include "../../../stdlib/public/SwiftShims/Visibility.h"
```
with this:
```
#include "swift/shims/Visibility.h"
```
This is needed to allow SwiftCompilerSources to use C++ headers which include SwiftShims headers. Currently trying to do that results in errors:
```
swift/swift/include/swift/Demangling/../../../stdlib/public/SwiftShims/module.modulemap:1:8: error: redefinition of module 'SwiftShims'
module SwiftShims {
^
Builds.noindex/swift/swift/bootstrapping0/lib/swift/shims/module.modulemap:1:8: note: previously defined here
module SwiftShims {
^
```
This happens because the headers in both the source dir and the build dir refer to SwiftShims headers by relative path, and both the source root and the build root contain SwiftShims headers (which are equivalent, but since they are located in different dirs, Clang treats them as different modules).
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
Tidy up the metadata definitions.
* Generalize a number of metadata kinds for out-of-process clients
* Introduce conveniences to make runtime lookups easier
* Introduce TargetExistentialTypeExpression to TrailingObjects stops complaining about OverloadTokens being ambiguous
Note that there is no impact on the layout of the metadata - the changes here are all ABI-compatible.
Not all targets have a 16-byte type alignment guarantee. For the types
which are not naturally aligned, provide a type specific `operator new`
overload to ensure that we are properly aligning the type on allocation
as we run the risk of under-aligned allocations otherwise.
This should no longer be needed with C++17 and newer which do a two
phase `operator new` lookup preferring
`operator new(std::size, std::align_val_t)` if needed. The base type
would be fully pre-processed away. The empty base class optimization
should help ensure that we do not pay any extra size costs for the
alignment fixes.
As we are a C++14 codebase, we must locally implement some of the
standard type_traits utilities, namely `void_t`. We take the minimal
definition here, assuming that the compiler is up-to-date with C++14 DR
reports which fixed an issue in SFINAE. We use the SFINAE for detecting
the presence of the `operator new` overload to guide the over-alignment,
which is inherited through the new `swift::overaligned_type<>` base
type.
Annotate the known classes which request explicit alignment which is
non-pointer alignment. This list was identified by
`git grep ' alignas(.*) '`.
The current system is based on MetadataCompletionQueueEntry
objects which are allocated and then enqueued on dependencies.
Blocking is achieved using a condition variable associated
with the lock on the appropriate metadata cache. Condition
variables are inherently susceptible to priority inversions
because the waiting threads have no dynamic knowledge of
which thread will notify the condition. In the current system,
threads that unblock dependencies synchronously advance their
dependent metadata completions, which means the signaling
thread is unreliable even if we could represent it in condition
variables. As a result, the current system is wholly unsuited
for eliminating these priority inversions.
An AtomicWaitQueue is an object containing a lock. The queue
is eagerly allocated, and the lock is held, whenever a thread
is doing work that other threads might wish to block on. In
the metadata completion system, this means whenever we construct
a metadata cache entry and the metadata isn't already allocated
and transitively complete after said construction. Blocking
is done by safely acquiring a shared reference to the queue
object (which, in the current implementation, requires briefly
taking a lock that's global to the surrounding metadata cache)
and then acquiring the contained lock. For typical lock
implementations, this avoids priority inversions by temporarily
propagating the priority of waiting threads to the locking
threads.
Dependencies are unblocked by simply releasing the lock held
in the queue. The unblocking thread doesn't know exactly what
metadata are blocked on it and doesn't make any effort to
directly advance their completion; instead, the blocking
thread will wake up and then attempt to advance the dependent
metadata completion itself, eliminating a source of priority
overhang that affected the old system. Successive rounds of
unblocking (e.g. when a metadata makes partial progress but
isn't yet complete) can be achieved by creating a new queue
and unlocking the old one. We can still record dependencies
and use them to dynamically diagnose metadata cycles.
The new system allocates more eagerly than the old one.
Formerly, metadata completions which were never blocked never
needed to allocate a MetadataCompletionQueueEntry; we were
then unable to actually deallocate those entries once they
were allocated. The new system will allocate a queue for
most metadata completions, although, on the positive side,
we can reliably deallocate these queues. Cache entries are
also now slightly smaller because some of the excess storage
for status has been folded into the queue.
The fast path of an actual read of the metadata remains a
simple load-acquire. Slow paths may require a bit more
locking. On Darwin, the metadata cache lock can now use
os_unfair_lock instead of pthread_mutex_t (which is a massive
improvement) because it does not need to support associated
condition variables.
The excess locking could be eliminated with some sort of
generational scheme. Sadly, those are not portable, and I
didn't want to take it on up-front.
rdar://76127798
os_unfair_lock is much smaller than pthread_mutex_t (4 bytes versus 64) and a bit faster.
However, it doesn't support condition variables. Most of our uses of Mutex don't use condition variables, but a few do. Introduce ConditionMutex and StaticConditionMutex, which allow condition variables and continue to use pthread_mutex_t.
On all other platforms, we continue to use the same backing mutex type for both Mutex and ConditionMutex.
rdar://problem/45412121
* [Runtime] Switch MetadataCache to ConcurrentReadableHashMap.
Use StableAddressConcurrentReadableHashMap. MetadataCacheEntry's methods for awaiting a particular state assume a stable address, where it will repeatedly examine `this` in a loop while waiting on a condition variable, so we give it a stable address to accommodate that. Some of these caches may be able to tolerate unstable addresses if this code is changed to perform the necessary table lookup each time through the loop instead. Some of them store metadata inline and we assume metadata never moves, so they'll have to stay this way.
* Have StableAddressConcurrentReadableHashMap remember the last found entry and check that before doing a more expensive lookup.
* Make a SmallMutex type that stores the mutex data out of line, and use it to get LockingConcurrentMapStorage to fit into the available space on 32-bit.
rdar://problem/70220660
This gives us faster lookups and a small advantage in memory usage. Most of these maps need stable addresses for their entries, so we add a level of indirection to ConcurrentReadableHashMap for these cases to accommodate that. This costs some extra memory, but it's still a net win.
A new StableAddressConcurrentReadableHashMap type handles this indirection and adds a convenience getOrInsert to take advantage of it.
ConcurrentReadableHashMap is tweaked to avoid any global constructors or destructors when using it as a global variable.
ForeignWitnessTables does not need stable addresses and it now uses ConcurrentReadableHashMap directly.
rdar://problem/70056398
The new function swift_compareProtocolConformanceDescriptors calls
through to the preexisting code in MetadataCacheKey which has been
extracted out from MetadataCacheKey::compareWitnessTables into a new
public static function
MetadataCacheKey::compareProtocolConformanceDescriptors.
The new function's availability is "future" for now.
This removes the last reference to the `llvm::` namespace in the
standard library. All uses of the LLVMSupport library now are
namespaced into the `__swift::__runtime` namespace. This allows us to
incrementally vend the LLVMSupport library and make the separation
explicit.
This adds a new copy of LLVMSupport into the runtime. This is the final
step before changing the inline namespace for the runtime support. This
will allow us to avoid the ODR violations from the header definitions of
LLVMSupport.
LLVMSupport forked at: 22492eead218ec91d349c8c50439880fbeacf2b7
Changes made to LLVMSupport from that revision:
process.inc forward declares `_beginthreadex` due to compilation issues due to custom flag handling
API changes required that we alter the `Deallocate` routine to account
for the alignment.
This is a temporary state, meant to simplify the process. We do not use
the entire LLVMSupport library and there is no value in keeping the
entire library. Subsequent commits will prune the library to the needs
for the runtime.
This was identified by UBSAN: signed-integer-overflow. Explicitly mark
the value as unsigned to ensure that the value does not overflow. There
is a second instance of a raw literal, however, because it is a `*=` the
value is implicitly understood to be unsigned.
Rather than scanning through the generic parameters and generic requirements
each time we form a key for the generic metadata cache, compute these
values once, when the cache itself is first initialized.
