Conformance checks now walk the superclass chain in two stages: stage 1 only walks superclasses that have already been instantiated. When there's a negative result and there's an uninstantiated superclass, then stage 2 will walk the uninstantiated superclasses.
The infinite recursion would occur in this scenario:
class Super<T: P> {}
class Sub: Super<Sub>, P {}
Instantiating the metadata for Super requires looking up the conformance for Sub: P. Conformance checking for Sub would instantiate the metadata for Super to check for a Super: P conformance.
The compiler does not allow the conformance to come from a superclass in this situation. This does not compile:
class Super<T: P>: P {}
class Sub: Super<Sub> {}
Therefore it's not necessary to look at Super when finding the conformance for Sub: P in this particular case. The trick is knowing when to skip Super.
We do need to instantiate Super in the general case, otherwise we can get false negatives. This was addressed in a80fe8536b, which walks the full superclass chain during conformance checks, even if the superclass has not yet been instantiated. Unfortunately, that causes this infinite recursion.
This fix modifies that fix to make superclass instantiation conditional. The result is the ability to choose between the old algorithm (which skipped uninstantiated superclasses, albeit somewhat by accident) and the new one. A small wrapper then runs the check with the old algorithm, and then only if the old algorithm fails and there is an uninstantiated superclass, it runs with the new one.
Uninstantiated superclasses are uncommon and transient (you only run into this while metadata is in the process of being constructed) so 99.9999% of the time we'll just run the first stage and be done, and performance should remain the same as before.
rdar://80532245
The getSuperclassForMaybeIncompleteMetadata function assumes Swift metadata, and can malfunction strangely when given a pure ObjC class. This is rare, as we usually get the Swift wrapper metadata instead, but possible when using calls like objc_copyClassList.
Fix this by checking for isTypeMetadata before doing anything that assumes Swift-metadata-ness.
rdar://80336030
tryGetCompleteMetadataNonblocking crashes on artificial subclasses due to the NULL type descriptor. Explicitly check for artificial subclasses in getSuperclassForMaybeIncompleteMetadata and immediately return their Superclass field. Artificial subclasses are always fully initialized so we don't need to do anything special for them.
rdar://72583931
When we rely on a protocol conformance, and the type in question has multiple
conformances to that protocol in its inheritance chain, emit a runtime warning.
It's quite tricky to cause this problem - you need a type in one dylib that is
extended to conform to a protocol in another dylib, subclassed in another module
and then some subclass has protocol conformance added as well. If they're in
the same module, the compiler will give an error and prevent the problem
completely.
rdar://73364629
When constructing generic type metadata, we can end up checking protocol conformances for metadata that isn't fully built, and in particular the superclass field may not be set up yet. Handle this case properly by falling back to looking up the superclass by mangled name from the nominal type descriptor when necessary.
Previously we were just chasing Superclass fields, which would cause us to terminate the search early sometimes when concurrently looking up the same type from multiple threads. When a protocol conformance is on a superclass, this causes the search to fail incorrectly, causing type lookup failures.
This only arises on the very first attempt to instantiate a given type, since the lookup will be cached (and all metadata fully initialized) after the first success.
rdar://72583931
Take the existing CompatibilityOverride mechanism and generalize it so it can be used in both the runtime and Concurrency libraries. The mechanism is preprocessor-heavy, so this requires some tricks. Use the SWIFT_TARGET_LIBRARY_NAME define to distinguish the libraries, and use a different .def file and mach-o section name accordingly.
We want the global/main executor functions to be a little more flexible. Instead of using the override mechanism, we expose function pointers that can be set by the compatibility library, or by any other code that wants to use a custom implementation.
rdar://73726764
Match the logic used in the non-shared-cache case, where we walk up the class hierarchy to find the class where the conformance actually applies. This is important in cases like:
class Super<T> : Proto {}
class Sub: Super<Int> {}
Looking up the conformance `Sub: Proto` without this logic causes it to attempt to find the witness table for `Sub<Int>`, which fails. The correct logic looks up the witness table for `Super<Int>`.
While we're in there, fix a bug in the shared cache validation code (std::find wasn't checked for failure correctly) and add a `SWIFT_DEBUG_ENABLE_SHARED_CACHE_PROTOCOL_CONFORMANCES` environment variable to allow us to turn the shared cache integration off at runtime.
rdar://75431771
When available, take advantage of precomputed protocol conformances in the shared cache on Darwin platforms to accelerate conformance lookups and cut down on memory usage.
We consult the precomputed conformances before doing the runtime's standard conformance lookup. When a conformance is precomputed, this avoids the slow linear scan of all loaded conformances for the first access, and it avoids the memory cost of storing the conformance in the cache.
When the shared cache has no images overridden (the normal case), then we can also skip scanning conformances in the shared cache in all circumstances, because the precomputed conformances will always cover those results. This greatly speeds up the slow linear scan for the initial lookup of anything that doesn't have a conformance in the shared cache, including lookups with conformances in apps or app frameworks, and negative lookups.
A validation mode is available by setting the SWIFT_DEBUG_VALIDATE_SHARED_CACHE_PROTOCOL_CONFORMANCES environment variable. When enabled, results from the precomputed conformances are compared with the results from a slow scan of the conformances in the shared cache. This extremely slow, but good at catching bugs in the system.
When the calls for precomputed conformances are unavailable, the new code is omitted and we remain in the same situation as before, with the runtime performing all lookups by scanning conformance sections in all loaded Swift images.
rdar://71128983
In the uncached case, we'd scan conformances, cache them, then re-query the cache. This worked fine when the cache always grew, but now we clear the cache when loading new Swift images into the process. If that happens between the scan and the re-query, we lose the entry and return a false negative.
Instead, track what we've found in the scan in a separate local table, then query that after completing the scan.
While we're in there, fix a bug in TypeLookupError where operator= accidentally copied this->Context instead of other.Context. This caused the runtime to crash when trying to print error messages due to the false negative.
Add a no-parameter constructor to TypeLookupErrorOr<> to distinguish the case where it's being initialized with nothing from the case where it's being initialized with nullptr.
This gives us build-time warnings about format string mistakes, like we would get if we called the built-in asprintf directly.
Make TypeLookupError's format string constructor a macro instead so that its callers can get these build-time warnings.
This reveals various mistakes in format strings and arguments in the runtime, which are now fixed.
rdar://73417805
to use it.
ConcurrentReadableHashMap is lock-free for readers, with writers using a lock to
ensure mutual exclusion amongst each other. The intent is to eventually replace
all uses ConcurrentMap with ConcurrentReadableHashMap.
ConcurrentReadableHashMap provides for relatively quick lookups by using a hash
table. Rearders perform an atomic increment/decrement in order to inform writers
that there are active readers. The design attempts to minimize wasted memory by
storing the actual elements out-of-line, and having the table store indices into
a separate array of elements.
The protocol conformance cache now uses ConcurrentReadableHashMap, which
provides faster lookups and less memory use than the previous ConcurrentMap
implementation. The previous implementation caches
ProtocolConformanceDescriptors and extracts the WitnessTable after the cache
lookup. The new implementation directly caches the WitnessTable, removing an
extra step (potentially a quite slow one) from the fast path.
The previous implementation used a generational scheme to detect when negative
cache entries became obsolete due to new dynamic libraries being loaded, and
update them in place. The new implementation just clears the entire cache when
libraries are loaded, greatly simplifying the code and saving the memory needed
to track the current generation in each negative cache entry. This means we need
to re-cache all requested conformances after loading a dynamic library, but
loading libraries at runtime is rare and slow anyway.
rdar://problem/67268325
Previously, when an attempt was made to instantiate a generic metadata
whose argument was constrained to subclass a superclass Super with an
existential type E that had a superclass constraint to a subclass Sub of
that same superclass Super, the instantiation would fail at runtime,
despite the fact that the generic instantiation was allowed to
type-check.
The result was a runtime failure to instantiate generic metadata
resulting eventually in a crash.
Here, handling for that situation is added. When checking generic
requirements at runtime, when a superclass requirement is encountered,
an existential type is checked for. If that existential type has a
superclass constraint and if that superclass constraint is a subclass of
the superclass requirement, the check is determined to be satisfactory.
rdar://problem/64672291
This cleans up some more `llvm::` leakage in the runtime when built into
a static library. With this change we are down to 3 leaking symbols in
the static library related to a missed ADT (`StringSwitch`).
Rather than using the forward declaration for the LLVMSupport types,
expect to be able to use the full declaration. Because these are
references in the implementation, there is no reason to use a forward
declaration as the full types need to be declared for use. The LLVM
headers will provide the declaration and definition for the types. This
is motivated by the desire to ensure that the LLVMSupport symbols are
properly namespaced to avoid ODR violations in the runtime.
When constructing the metadata for a type Gen<T : Super>
where Super is a superclass constraint, the generic argument K at which
the metadata for Gen is being instantiated is verified to be a subclass
of Super via _checkGenericRequirements.
Previously, that check was done using swift_dynamicCastMetatype. That
worked for the most part but provided an incorrect answer if the
metadata for K was not yet complete. These classes are incomplete more
often thanks to __swift_instantiateConcreteTypeFromMangledNameAbstract.
That issue occurred concretely in the following case:
Framework with Library Evolution enabled:
open class Super { ... }
public struct Gen<T : Super> {
}
Target in a different resilience domain from that framework:
class Sub : Super {
var gen: Gen<Sub>?
}
Here, the mechanism for checking whether the generic argument K at which
the metadata for Gen is being instantiated handles the case where K's
metadata is incomplete. At worst, every superclass name from super(K)
up to Super are demangled to instantiate metadata. A number of faster
paths are included as well.
rdar://problem/60790020
The ConformanceCandidate constructor would eagerly instantiate metadata for
all non-generic types in the conformance cache. This was not the intention
of the code though, because it can compare nominal type descriptors --
which are emitted statically -- for those types that have them, namely
everything except for foreign and Objective-C classes.
Change the order of two calls so that the lazy path has a chance to run.
This fixes a crash when type metadata uses weakly-linked symbols
which are not available, which can come up in backward deployment
scenarios.
Fixes <rdar://problem/59460603>.
An extension on a class creates a conformance record that's always visible even when that class is not present at runtime. In that case, the type pointer in the conformance record is NULL. The runtime did not like this, and crashed. This fixes it to ignore such records instead.
rdar://problem/54054895
This is a one-to-many cache that's more speculative than the cache mapping mangled names to context descriptors. Entries found in the cache need to be verified for a match before they can be returned. However, this allows scanning conformance records up front and building up the cache in one scan rather than performing an expensive scan of all conformance records every time the mangled name cache misses.
rdar://problem/53560010
When we find a protocol conformance descriptor for a given type, make sure
we adjust to the conforming type of that descriptor (following the superclass
chain as needed) before instantiating the witness table.
Fixes rdar://problem/49741838.
Instead of capturing SubstGenericParametersFromMetadata and SubstGenericParametersFromWrittenArgs by value, capture by reference.
This avoids those instances to be copied and thus avoids a lot of mallocs.
SR-10028
rdar://problem/48575729
This dramatically reduces the number of needed malloc calls.
Unfortunately I had to add the implementation of SmallVectorBase::grow_pod to the runtime, as we don't link LLVM. This is a bad hack, but better than re-inventing a new SmallVector implementation.
SR-10028
rdar://problem/48575729
We would not previously symbolicate the stack trace and as a result
would not display the stack trace. Add symbolication support to the
runtime to actually make use of the captured stack trace. This allows
us to get a stack trace when the standard library or swift code reports
a fatal error.
In our initial approach for resolving metadata dependency cycles with classes, non-transitively complete superclass metadata was fetched by the subclass's metadata completion function and passed to `swift_initClassMetadata`. That could mean generating quite a lot of code in the completion function, and so we fairly recently changed it so that `swift_initClassMetadata` instead fetched the superclass metadata via a demangling. Unfortunately, the metadata demangler only fetches _abstract_ metadata by default, and class metadata cannot be considered even non-transitively complete when its superclass reference not at that stage. If the superclass metadata is being completed on one thread, and a subclass is being completed on another, and the subclass installs the incomplete superclass metadata in its superclass field and attempts to register the subclass with the Objective-C runtime, the runtime may crash reading the incompletely-initialized superclass.
The proper fix is to make `swift_initClassMetadata` fetch non-transitively complete metadata for the superclass, delaying completion if that metadata is unavailable. Unfortunately, that can't actually be implemented on top of `swift_initClassMetadata` because that function has no means of reporting an unsatisfied dependency to its caller, and we can no longer simply change its signature without worrying about a small of internal code that might still be using it. We cannot simply perform a blocking metadata request in `swift_initClassMetadata` because it is deeply problematic to block within a metadata completion function. The solution is therefore to add a `swift_initClassMetadata2` which has the ability to report unsatisfied dependencies. That was done in #22386; this patch builds on that by teaching the compiler to generate code to actually use it. It is therefore not safe to use this patch if you might be running on an OS that only provides the old runtime function, but that should be a temporary Apple-internal problem.
Fixes rdar://47549859.
Note that I've called out a couple of suspicious places where we
are requesting abstract metadata for superclasses but probably
need to be requesting something more complete.
