In order to be able to debug, for example, a Linux process from a macOS host, we
need to be able to initialize a ReflectionContext without Objective-C
interoperability. This patch turns ObjCInterOp into another template trait, so
it's possible to instantiate a non-ObjC MetadataReader on a system built with
ObjC-interop (but not vice versa).
This patch changes the class hierarchy to
TargetMetadata<Runtime>
|
TargetHeapMetadata<Runtime>
|
TargetAnyClassMetadata<Runtime>
/ \
/ TargetAnyClassMetadataObjCInterop<Runtime>
/ \
TargetClassMetadata<Runtime, TargetAnyClassMetadata<Runtime>> \
\
TargetClassMetadata<Runtime, TargetAnyClassMetadataObjCInterop<Runtime>>
TargetAnyClassMetadataObjCInterop inherits from TargetAnyClassMetadata because
most of the implementation is the same. This choice makes TargetClassMetadata a
bit tricky. In this patch I went with templating the parent class.
rdar://87179578
* Refactor Bincompat
Organize everything around internal functions that test for
a particular OS version.
Correctly handle cases where we don't know the version of the app.
Make all bincompat functions consistently return `true` for the
legacy semantics, `false` for new semantics. Consistently name
them all to reflect this.
* Conditionalize the support for SR-14635
SR-14635 pointed out a hole in the updated dynamic casting logic
that allowed certain casts that should have been illegal.
In particular, when casting certain types to Obj-C protocols,
the Swift value gets boxed; we would permit the cast to succeed
whenever the resulting box satisfied the protocol. For example,
this allowed any Swift value to be cast to `NSCopying` regardless of
whether or not it implemented the required `copy(with:)` method.
This was fixed in #37683 to reject such casts but of course some folks were
depending on this behavior to pass Swift data into Obj-C functions.
(The properly supported approach for passing arbitrary Swift data into
Obj-C functions is to cast the Swift value to `AnyObject`.)
This change makes that new behavior conditional. For now,
the legacy semantics are enabled on Apple platforms and the
new semantics are in use everywhere else. This will allow
us to gradually enable enforcement of the new behavior over
time.
* Just skip this test on Apple platforms, since it is inconsistently implemented there (and is therefore not really testable)
Conformance checks now walk the superclass chain in two stages: stage 1 only walks superclasses that have already been instantiated. When there's a negative result and there's an uninstantiated superclass, then stage 2 will walk the uninstantiated superclasses.
The infinite recursion would occur in this scenario:
class Super<T: P> {}
class Sub: Super<Sub>, P {}
Instantiating the metadata for Super requires looking up the conformance for Sub: P. Conformance checking for Sub would instantiate the metadata for Super to check for a Super: P conformance.
The compiler does not allow the conformance to come from a superclass in this situation. This does not compile:
class Super<T: P>: P {}
class Sub: Super<Sub> {}
Therefore it's not necessary to look at Super when finding the conformance for Sub: P in this particular case. The trick is knowing when to skip Super.
We do need to instantiate Super in the general case, otherwise we can get false negatives. This was addressed in a80fe8536b, which walks the full superclass chain during conformance checks, even if the superclass has not yet been instantiated. Unfortunately, that causes this infinite recursion.
This fix modifies that fix to make superclass instantiation conditional. The result is the ability to choose between the old algorithm (which skipped uninstantiated superclasses, albeit somewhat by accident) and the new one. A small wrapper then runs the check with the old algorithm, and then only if the old algorithm fails and there is an uninstantiated superclass, it runs with the new one.
Uninstantiated superclasses are uncommon and transient (you only run into this while metadata is in the process of being constructed) so 99.9999% of the time we'll just run the first stage and be done, and performance should remain the same as before.
rdar://80532245
The getSuperclassForMaybeIncompleteMetadata function assumes Swift metadata, and can malfunction strangely when given a pure ObjC class. This is rare, as we usually get the Swift wrapper metadata instead, but possible when using calls like objc_copyClassList.
Fix this by checking for isTypeMetadata before doing anything that assumes Swift-metadata-ness.
rdar://80336030
tryGetCompleteMetadataNonblocking crashes on artificial subclasses due to the NULL type descriptor. Explicitly check for artificial subclasses in getSuperclassForMaybeIncompleteMetadata and immediately return their Superclass field. Artificial subclasses are always fully initialized so we don't need to do anything special for them.
rdar://72583931
When we rely on a protocol conformance, and the type in question has multiple
conformances to that protocol in its inheritance chain, emit a runtime warning.
It's quite tricky to cause this problem - you need a type in one dylib that is
extended to conform to a protocol in another dylib, subclassed in another module
and then some subclass has protocol conformance added as well. If they're in
the same module, the compiler will give an error and prevent the problem
completely.
rdar://73364629
When constructing generic type metadata, we can end up checking protocol conformances for metadata that isn't fully built, and in particular the superclass field may not be set up yet. Handle this case properly by falling back to looking up the superclass by mangled name from the nominal type descriptor when necessary.
Previously we were just chasing Superclass fields, which would cause us to terminate the search early sometimes when concurrently looking up the same type from multiple threads. When a protocol conformance is on a superclass, this causes the search to fail incorrectly, causing type lookup failures.
This only arises on the very first attempt to instantiate a given type, since the lookup will be cached (and all metadata fully initialized) after the first success.
rdar://72583931
Take the existing CompatibilityOverride mechanism and generalize it so it can be used in both the runtime and Concurrency libraries. The mechanism is preprocessor-heavy, so this requires some tricks. Use the SWIFT_TARGET_LIBRARY_NAME define to distinguish the libraries, and use a different .def file and mach-o section name accordingly.
We want the global/main executor functions to be a little more flexible. Instead of using the override mechanism, we expose function pointers that can be set by the compatibility library, or by any other code that wants to use a custom implementation.
rdar://73726764
Match the logic used in the non-shared-cache case, where we walk up the class hierarchy to find the class where the conformance actually applies. This is important in cases like:
class Super<T> : Proto {}
class Sub: Super<Int> {}
Looking up the conformance `Sub: Proto` without this logic causes it to attempt to find the witness table for `Sub<Int>`, which fails. The correct logic looks up the witness table for `Super<Int>`.
While we're in there, fix a bug in the shared cache validation code (std::find wasn't checked for failure correctly) and add a `SWIFT_DEBUG_ENABLE_SHARED_CACHE_PROTOCOL_CONFORMANCES` environment variable to allow us to turn the shared cache integration off at runtime.
rdar://75431771
When available, take advantage of precomputed protocol conformances in the shared cache on Darwin platforms to accelerate conformance lookups and cut down on memory usage.
We consult the precomputed conformances before doing the runtime's standard conformance lookup. When a conformance is precomputed, this avoids the slow linear scan of all loaded conformances for the first access, and it avoids the memory cost of storing the conformance in the cache.
When the shared cache has no images overridden (the normal case), then we can also skip scanning conformances in the shared cache in all circumstances, because the precomputed conformances will always cover those results. This greatly speeds up the slow linear scan for the initial lookup of anything that doesn't have a conformance in the shared cache, including lookups with conformances in apps or app frameworks, and negative lookups.
A validation mode is available by setting the SWIFT_DEBUG_VALIDATE_SHARED_CACHE_PROTOCOL_CONFORMANCES environment variable. When enabled, results from the precomputed conformances are compared with the results from a slow scan of the conformances in the shared cache. This extremely slow, but good at catching bugs in the system.
When the calls for precomputed conformances are unavailable, the new code is omitted and we remain in the same situation as before, with the runtime performing all lookups by scanning conformance sections in all loaded Swift images.
rdar://71128983
In the uncached case, we'd scan conformances, cache them, then re-query the cache. This worked fine when the cache always grew, but now we clear the cache when loading new Swift images into the process. If that happens between the scan and the re-query, we lose the entry and return a false negative.
Instead, track what we've found in the scan in a separate local table, then query that after completing the scan.
While we're in there, fix a bug in TypeLookupError where operator= accidentally copied this->Context instead of other.Context. This caused the runtime to crash when trying to print error messages due to the false negative.
Add a no-parameter constructor to TypeLookupErrorOr<> to distinguish the case where it's being initialized with nothing from the case where it's being initialized with nullptr.
This gives us build-time warnings about format string mistakes, like we would get if we called the built-in asprintf directly.
Make TypeLookupError's format string constructor a macro instead so that its callers can get these build-time warnings.
This reveals various mistakes in format strings and arguments in the runtime, which are now fixed.
rdar://73417805
to use it.
ConcurrentReadableHashMap is lock-free for readers, with writers using a lock to
ensure mutual exclusion amongst each other. The intent is to eventually replace
all uses ConcurrentMap with ConcurrentReadableHashMap.
ConcurrentReadableHashMap provides for relatively quick lookups by using a hash
table. Rearders perform an atomic increment/decrement in order to inform writers
that there are active readers. The design attempts to minimize wasted memory by
storing the actual elements out-of-line, and having the table store indices into
a separate array of elements.
The protocol conformance cache now uses ConcurrentReadableHashMap, which
provides faster lookups and less memory use than the previous ConcurrentMap
implementation. The previous implementation caches
ProtocolConformanceDescriptors and extracts the WitnessTable after the cache
lookup. The new implementation directly caches the WitnessTable, removing an
extra step (potentially a quite slow one) from the fast path.
The previous implementation used a generational scheme to detect when negative
cache entries became obsolete due to new dynamic libraries being loaded, and
update them in place. The new implementation just clears the entire cache when
libraries are loaded, greatly simplifying the code and saving the memory needed
to track the current generation in each negative cache entry. This means we need
to re-cache all requested conformances after loading a dynamic library, but
loading libraries at runtime is rare and slow anyway.
rdar://problem/67268325
Previously, when an attempt was made to instantiate a generic metadata
whose argument was constrained to subclass a superclass Super with an
existential type E that had a superclass constraint to a subclass Sub of
that same superclass Super, the instantiation would fail at runtime,
despite the fact that the generic instantiation was allowed to
type-check.
The result was a runtime failure to instantiate generic metadata
resulting eventually in a crash.
Here, handling for that situation is added. When checking generic
requirements at runtime, when a superclass requirement is encountered,
an existential type is checked for. If that existential type has a
superclass constraint and if that superclass constraint is a subclass of
the superclass requirement, the check is determined to be satisfactory.
rdar://problem/64672291
This cleans up some more `llvm::` leakage in the runtime when built into
a static library. With this change we are down to 3 leaking symbols in
the static library related to a missed ADT (`StringSwitch`).
Rather than using the forward declaration for the LLVMSupport types,
expect to be able to use the full declaration. Because these are
references in the implementation, there is no reason to use a forward
declaration as the full types need to be declared for use. The LLVM
headers will provide the declaration and definition for the types. This
is motivated by the desire to ensure that the LLVMSupport symbols are
properly namespaced to avoid ODR violations in the runtime.
When constructing the metadata for a type Gen<T : Super>
where Super is a superclass constraint, the generic argument K at which
the metadata for Gen is being instantiated is verified to be a subclass
of Super via _checkGenericRequirements.
Previously, that check was done using swift_dynamicCastMetatype. That
worked for the most part but provided an incorrect answer if the
metadata for K was not yet complete. These classes are incomplete more
often thanks to __swift_instantiateConcreteTypeFromMangledNameAbstract.
That issue occurred concretely in the following case:
Framework with Library Evolution enabled:
open class Super { ... }
public struct Gen<T : Super> {
}
Target in a different resilience domain from that framework:
class Sub : Super {
var gen: Gen<Sub>?
}
Here, the mechanism for checking whether the generic argument K at which
the metadata for Gen is being instantiated handles the case where K's
metadata is incomplete. At worst, every superclass name from super(K)
up to Super are demangled to instantiate metadata. A number of faster
paths are included as well.
rdar://problem/60790020
The ConformanceCandidate constructor would eagerly instantiate metadata for
all non-generic types in the conformance cache. This was not the intention
of the code though, because it can compare nominal type descriptors --
which are emitted statically -- for those types that have them, namely
everything except for foreign and Objective-C classes.
Change the order of two calls so that the lazy path has a chance to run.
This fixes a crash when type metadata uses weakly-linked symbols
which are not available, which can come up in backward deployment
scenarios.
Fixes <rdar://problem/59460603>.
An extension on a class creates a conformance record that's always visible even when that class is not present at runtime. In that case, the type pointer in the conformance record is NULL. The runtime did not like this, and crashed. This fixes it to ignore such records instead.
rdar://problem/54054895
This is a one-to-many cache that's more speculative than the cache mapping mangled names to context descriptors. Entries found in the cache need to be verified for a match before they can be returned. However, this allows scanning conformance records up front and building up the cache in one scan rather than performing an expensive scan of all conformance records every time the mangled name cache misses.
rdar://problem/53560010
When we find a protocol conformance descriptor for a given type, make sure
we adjust to the conforming type of that descriptor (following the superclass
chain as needed) before instantiating the witness table.
Fixes rdar://problem/49741838.
