When we generate code that asks for complete metadata for a fully concrete specific type that
doesn't have trivial metadata access, like `(Int, String)` or `[String: [Any]]`,
generate a cache variable that points to a mangled name, and use a common accessor function
that turns that cache variable into a pointer to the instantiated metadata. This saves a bunch
of code size, and should have minimal runtime impact, since the demangling of any string only
has to happen once.
This mostly just works, though it exposed a couple of issues:
- Mangling a type ref including objc protocols didn't cause the objc protocol record to get
instantiated. Fixed as part of this patch.
- The runtime type demangler doesn't correctly handle retroactive conformances. If there are
multiple retroactive conformances in a process at runtime, then even though the mangled string
refers to a specific conformance, the runtime still just picks one without listening to the
mangler. This is left to fix later, rdar://problem/53828345.
There is some more follow-up work that we can do to further improve the gains:
- We could improve the runtime-provided entry points, adding versions that don't require size
to be cached, and which can handle arbitrary metadata requests. This would allow for mangled
names to also be used for incomplete metadata accesses and improve code size of some generic
type accessors. However, we'd only be able to take advantage of the new entry points in
OSes that ship a new runtime.
- We could choose to always symbolic reference all type references, which would generally reduce
the size of mangled strings, as well as make runtime demangling more efficient, since it wouldn't
need to hit the runtime caches. This would however require that we be able to handle symbolic
references across files in the MetadataReader in order to avoid regressing remote mirror
functionality.
Dynamic replacement can only really get away with replacing opaque return types if a function's
opaque type has never been instantiated in the first place. Meanwhile, repeatedly instantiating
the metadata is too slow for many clients to tolerate. Fixes rdar://problem/53213600.
This is to support dynamic function replacement of functions with opaque
result type.
This approach requires that all state is thrown away (that could contain the
old returned type for an opaque type) between replacements.
rdar://48887938
If a class has resilient metadata from our point of view, it might
still not have a class stub, if its entire inheritance chain is
defined in a single module.
Note that inserting a superclass is still a resilient operation;
the only way to change a class from having static metadata to having
a class stub is to change it's root class, which is not something
we can do resiliently.
Instead of a wholly separate lazyness mechanism for foreign metadata where
the first call to getAddrOfForeignTypeMetadataCandidate() would emit the
metadata, emit it using the lazy metadata mechanism.
This eliminates some code duplication. It also ensures that foreign
metadata is only emitted once per SIL module, and not once per LLVM
module, avoiding duplicate copies that must be ODR'd away in multi-threaded
mode.
This fixes the test case from <rdar://problem/49710077>.
Non-generic classes with resilient ancestry do not have statically-emitted
metadata, so we can now emit an Objective-C resilient class stub instead.
Also, when emitting an Objective-C category, reference the class stub if
the class has resilient ancestry; previously this case would hit an assert.
Note that class stubs always start with a zero word, with the address point
pointing immediately after. This works around a linker issue, where the
linker tries to coalesce categories and gets confused upon encountering a
class stub.
This consolidates the various doesClassMetadataRequire*() checks, making
them more managable.
This also adds a forth state, ClassMetadataStrategy::Update. This will be used
when deploying to the new Objective-C runtime. For now it's not plumbed through.
Progress on <rdar://problem/47649465>.
In our initial approach for resolving metadata dependency cycles with classes, non-transitively complete superclass metadata was fetched by the subclass's metadata completion function and passed to `swift_initClassMetadata`. That could mean generating quite a lot of code in the completion function, and so we fairly recently changed it so that `swift_initClassMetadata` instead fetched the superclass metadata via a demangling. Unfortunately, the metadata demangler only fetches _abstract_ metadata by default, and class metadata cannot be considered even non-transitively complete when its superclass reference not at that stage. If the superclass metadata is being completed on one thread, and a subclass is being completed on another, and the subclass installs the incomplete superclass metadata in its superclass field and attempts to register the subclass with the Objective-C runtime, the runtime may crash reading the incompletely-initialized superclass.
The proper fix is to make `swift_initClassMetadata` fetch non-transitively complete metadata for the superclass, delaying completion if that metadata is unavailable. Unfortunately, that can't actually be implemented on top of `swift_initClassMetadata` because that function has no means of reporting an unsatisfied dependency to its caller, and we can no longer simply change its signature without worrying about a small of internal code that might still be using it. We cannot simply perform a blocking metadata request in `swift_initClassMetadata` because it is deeply problematic to block within a metadata completion function. The solution is therefore to add a `swift_initClassMetadata2` which has the ability to report unsatisfied dependencies. That was done in #22386; this patch builds on that by teaching the compiler to generate code to actually use it. It is therefore not safe to use this patch if you might be running on an OS that only provides the old runtime function, but that should be a temporary Apple-internal problem.
Fixes rdar://47549859.
Use the `IRLInkage::InternalLinkOnceODR` linkage rather than computing
that in a couple of sites. This gives us semantic meaning to the
linkage being applied. NFC.
Extend the key-path pattern with a representation of the generic environment
of the key-path, which includes the generic parameters and generic
requirements of the environment.
When a type in keypath metadata is non-dependent, use a mangled type name
rather than a symbolic reference to an accessor function.
Part of rdar://problem/38038799.
Switch key path metadata over to mangled names for each of the places it
refers to either a type metadata accessor or a witness table accessor. For
now, the mangled name is a symbolic reference to the existing accessors.
Part of rdar://problem/38038799.
In order to handle LinkOnceODR semantics correctly across various object
formats, introduce a new helper ApplyIRLinkage. This abstracts the need
to create a COMDAT group and set it on the GlobalValue. Adjust all
sites where we set the IR linkage attributes to use this mechanism
instead to avoid having to track down symbols not being added to a
COMDAT group.
Rather than mapping all of the builtin floating-point and vector
types down to the type metadata symbols for power-of-two integers,
map down to the type metadata symbols provided by the runtime.
This allows us to round-trip type metadata <-> mangled name for all of
the builtin types that have such symbols. When we don’t have a runtime
symbol, we will fail to link if that builtin type gets referenced.
Add `@autoclosure` to parameter flags associated with
function type metadata, which makes it possible to correctly
round-trip mangled name <-> metadata of function types which
have parameters marked as `@autoclosure`.
Resolves: rdar://problem/45489901
Extending the mangling of symbolic references to also include indirect
symbolic references. This allows mangled names to refer to context
descriptors (both type and protocol) not in the current source file.
For now, only permit indirect symbolic references within the current module,
because remote mirrors (among other things) is unable to handle relocations.
Co-authored-by: Joe Groff <jgroff@apple.com>
There are some cases where the AST’s notion of what constitutes function parameter flags
differ from the ABI notion. Specifically, these are @escaping and @autoclosure, which
are both modeled as types in the ABI. Teach IRGen to look at the ABI’s version of
function parameter flags when determining whether to pass parameter flags
to swift_getFunctionType([0-3])?.
Without this fix, we would end up with mismatches between the function types
built by IRGen and those built from mangled names. Part of
rdar://problem/37551850.
Encode default associated type witnesses using a sentinel prefix byte
(0xFF) in the mangled name rather than as a second low bit on the
reference. Align all of the mangled names used for type references to
2 bytes (so we get that low bit regardless) and separate the symbol
names for default associated type witnesses vs. other kinds of
metadata or reflection metadata.
Rather than storing associated type metadata access functions in
witness tables, initially store a pointer to a mangled type name.
On first access, demangle that type name and replace the witness
table entry with the resulting type metadata.
This reduces the code size of protocol conformances, because we no
longer need to create associated type metadata access functions for
every associated type, and the mangled names are much smaller (and
sharable). The same code size improvements apply to defaulted
associated types for resilient protocols, although those are more
rare. Witness tables themselves are slightly smaller, because we
don’t need separate private entries in them to act as caches.
On the caller side, associated type metadata is always produced via
a call to swift_getAssociatedTypeWitness(), which handles the demangling
and caching behavior.
In all, this reduces the size of the standard library by ~70k. There
are additional code-size wins that are possible with follow-on work:
* We can stop emitting type metadata access functions for non-resilient
types that have constant metadata (like `Int`), because they’re only
currently used as associated type metadata access functions.
* We can stop emitting separate associated type reflection metadata,
because the reflection infrastructure can use these mangled names
directly.
Similar to the non-resilient case, except we also emit a 'relocation
function'. The class descriptor now contains this relocation function
if the class has resilient ancestry, and the relocation function
calls the runtime's swift_relocateClassMetadata() entry point.
The metadata completion function calls swift_initClassMetadata() and
does layout, just like the non-resilient case.
Fixes <rdar://problem/40810002>.
In-place initialization means the class has a symbol we can reference
from the category, so there's nothing to do on the IRGen side.
For JIT mode, we just need to realize the class metadata by calling an
accessor instead of directly referencing the symbol though.
- doesClassMetadataRequireRelocation() -- returns true if we must
allocate new metadata at runtime and fill it in, because the class
has multiple instantiations (generic case) or because the total size
of the metadata is not known at compile time (resilient ancestry).
- doesClassMetadataRequireInitialization() -- weaker condition than
the above. It's true if the metadata must be relocated, but it is
also true if the metadata has otherwise fixed size but must be
filled in dynamically. This occurs if the class has generic
ancestry but is itself not generic, or if the class has
resiliently-sized fields, or missing members.
For now, we don't actually care about the distinciton anywhere,
because we cannot do in-place initialization of class metadata yet.
This makes it easier to grep for and eventually remove the
remaining usages.
It also allows you to write FunctionType::get({}, ...) to call the
ArrayRef overload empty parameter list, instead of picking the Type
overload and calling it with an empty Type() value.
While I"m at it, in a few places instead of renaming just clean up
usages where it was completely mechanical to do so.
We’re not using this parameter, and don’t expect to do so in the future,
so remove it. Also fold away TypeBase::usesNativeReferenceCounting()
and irgen::getReferenceCountingForType(), both of which are trivial.
Previously, when a tuple type had non-fixed layout, we would compute
a layout by building the metadata for that tuple type and then
extracting the layout from the VWT. This can be quite expensive
because it involves constructing the exact metadata for types like
arrays and functions despite those types being fixed-layout across
all instantiations. It also tends to cause unnecessary recursive-type
issues, especially with enums where tuples are currently used to model
cases with mutliple payloads. Since we just need a layout, computing
it directly from element layouts instead of constructing metadata for
the formal type lets us take advantage of all the other fast paths for
layout construction, e.g. for fixed types and single-field aggregates.
This is a good improvement overall, but it also serves to alleviate
some of the problems of rdar://40810002 / SR-7876 in a way that
might be suitable for integration to 4.2.
- `swift_getForeignTypeMetadata` is now a request/response function.
- The initialization function is now a completion function, and the
pointer to it has moved into the type descriptor.
- The cache variable is no longer part of the ABI; it's an
implementation detail of the access function.
- The two points above mean that there is no special header on foreign
type metadata and therefore that they can be marked constant when
there isn't something about them that needs to be initialized.
The only foreign-metadata initialization we actually do right now is
of the superclass field of a foreign class, and since that relationship
is a proper DAG, it's not actually possible to have recursive
initialization problems. But this is the right long-term thing to do,
and it removes one of the last two clients of once-based initialization.
The prefab'ed value witness tables for reference storage types are a
premature optimization. Not all scenarios are covered, and those that
are "look suspect" according to John McCall.
The central thrust of this patch is to get these metadata initializations
off of `swift_once` and onto the metadata-request system where we can
properly detect and resolve dependencies. We do this by first introducing
runtime support for resolving metadata requests for "in-place"
initializations (committed previously) and then teaching IRGen to actually
generate code to use them (this patch).
A non-trivial amount of this patch is just renaming and refactoring some of
existing infrastructure that was being used for in-place initializations to
try to avoid unnecessary confusion.
The remaining cases that are still using `swift_once` resolution of
metadata initialization are:
- non-generic classes that can't statically fill their superclass or
have resilient internal layout
- foreign type metadata
Classes require more work because I'd like to switch at least the
resilient-superclass case over to using a pattern much more like what
we do with generic class instantiation. That is, I'd like in-place
initialization to be reserved for classes that actually don't need
relocation.
Foreign metadata should also be updated to the request/dependency scheme
before we declare ABI stability. I'm not sure why foreign metadata
would ever require a type to be resolved, but let's assume it's possible.
Fixes part of SR-7876.
Switch one entry point in the runtime (swift_getExistentialTypeMetadata)
to use ProtocolDescriptorRef rather than a protocol descriptor. Update
IRGen to produce ProtocolDescriptorRef instances for its calls, setting
the discriminator bit appropriately.
Within the runtime, verify that all instances of ProtocolDescriptorRef have
the right layout, i.e., the discriminator bit is set for @objc protocols
but not Swift protocols.
Otherwise, we might directly use uninitialized metadata, because we think
the type has a fixed size, even if it was defined in a different module
built with resilience.
Fixes <rdar://problem/40034143>, for real this time.
More groundwork for protocols with superclass constraints.
In several places we need to distinguish between existential
types that have a superclass term (MyClass & Proto) and
existential types containing a protocol with a superclass
constraint.
This is similar to how I can write 'AnyObject & Proto', or
write 'Proto1 & Proto2' where Proto1 has an ': AnyObject'
in its inheritance clause.
Note that some of the usages will be revisited later as
I do more refactoring and testing. This is just a first pass.
Because the runtime is compacted into the standard library, functions
which are normally imported are actually local definitions. Use module
level named metadata to identify the module as being the swift standard
library. Refactor the condition slightly to improve code readability.
This addresses SR-7107!
This work-around is no longer needed now that the full fix landed in
https://github.com/apple/swift/pull/16615. The argument is left with a warning
to help with migration between compilers with the work-around and compilers with
the full fix (see also rdar://problem/40502379).
Fixes rdar://problem/40476573.
IRGen can introduce calls to type metadata accessors for types that
should not be visible to the current translate, which can manifest in
linker errors within a module (for references to private types when
whole module optimization is disabled) or across modules (for
references to private/internal types in another module). Introduce a
new compiler flag `-emit-public-type-metadata-accessors` that emits
all type metadata accessors with public linkage, to work around the
problem in affected projects. This flag is intended to go away once we
have a proper solution.
This bug has been around in Swift "forever", but compiling the
overlays using -enable-resilience has exacerbated the problem and
caused regressions. This is a short-term fix to
rdar://problem/40229755 while we work on the correct long-term fix.
