The inputs to the hash function is pointers that have a predictable patten. The
hashes that we were generating and using for the metadata caches were not very
good, and as a result we generated very deep search trees. A small change that
improved the utilization of the 'length' field and another bit-rotate round
improved the quality of the hash function.
I am going to attach to the github commit two pictures. The first picture is the
binary with the old hash. The first tree is very deep and sparse. The second
picture is with the new hash function and the tree is very wide and uniform. I
used the benchmark 'TypeFlood' in debug mode to generate huge amounts of
metadata.
Before this commit we only checked the conformance of the first protocol that
matched the conformance entry, which is incorrect. We need to check all of the
entries that match the key before deciding that a type does not conform to a
protocol.
This commit fixes repl_conformance_lookup.swift
This change cuts the number of mallocs() in the metadata caches in half.
The current metadata cache data structure uses a linked list for each entry in
the tree to handle collissions. This means that we need at least two memory
allocations for each entry, one for the tree node and one for the linked list
node.
This commit changes the map used by the metadata caches from an open hash map
(that embeds a linked list at each entry) into an closed map that uses a
different hash value for each entry. With this change we no longer accept
collissions and it is now the responsibility of the user to prevent collissions.
The new get/trySet API makes this responsibility explicit. The new design also
goes well with the current design where hashing is done externally and the fact
that we don't save the full key, just the hash and the value to save memory.
This change reduces the number of allocated objects per entry in half. Instead
of allocating two 32-byte objects (one for the tree node and one for the linked
list) we just allocate a single entry that contains the hash and the value.
Unfortunately, values that are made of two 64-bit pointers (like protocol
conformance entries) are now too big for the 32-byte tree entry and are rounded
up to 48 bytes. In practice this is not a big deal because malloc has 48-byte pool
entries.
...and explicitly mark symbols we export, either for use by executables or for runtime-stdlib interaction. Until the stdlib supports resilience we have to allow programs to link to these SPI symbols.
These had similar problems to the protocol conformance sections during
Linux bringup. There are relative offsets from the main reflection
section to the two string sections, so their position relative to each
other must remain fixed.
This is the first patch in a series that will allow new protocol
requirements to be added resiliently, with the runtime filling in
default implementations in witness tables.
First, this adds a new flag to the protocol descriptor indicating
that the protocol is resilient. In this case, there are two
additional fields, MinimumWitnessTableSizeInWords and
DefaultWitnessTableSizeInWords, followed by tail-allocated
default witnesses.
The swift_getGenericWitnessTable() entry point now fills in the
default witnesses from the protocol if the given witness table
template is smaller than the expected witness table size.
This also changes the layout of instantiated witness tables to move
the address point to the end of private data. Previously the private
data came after the requirements, but this meant that adding new
requirements would require sliding the private data at runtime and
accessing it indirectly. It is much simpler to access it from
negative offsets instead.
I updated IRGen to emit the new metadata, but currently all protocols
are flagged as not resilient, and default witnesses are not emitted;
this will come in a subsequent patch once some more plumbing is
in place.
To avoid generating GOT entries for references to protocols defined
in the current module, I had to add some hacks to the existing hack
for this. I'll hopefully clean this up in a principled manner later.
This comes with a fix for a null pointer dereference in _typeByName()
that would pop with foreign classes that do not have a
NominalTypeDescriptor.
Also, I decided to back out part of the change for now, where the
NominalTypeDescriptor references an accessor function instead of a
pattern, since this broke LLDB, which reaches into the pattern to
get the generic cache.
Soon we will split off the generic cache from the pattern, and at
that time we can change the NominalTypeDescriptor to point at the
cache. But for now, let's avoid needless churn in LLDB by keeping
that part of the setup unchanged.
Change conformance records to reference NominalTypeDescriptors instead of
metadata patterns for resilient or generic types.
For a resilient type, we don't know if the metadata is constant or not,
so we can't directly reference either constant metadata or the metadata
template.
Also, whereas previously NominalTypeDescriptors would point to the
metadata pattern, they now point to the metadata accessor function.
This allows the recently-added logic for instantiating concrete types
by name to continue working.
In turn, swift_initClassMetadata_UniversalStrategy() would reach into
the NominalTypeDescriptor to get the pattern out, so that its bump
allocator could be used to allocate ivar tables. Since the pattern is
no longer available this way, we have to pass it in as a parameter.
In the future, we will split off the read-write metadata cache entry
from the pattern; then swift_initClassMetadata_UniversalStrategy() can
just take a pointer to that, since it doesn't actually need anything
else from the pattern.
Since Clang doesn't guarantee alignment for function pointers, I had
to kill the cute trick that packed the NominalTypeKind into the low
bits of the relative pointer to the pattern; instead the kind is now
stored out of line. We could fix this by packing it with some other
field, or keep it this way in case we add new flags later.
Now that generic metadata is instantiated by calling accessor functions,
this change removes the last remaining place that metadata patterns were
referenced from outside the module they were defined in. Now, the layout
of the metadata pattern and the behavior of swift_getGenericMetadata()
is purely an implementation detail of generic metadata accessors.
This patch allows two previously-XFAIL'd tests to pass.
This pull request broke the following tests on several build configurations
(eg --preset=buildbot,tools=RA,stdlib=DA)
1_stdlib/Reflection.swift
1_stdlib/ReflectionHashing.swift
1_stdlib/UnsafePointer.swift.gyb
This reverts commit c223a3bf06, reversing
changes made to 5c2bb09b09.
Changes:
- Reverted commit reverting original SR-88 commit
- Removed mirror children helper collections and related code
- Rewrote some tests to keep them working properly
- Wrote two more tests for the three pointer APIs to ensure no crashes if created using a value > Int64.max
This reverts commit 8917eb0e5a.
Instead of directly emitting calls to swift_getGenericMetadata*() and
referencing metadata templates, call a metadata accessor function
corresponding to the UnboundGenericType of the NominalTypeDecl.
The body of this accessor forwards arguments to a runtime metadata
instantiation function, together with the template.
Also, move some code around, so that metadata accesses which are
only done as part of the body of a metadata accessor function are
handled separately in emitTypeMetadataAccessFunction().
Apart from protocol conformances, this means metadata templates are
no longer referenced from outside the module where they were defined.
Recent changes added support for resiliently-sized enums, and
enums resilient to changes in implementation strategy.
This patch adds resilient case numbering, fixing the problem
where adding new payload cases would break existing code by
changing the numbering of no-payload cases.
The problem is that internally, enum cases are numbered with payload
cases coming first, followed by no-payload cases. While each list
is itself in declaration order, with new additions coming at the
end, we need to partition it to give us a fast runtime test for
"is this a payload or no-payload case index."
The resilient numbering strategy used here is that the getEnumTag
and destructiveInjectEnumTag value witness functions now take a
tag index in the range [-ElementsWithPayload..ElementsWithNoPayload-1].
Payload elements are numbered in *reverse* declaration order, so
adding new payload cases yields decreasing tag indices, and adding
new no-payload cases yields increasing tag indices, allowing use
sites to be resilient.
This adds the adjustment between 'fragile' and 'resilient' tag
indices in a somewhat unsatisfying manner, because the calculation
could be pushed down further into EnumImplStrategy, simplifying
both the IRGen code and the generated IR. I'll clean this up later.
In the meantime, clean up some other stuff in GenEnum.cpp, mostly
abstracting code that walks cases.
Jira: SR-88
Changes:
- Removed stdlib type conformances to _Reflectable
- Conformed stdlib types to CustomReflectable, CustomPlaygroundQuickLookable
- Rewrote dump() function to not use _reflect()
- CGRect, CGPoint, CGSize now conform to CustomDebugStringConvertible
- Rewrote unit tests for compatibility with new API
Decrease the size of nominal type descriptors and make them true-const by relative-addressing the other metadata they need to reference, which should all be included in the same image as the descriptor itself. Relative-referencing string constants exposes a bug in the Apple linker, which crashes when resolving relative relocations to coalesceable symbols (rdar://problem/22674524); work around this for now by revoking the `unnamed_addr`-ness of string constants that we take relative references to. (I haven't tested whether GNU ld or gold also have this problem on Linux; it may be possible to conditionalize the workaround to only apply to Darwin targets for now.)
_swift_buildDemanglingForMetadata() is being built for the non-ObjC interop
case as a result of a change introduced in an earlier version of the patch for
SR-381 that was not correctly backed out.
replace ProtocolConformanceTypeKind with TypeMetadataRecordKind
metadata reference does not need to be indirectable
more efficient check for protocol conformances
remove swift_getMangledTypeName(), not needed yet
kill off Remangle.cpp for non-ObjC builds
cleanup
cleanup
cleanup comments
The runtime support for casting optionals introduced in 35cb1afa
resulted in a redundant retain.
Fixes SR-459: Weakened optionals don't zero...
rdar://24057977.
This case was previously ignoring the DestroyOnFailure flag, so
we had a leak if a cast to an existential metatype failed for
certain types (tuples, structs, etc).
