Until recently, `MemoryReader` had a single function `resovlePointer` which did two things, and has a somewhat vague name. The two things were:
1. Tool-specific mapping between real addresses and tagged addresses (first implemented in `swift-reflection-dump` and then later in lldb)
2. Finding a "symbol" for a given address
Recently, `resolvePointerAsSymbol` was added, which overloaded the term "resolve" and it added another way to deal with symbols for addresses. Symbols themselves were a bit muddled, as `swift-reflection-dump` was dealing with dynamic symbols aka bindings, while lldb was dealing in regular (static) symbols.
This change separates these two parts of functionality, and also divides symbol lookup into two cases. The API surface will now be:
1. `resolvePointer` for mapping/tagging addresses
3. `getSymbol` for looking up a symbol for an address
4. `getDynamicSymbol` for looking up a dyld binding for an address
Note: each of these names could be improved. Some alternative terms: `lookup` instead of `get`, `Binding` or `BindingName` instead of `DynamicSymbol`. Maybe even another term instead of "resolve". Suggestions welcome!
Currently, `swift-reflection-dump` supports `getDynamicSymbol` but not `getSymbol`. For lldb it's the reverse, `getSymbol` is supported but `getDynamicSymbol` needs to be implemented.
For everything but lldb, this change is NFC. For lldb it fixes a bug where `LLDBMemoryReader` returns regular symbols where we should instead be returning dynamic symbols.
In order to be able to debug, for example, a Linux process from a macOS host, we
need to be able to initialize a ReflectionContext without Objective-C
interoperability. This patch turns ObjCInterOp into another template trait, so
it's possible to instantiate a non-ObjC MetadataReader on a system built with
ObjC-interop (but not vice versa).
This patch changes the class hierarchy to
TargetMetadata<Runtime>
|
TargetHeapMetadata<Runtime>
|
TargetAnyClassMetadata<Runtime>
/ \
/ TargetAnyClassMetadataObjCInterop<Runtime>
/ \
TargetClassMetadata<Runtime, TargetAnyClassMetadata<Runtime>> \
\
TargetClassMetadata<Runtime, TargetAnyClassMetadataObjCInterop<Runtime>>
TargetAnyClassMetadataObjCInterop inherits from TargetAnyClassMetadata because
most of the implementation is the same. This choice makes TargetClassMetadata a
bit tricky. In this patch I went with templating the parent class.
rdar://87179578
We remove the existing `swift_reflection_iterateAsyncTaskAllocations` API that attempts to provide all necessary information about a tasks's allocations starting from the task. Instead, we split it into two pieces: `swift_reflection_asyncTaskSlabPointer` to get the first slab for a task, and `+swift_reflection_asyncTaskSlabAllocations` to get the allocations in a slab, and a pointer to the next slab.
We also add a dummy metadata pointer to the beginning of each slab. This allows tools to identify slab allocations on the heap without needing to locate every single async task object. They can then use `swift_reflection_asyncTaskSlabAllocations` on such allocations to find out about the contents.
rdar://82549631
We remove the existing `swift_reflection_iterateAsyncTaskAllocations` API that attempts to provide all necessary information about a tasks's allocations starting from the task. Instead, we split it into two pieces: `swift_reflection_asyncTaskSlabPointer` to get the first slab for a task, and `+swift_reflection_asyncTaskSlabAllocations` to get the allocations in a slab, and a pointer to the next slab.
We also add a dummy metadata pointer to the beginning of each slab. This allows tools to identify slab allocations on the heap without needing to locate every single async task object. They can then use `swift_reflection_asyncTaskSlabAllocations` on such allocations to find out about the contents.
rdar://82549631
Mangling can fail, usually because the Node structure has been built
incorrectly or because something isn't supported with the old remangler.
We shouldn't just terminate the program when that happens, particularly
if it happens because someone has passed bad data to the demangler.
rdar://79725187
MetadataReader can be given corrupt or garbage data and we need to be able to handle it gracefully. When reading metadata, the full size to read is calculated from partial data. When we're given bad data, these calculated sizes can be enormous, up to 4GB. Trying to read that much data can cause address space exhaustion which leads to unpleasantness.
To avoid this, fix a limit of 1MB on metadata sizes, and fail early if the size is larger. No real-world metadata should ever be that large.
We also switch these potentially large calls to use the readBytes variant that returns a unique_ptr, rather than allocating a buffer and reading into it. Our clients typically implement that as the primitive, so this avoids an unnecessary extra data copy and extra address space usage for them. Clients that implement reading into a provided buffer as the primitive should see the same performance as before.
rdar://78621784
Implement name mangling, type metadata, runtime demangling, etc. for
global-actor qualified function types. Ensure that the manglings
round-trip through the various subsystems.
Implements rdar://78269642.
These calls can fail when passed absurd sizes, which can happen when we try to read data that's corrupt or doesn't contain what we think it should. Fail gracefully instead of crashing.
rdar://78210820
* Move differentiability kinds from target function type metadata to trailing objects so that we don't exhaust all remaining bits of function type metadata.
* Differentiability kind is now stored in a tail-allocated word when function type flags say it's differentiable, located immediately after the normal function type metadata's contents (with proper alignment in between).
* Add new runtime function `swift_getFunctionTypeMetadataDifferentiable` which handles differentiable function types.
* Fix mangling of different differentiability kinds in function types. Mangle it like `ConcurrentFunctionType` so that we can drop special cases for escaping functions.
```
function-signature ::= params-type params-type async? sendable? throws? differentiable? // results and parameters
...
differentiable ::= 'jf' // @differentiable(_forward) on function type
differentiable ::= 'jr' // @differentiable(reverse) on function type
differentiable ::= 'jd' // @differentiable on function type
differentiable ::= 'jl' // @differentiable(_linear) on function type
```
Resolves rdar://75240064.
* Limit the recursion depth when trying to get the mangling for a context descriptor
This should help guard against corrupted data in the target app when debugging.
* Only decrement the recursion limit once for each parent visit
Add `async` to the type system. `async` can be written as part of a
function type or function declaration, following the parameter list, e.g.,
func doSomeWork() async { ... }
`async` functions are distinct from non-`async` functions and there
are no conversions amongst them. At present, `async` functions do not
*do* anything, but this commit fully supports them as a distinct kind
of function throughout:
* Parsing of `async`
* AST representation of `async` in declarations and types
* Syntactic type representation of `async`
* (De-/re-)mangling of function types involving 'async'
* Runtime type representation and reconstruction of function types
involving `async`.
* Dynamic casting restrictions for `async` function types
* (De-)serialization of `async` function types
* Disabling overriding, witness matching, and conversions with
differing `async`
This removes the last reference to the `llvm::` namespace in the
standard library. All uses of the LLVMSupport library now are
namespaced into the `__swift::__runtime` namespace. This allows us to
incrementally vend the LLVMSupport library and make the separation
explicit.
Resolve mangled names containing symbolic references to indirect opaque type descriptors from other
dylibs by demangling the referenced symbol name, like we do for other kinds of context descriptor.
Add an OpaqueArchetypeTypeRef that can represent unresolved opaque types in the Reflection library.
ownsAddress was a simple range check on images, but that won't find Metadatas that get allocated on the heap. If an address isn't found, try reading it as a Metadata and doing a range check on the type context descriptor too.
rdar://problem/60981575
Newer Objective-C runtimes implement a size optimization in class_rw_t
which requires an additional indirection to get to the class_ro_t pointer.
Thanks to Davide for getting to the bottom of this!
Pointer data in some remote reflection targets may required relocation, or may not be
fully resolvable, such as when we're dumping info from a single image on disk that
references other dynamic libraries. Add a `RemoteAbsolutePointer` type that can hold a
symbol, offset, or combination of both, and add APIs to `MemoryReader` and `MetadataReader`
for reading pointers that can get unresolved relocation info from an image, or apply
relocations to pointer information. MetadataReader can use the symbol name information to
fill in demanglings of symbolic-reference-bearing mangled names by using the information
from the symbol name to fill in the name even though the context descriptors are not
available.
For now, this is NFC (MemoryReader::resolvePointer just forwards the pointer data), but
lays the groundwork for implementation of relocation in ObjectMemoryReader.
TypeRefBuilder and MetadataReader had nearly identical symbolic reference resolvers,
but diverged because TypeRefBuilder had its own local/remote address management mechanism,
and because TypeRefBuilder tries to resolve opaque types to their underlying types, whereas
other MetadataReader clients want to preserve them as written in source. The first problem
has been addressed by making TypeRefBuilder use `RemoteRef` everywhere, and the second
can be handled with a flag (and might be able to be handled more elegantly with some more
refactoring of general opaque type handling in MetadataReader).
Instead of passing around raw local pointers and references, and spreading
tricky offset arithmetic around with the Local/RemoteAddress fields in
ReflectionInfo, have the TypeRefBuilder code use RemoteRefs everywhere,
which keep the remote/local mapping together in one unit and provide
centralized API for this logic.
This doesn't yet change how code uses the RemoteRef address data to
follow pointers across objects, for things like reading type refs, but
that should be much easier to do after this lands.
By including the trailing mangled name reference in the baseSize, we computed the wrong offset for
the generic parameter header, and then miscomputed the size of the trailing generic context info.
This would lead to accesses into the context sometimes reading from uninitialized memory.
Fixes rdar://problem/55711107
Resolving a direct relative reference given a RemoteRef doesn't need the MetadataReader,
since the offset should already be in the local buffer; we can add it to RemoteRef's
saved remote address and get a new remote address. Refactor the API to make as much as
possible of it available directly on RemoteRef.
The only thing the Runtime affects is the width of the StoredPointer for the remote address, for
which storing a uint64_t ought to be enough for anyone we care about so far. This will make it
easier to store and use RemoteRefs in code that isn't or shouldn't ideally be templatized on
Runtime (such as TypeRefBuilder, and ultimately ReflectionContext, from the Reflection library.)
This makes for a cleaner and less implicit-context-heavy API, and makes it easier for symbolic
reference resolvers to do context-dependent things (like map the in-memory base address back to a
remote address in MetadataReader).
