Not quite NFC because apparently the representation bleeds into what's
accepted in some situations where we're supposed to be warning about
conflicts and then making an arbitrary choice. But what we're doing
is nonsense, so we definitely need to break behavior here.
This is setting up for isolated(any) and isolated(caller). I tried
to keep that out of the patch as much as possible, though.
This library uses GenericMetadataBuilder with a ReaderWriter that can read data and resolve pointers from MachO files, and emit a JSON representation of a dylib containing the built metadata.
We use LLVM's binary file readers to parse the MachO files and resolve fixups so we can follow pointers. This code is somewhat MachO specific, but could be generalized to other formats that LLVM supports.
rdar://116592577
Create a version of the metadata specialization code which is abstracted so that it can work in different contexts, such as building specialized metadata from dylibs on disk rather than from inside a running process.
The GenericMetadataBuilder class is templatized on a ReaderWriter. The ReaderWriter abstracts out everything that's different between in-process and external construction of this data. Instead of reading and writing pointers directly, the builder calls the ReaderWriter to resolve and write pointers. The ReaderWriter also handles symbol lookups and looking up other Swift types by name.
This is accompanied by a simple implementation of the ReaderWriter which works in-process. The abstracted calls to resolve and write pointers are implemented using standard pointer dereferencing.
A new SWIFT_DEBUG_VALIDATE_EXTERNAL_GENERIC_METADATA_BUILDER environment variable uses the in-process ReaderWriter to validate the builder by running it in parallel with the existing metadata builder code in the runtime. When enabled, the GenericMetadataBuilder is used to build a second copy of metadata built by the runtime, and the two are compared to ensure that they match. When this environment variable is not set, the new builder code is inactive.
The builder is incomplete, and this initial version only works on structs. Any unsupported type produces an error, and skips the validation.
rdar://116592420
I am doing this in preparation for adding options to SILParameterInfo/
SILResultInfo that state that a parameter/result is transferring. Even though I
could have just introduced a new bit here, I instead streamlined the interface
of SILParameterInfo/SILResultInfo to use an OptionSet instead of individual bits
to make it easier to add new flags here. The reason why it is easier is that
along API (e.x.: function argument) boundaries one does not have to marshal each
field or pass each field. Instead one can just pass the whole OptionSet as an
opaque thing. Using this I was able to change serialization/deserialization of
SILParameterInfo/SILResultInfo so that one does not need to update them if one
adds new fields!
The reason why I am doing this for both SILParameterInfo/SILResultInfo in the
same commit is because they share code in the demangler that I did not want to
have to duplicate in an intervening commit. By changing them both at the same
type, I didn't have to change anything without an actual need to.
I am doing this in a separate commit from adding transferring support so I can
validate correctness using the tests for the options already supported
(currently only differentiability).
rdar://119329771
This layout allows adding pre-specializations for trivial types that have a different size, but the same stride. This is especially useful for collections, where the stride is the important factor.
Function body macros allow one to introduce a function body for a
particular function, either providing a body for a function that
doesn't have one, or wholesale replacing the body of a function that
was written with a new one.
Using symbolic references instead of a text based mangling avoids the
expensive type descriptor scan when objective c protocols are requested.
rdar://111536582
The more awkward setup with pushGenericParams()/popGenericParams()
is required so that when demangling requirements and field types
we get the correct pack-ness for each type parameter. The pack-ness
is encoded as part of the generic signature, and not as part of
the type parameter mangling itself.
Fixes the ASTDemangler issue from rdar://problem/115459973.
When appending to an empty `CharVector`, we were inadvertently relying
on `memcpy(NewObjects, NULL, 0)` being a no-op, whereas in practice
the C standard says it's UB because of the `NULL` pointer.
Fix by only calling `memcpy()` if we actually have bytes to copy.
rdar://114447171
The old behavior was only correct when building substituted types,
ie, if createTupleType() was never called with a pack expansion type.
This was the case in the runtime's MetadataLookup which applies
substitutions to an interface type to ultimately construct metadata
for a fully-concrete type, but not in the ASTDemangler, where we
actually build interface types.
Since TypeDecoder doesn't have any way to query the kind of type
it just built, let's just instead make this decision inside the
implementation of the type builder concept.
Fixes https://github.com/apple/swift/issues/67322.
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
We clear the NodeFactory to prevent unbounded buildup of allocated memory, but this is done too eagerly. In particular, normalizeReflectionName can end up clearing the factory while the calling code is still using nodes that were allocated from it.
To keep peak memory usage low while avoiding this problem, we introduce a checkpoint mechanism in NodeFactory. A checkpoint can be pushed and then subsequently popped. When a checkpoint is popped, only the nodes allocated since the checkpoint was pushed are invalidated and the memory reclaimed. This allows us to quickly clear short-lived nodes like those created in normalizeReflectionName, while preserving longer-lived nodes used in code calling it. Uses of clearNodeFactory are replaced with this checkpoint mechanism.
rdar://106547092
This executable is intended to be installed in the toolchain and act as
an executable compiler plugin just like other 'macro' plugins.
This plugin server has an optional method 'loadPluginLibrary' that
dynamically loads dylib plugins.
The compiler has a newly added option '-external-plugin-path'. This
option receives a pair of the plugin library search path (just like
'-plugin-path') and the corresponding "plugin server" path, separated
by '#'. i.e.
-external-plugin-path
<plugin library search path>#<plugin server executable path>
For exmaple, when there's a macro decl:
@freestanding(expression)
macro stringify<T>(T) -> (T, String) =
#externalMacro(module: "BasicMacro", type: "StringifyMacro")
The compiler look for 'libBasicMacro.dylib' in '-plugin-path' paths,
if not found, it falls back to '-external-plugin-path' and tries to find
'libBasicMacro.dylib' in them. If it's found, the "plugin server" path
is launched just like an executable plugin, then 'loadPluginLibrary'
method is invoked via IPC, which 'dlopen' the library path in the plugin
server. At the actual macro expansion, the mangled name for
'BasicMacro.StringifyMacro' is used to resolve the macro just like
dylib plugins in the compiler.
This is useful for
* Isolating the plugin process, so the plugin crashes doesn't result
the compiler crash
* Being able to use library plugins linked with other `swift-syntax`
versions
rdar://105104850
Remove the forward declaration for `llvm::raw_ostream` as this creates
ambiguity for `__swift::__runtime::llvm::StringRef` and
`llvm::StringRef` as now `llvm` is made visible to the namespace lookup
rules.
Extend the name mangling scheme for macro expansions to cover attached
macros, and use that scheme for the names of macro expansions buffers.
Finishes rdar://104038303, stabilizing file/buffer names for macro
expansion buffers.
Use the name mangling scheme we've devised for macro expansions to
back the implementation of the macro expansion context's
`getUniqueName` operation. This way, we guarantee that the names
provided by macro expansions don't conflict, as well as making them
demangleable so we can determine what introduced the names.
