LLVM deprecated, renamed, and removed a bunch of APIs. This patch
contains a lot of the changes needed to deal with that.
The SetVector type changed the template parameters.
APInt updated multiple names, countPopulation became popcount,
getAllOnesValue became getAllOnes, getNullValue became getZero, etc...
Clang type nullability check stopped taking a clang AST context.
The LLVM IRGen Function type stopped exposing basic block list directly,
but gained enough API surface that the translation isn't too bad.
(GenControl.cpp, LLVMMergeFunctions.cpp)
llvm::Optional had a transform function. That was being used in a couple
of places, so I've added a new implementation under STLExtras that
transforms valid optionals, otherwise it returns nullopt.
Ensure that context descriptor pointers are signed in the runtime by putting the ptrauth_struct attribute on the types.
We use the new __builtin_ptrauth_struct_key/disc to conditionally apply ptrauth_struct to TrailingObjects based on the signing of the base type, so that pointers to TrailingObjects get signed when used with a context descriptor pointer.
We add new runtime entrypoints that take signed pointers where appropriate, and have the compiler emit calls to the new entrypoints when targeting a sufficiently new OS.
rdar://111480914
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Setting closure and local discriminators depends on an in-order walk
of the AST. For macros, it was walking into both macro expansions and
arguments. However, this doesn't work well with lazy macro expansions,
and could result in some closures/local variables not getting
discriminators set at all.
Make the assignment of discriminators only walk macro arguments, and
then lazily assign discriminators for anything within a macro
expansion or in ill-formed code. This replaces the single global "next
autoclosure discriminator" scheme with a per-DeclContext scheme, that
is more reliable/robust, although it does mean that discriminators
of closures and locals within macro expansions are dependent on
ordering. That shouldn't matter, because these are local values.
Fixes rdar://108682196.
Suggesting the addition of an `@available` attribute on Linux and Windows is
inapprorpriate given that Swift is not ABI stable on those platforms.
Resolves rdar://107387133
* Factor out ASTContext plugin loading to newly introduced 'PluginLoader'
* Insert 'DependencyTracker' to 'PluginLoader'
* Add dependencies right before loading the plugins
rdar://104938481
For a `@Testable` import in program source, if a Swift interface dependency is discovered, and has an adjacent binary `.swiftmodule`, open up the module, and pull in its optional dependencies. If an optional dependency cannot be resolved on the filesystem, fail silently without raising a diagnostic.
The only state `getSwiftName` uses is its argument, and can be made a free function. Of
note the inverse operation, `getKnownFoundationEntity`, is also a free function. This
removes the requirement that callers have an `ASTContext` instance.
(cherry-picked from commit fb524c0b86)
`LoadedLibraryPlugin` is currently just a wrapper of `void` pointer from
`dlopen`. This will be convenient for abstracting platform specific
dynamic linrary handling.
Using a virutal output backend to capture all the outputs from
swift-frontend invocation. This allows redirecting and/or mirroring
compiler outputs to multiple location using different OutputBackend.
As an example usage for the virtual outputs, teach swift compiler to
check its output determinism by running the compiler invocation
twice and compare the hash of all its outputs.
Virtual output will be used to enable caching in the future.
* Argument to '-load-plugin-library' now must have a filename that's
'{libprefix}{modulename}.{sharedlibraryextension}'
* Load '-load-plugin-library' plugins are now lazily loaded in
'CompilerPluginLoadRequest'
* Remove ASTContext.LoadedSymbols cache because they are cached by
'ExternalMacroDefinitionRequest' anyway
* `-load-plugin-executable` format validation is now in
'ParseSearchPathArgs'
This executable is intended to be installed in the toolchain and act as
an executable compiler plugin just like other 'macro' plugins.
This plugin server has an optional method 'loadPluginLibrary' that
dynamically loads dylib plugins.
The compiler has a newly added option '-external-plugin-path'. This
option receives a pair of the plugin library search path (just like
'-plugin-path') and the corresponding "plugin server" path, separated
by '#'. i.e.
-external-plugin-path
<plugin library search path>#<plugin server executable path>
For exmaple, when there's a macro decl:
@freestanding(expression)
macro stringify<T>(T) -> (T, String) =
#externalMacro(module: "BasicMacro", type: "StringifyMacro")
The compiler look for 'libBasicMacro.dylib' in '-plugin-path' paths,
if not found, it falls back to '-external-plugin-path' and tries to find
'libBasicMacro.dylib' in them. If it's found, the "plugin server" path
is launched just like an executable plugin, then 'loadPluginLibrary'
method is invoked via IPC, which 'dlopen' the library path in the plugin
server. At the actual macro expansion, the mangled name for
'BasicMacro.StringifyMacro' is used to resolve the macro just like
dylib plugins in the compiler.
This is useful for
* Isolating the plugin process, so the plugin crashes doesn't result
the compiler crash
* Being able to use library plugins linked with other `swift-syntax`
versions
rdar://105104850
This modifies the ClangImporter to introduce an opaque placeholder
representation for forward declared Objective-C interfaces and
protocols when imported into Swift.
In the compiler, the new functionality is hidden behind a frontend
flag -enable-import-objc-forward-declarations, and is on by default
for language mode >6.
The feature is disabled entirely in LLDB expression evaluation / Swift
REPL, regardless of language version.
Previously enum AccessLimitKind was
added to distinguish access scopes b/t package and public while keeping
DeclContext null but it proved to be too limiting. This PR creates package specific entries for DeclContext and
ASTHierarchy. It create a new class PackageUnit that can be set as the parent DeclContext of ModuleDecl. This PR
contains addition of such entries but not the use of them; the actual use of them will be in the upcoming PRs.
Resolves rdar://106155600
Executable compiler plugins are programs invoked by the host compiler
and communicate with the host with IPC via standard IO (stdin/stdout.)
Each message is serialized in JSON, prefixed with a header which is a
64bit little-endian integer indicating the size of the message.
* Basic/ExecuteWithPipe: External program invocation. Lik
llvm::sys::ExecuteNoWait() but establishing pipes to the child's
stdin/stdout
* Basic/Sandbox: Sandboxed execution helper. Create command line
arguments to be executed in sandbox environment (similar to SwiftPM's
pluging sandbox)
* AST/PluginRepository: ASTContext independent plugin manager
* ASTGen/PluginHost: Communication with the plugin. Messages are
serialized by ASTGen/LLVMJSON
rdar://101508815
Instead of mangling class template specializations with the prefix "__CxxTemplateInst," simply set the decl name as the class templates plus the types that it is specialized on (so `vector<Int>` rather than `__CxxTemplateInstNSt3__16vectorIi...`).
This is mainly to improve diagnostics. As a side effect of this change, if anyone copies the name of a class template specializaiton from an error/warning and uses it in source code, the compiler will error (that class templates aren't available in swift) rather than silently passing only to cause serailization failures down the road.
This changes the scanner's behavior to "resolve" a discovered module's dependencies to a set of Module IDs: module name + module kind (swift textual, swift binary, clang, etc.).
The 'ModuleDependencyInfo' objects that are stored in the dependency scanner's cache now carry a set of kind-qualified ModuleIDs for their dependencies, in addition to unqualified imported module names of their dependencies.
Previously, the scanner's internal state would cache a module dependnecy as having its own set of dependencies which were stored as names of imported modules. This led to a design where any time we needed to process the dependency downstream from its discovery (e.g. cycle detection, graph construction), we had to query the ASTContext to resolve this dependency's imports, which shouldn't be necessary. Now, upon discovery, we "resolve" a discovered dependency by executing a lookup for each of its imported module names (this operation happens regardless of this patch) and store a fully-resolved set of dependencies in the dependency module info.
Moreover, looking up a given module dependency by name (via `ASTContext`'s `getModuleDependencies`) would result in iterating over the scanner's module "loaders" and querying each for the module name. The corresponding modules would then check the scanner's cache for a respective discovered module, and if no such module is found the "loader" would search the filesystem.
This meant that in practice, we searched the filesystem on many occasions where we actually had cached the required dependency, as follows:
Suppose we had previously discovered a Clang module "foo" and cached its dependency info.
-> ASTContext.getModuleDependencies("foo")
--> (1) Swift Module "Loader" checks caches for a Swift module "foo" and doesn't find one, so it searches the filesystem for "foo" and fails to find one.
--> (2) Clang Module "Loader" checks caches for a Clang module "foo", finds one and returns it to the client.
This means that we were always searching the filesystem in (1) even if we knew that to be futile.
With this change, queries to `ASTContext`'s `getModuleDependencies` will always check all the caches first, and only delegate to the scanner "loaders" if no cached dependency is found. The loaders are then no longer in the business of checking the cached contents.
To handle cases in the scanner where we must only lookup either a Swift-only module or a Clang-only module, this patch splits 'getModuleDependencies' into an alrady-existing 'getSwiftModuleDependencies' and a newly-added 'getClangModuleDependencies'.
When doing solver-based cursor info, we’ll type check the expression in question using a normal `typeCheckASTNodeAtLoc` call (not in code completion mode) and listen for any solutions that were produced during the type check.
A macro declaration contains the external module and type name of the
macro's implementation. Use that information to find the macro type
(via its type metadata accessor) in a loaded plugin, so we no longer
require the "allMacros" array. Instead, each macro implementation type
must be a public struct.
Since we are now fully dependent on the macro declaration for
everything about a macro except its kind, remove most of the query
infrastructure for compiler plugins.
Replace the macro registration scheme based on the allMacros array with
Allow more than one macro plugin to introduce a macro with the same
name, and let the constraint solver figure out which one to call. Also
eliminates a potential use-after-free if we somehow find additional
compiler plugins to load after having expanded a macro.
Type check user-defined macros plugins with user-provided type signatures.
Also, load plugin libraries with `RTLD_LOCAL` instead of `RTLD_GLOBAL` to prevent symbol collision between plugins. `llvm::sys::DynamicLibrary` only supports `RTLD_GLOBAL` so we use the plain `dlopen` instead. This does not work on Windows and needs to be fixed.
Friend PR: apple/swift-syntax#1042
Allow user-defined macros to be loaded from dynamic libraries and evaluated.
- Introduce a _CompilerPluginSupport module installed into the toolchain. Its `_CompilerPlugin` protocol acts as a stable interface between the compiler and user-defined macros.
- Introduce a `-load-plugin-library <path>` attribute which allows users to specify dynamic libraries to be loaded into the compiler.
A macro library must declare a public top-level computed property `public var allMacros: [Any.Type]` and be compiled to a dynamic library. The compiler will call the getter of this property to obtain and register all macros.
Known issues:
- We current do not have a way to strip out unnecessary symbols from the plugin dylib, i.e. produce a plugin library that does not contain SwiftSyntax symbols that will collide with the compiler itself.
- `MacroExpansionExpr`'s type is hard-coded as `(Int, String)`. It should instead be specified by the macro via protocol requirements such as `signature` and `genericSignature`. We need more protocol requirements in `_CompilerPlugin` to handle this.
- `dlopen` is not secure and is only for prototyping use here.
Friend PR: apple/swift-syntax#1022
Intro ASTContext::setIgnoreAdjacentModules to change module loading to
accept load only resilient modules from their swiftinterfaces, ignoring
the adjacent module and any silencing swiftinterfaces errors.
