'load-plugin-library', 'load-plugin-executable', '-plugin-path' and
'-external-plugin-path' should be searched in the order they are
specified in the arguments.
Previously, for example '-plugin-path' used to precede
'-external-plugin-path' regardless of the position in the arguments.
* Argument to '-load-plugin-library' now must have a filename that's
'{libprefix}{modulename}.{sharedlibraryextension}'
* Load '-load-plugin-library' plugins are now lazily loaded in
'CompilerPluginLoadRequest'
* Remove ASTContext.LoadedSymbols cache because they are cached by
'ExternalMacroDefinitionRequest' anyway
* `-load-plugin-executable` format validation is now in
'ParseSearchPathArgs'
This executable is intended to be installed in the toolchain and act as
an executable compiler plugin just like other 'macro' plugins.
This plugin server has an optional method 'loadPluginLibrary' that
dynamically loads dylib plugins.
The compiler has a newly added option '-external-plugin-path'. This
option receives a pair of the plugin library search path (just like
'-plugin-path') and the corresponding "plugin server" path, separated
by '#'. i.e.
-external-plugin-path
<plugin library search path>#<plugin server executable path>
For exmaple, when there's a macro decl:
@freestanding(expression)
macro stringify<T>(T) -> (T, String) =
#externalMacro(module: "BasicMacro", type: "StringifyMacro")
The compiler look for 'libBasicMacro.dylib' in '-plugin-path' paths,
if not found, it falls back to '-external-plugin-path' and tries to find
'libBasicMacro.dylib' in them. If it's found, the "plugin server" path
is launched just like an executable plugin, then 'loadPluginLibrary'
method is invoked via IPC, which 'dlopen' the library path in the plugin
server. At the actual macro expansion, the mangled name for
'BasicMacro.StringifyMacro' is used to resolve the macro just like
dylib plugins in the compiler.
This is useful for
* Isolating the plugin process, so the plugin crashes doesn't result
the compiler crash
* Being able to use library plugins linked with other `swift-syntax`
versions
rdar://105104850
This adds the following four new options:
- `-windows-sdk-root`
- `-windows-sdk-version`
- `-visualc-tools-root`
- `-visualc-tools-version`
Together these options make one the master of Windows SDK selection for
the Swift compilation. This is important as now that the injection is
no longer done by the user, we need to ensure that we have enough
control over the paths so that the synthesized overlay is going to map
the files to the proper location.
Add a compiler option `-load-plugin-executable <path>#<module names>`.
Where '<path>' is a path to a plugin executable, '<module-name>' is a
comma-separated module names the plugin provides.
Nothing is using it at this point. Actual plugin infratructure are
introduced in follow-up commits
Introduce `-plugin-path <path>` to add a search path where we will look
for compiler plugins. When resolving an external macro definition, look
for libraries in these search paths whose names match the module name
of the macro.
Implements rdar://105095761.
Introduces a concept of a dependency scanning action context hash, which is used to select an instance of a global dependency scanning cache which gets re-used across dependency scanning actions.
Allow user-defined macros to be loaded from dynamic libraries and evaluated.
- Introduce a _CompilerPluginSupport module installed into the toolchain. Its `_CompilerPlugin` protocol acts as a stable interface between the compiler and user-defined macros.
- Introduce a `-load-plugin-library <path>` attribute which allows users to specify dynamic libraries to be loaded into the compiler.
A macro library must declare a public top-level computed property `public var allMacros: [Any.Type]` and be compiled to a dynamic library. The compiler will call the getter of this property to obtain and register all macros.
Known issues:
- We current do not have a way to strip out unnecessary symbols from the plugin dylib, i.e. produce a plugin library that does not contain SwiftSyntax symbols that will collide with the compiler itself.
- `MacroExpansionExpr`'s type is hard-coded as `(Int, String)`. It should instead be specified by the macro via protocol requirements such as `signature` and `genericSignature`. We need more protocol requirements in `_CompilerPlugin` to handle this.
- `dlopen` is not secure and is only for prototyping use here.
Friend PR: apple/swift-syntax#1022
Two paths missed setting up overlays:
- `CompletionInstance` when checking files from dependencies
- `SwiftASTManager` when reading in files that it would later replace
all inputs with
(1) would cause the AST context not to be re-used, even though nothing
had changed. (2) caused all non-completion functionality to fail for any
symbols within files only specified by the overlay.
Resolves rdar://85508213.
The idea behind storing a StringRef was to reduce the memory footprint because we made the assumption that a `ModuleSearchPath` always outlives the `SearchPathOptions` it was created from. What I did not consider was that the path was referencing into a `std::vector`, which could get resized, thus invaliding the memory the `ModuleSearchPath`’s `StringRef` was pointing to, causing memory corruption.
To fix this, store the path string inisde the `ModuleSearchPath` itself. Since we store a `ModuleSearchPath` for every file inside that module search path in the `LookupTable`, by itself this would cause a new copy of the path to be stored for every file inside a module search path. To avoid this, make `ModuleSearchPath` ref counted and only store a reference to one shared `ModuleSearchPath` entry in the lookup table.
rdar://88888679
When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.
To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
Module1.framework
searchPath2/
Module1.framework
Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
We noticed some Swift clients rely on the serialized search paths in the module to
find dependencies and droping these paths altogether can lead to build failures like
rdar://85840921.
This change teaches the serialization to obfuscate the search paths and the deserialization
to recover them. This allows clients to keep accessing these paths without exposing
them when shipping the module to other users.
This commit adds a new frontend flag that applies debug path prefixing to the
paths serialized in swiftmodule files. This makes it possible to use swiftmodule
files that have been built on different machines by applying the inverse map
when debugging, in a similar fashion to source path prefixing.
The inverse mapping in LLDB will be handled in a follow up PR.
Second pass at #39138
Tests updated to handle windows path separators.
This reverts commit f5aa95b381.
This is intended to be used from LLDB to apply the remappings
specified in target.source-map to remap any serialized
Swiftmodule search paths that were prefixed using
`-prefix-serialized-debugging-options`.
Serialize the canonical name of the SDK used when building a swiftmodule
file and use it to ensure that the swiftmodule file is loaded only with
the same SDK. The SDK name must be passed down from the frontend.
This will report unsupported configurations like:
- Installing roots between incompatible SDKs without deleting the
swiftmodule files.
- Having multiple targets in the same project using different SDKs.
- Loading a swiftmodule created with a newer SDK (and stdlib) with an
older SDK.
All of these lead to hard to investigate deserialization failures and
this change should detect them early, before reaching a deserialization
failure.
rdar://78048939
This scanning mode allows swift-driver to query module dependencies in a batch
and in a more granular way. In short term, it could help solve a problem that
clang module dependencies may vary if target triple changes. In a longer term,
we could break a holistic dependencies graph into smaller pieces for better caching
and reusing.
This change doesn't include the implementation of using the specified scanner
arguments to set up Clang dependencies scanner. It will come in later commits.
-compile-module-from-interface action now takes arguments of -candidate-module-file.
If one of the candidate module files is up-to-date, the action emits a forwarding
module pointing to the candidate module instead of building a binary module.
Instead of taking paths of Swift module files from front-end command line
arguments, we should take a JSON file specifying details of explicit modules.
The advantages is (1) .swiftdoc and .swiftsourceinfo can be associated
with a .swiftmodule file, and (2) module names are explicitly used as
keys in the JSON input so we don't need to eagerly deserialize a .swiftmodule
file to collect the module name.
- No need to hash input values first
- Pass many values to a single hash_combine to save on intermediates
- Use hash_combine_range instead of a loop of hash_combines
No functionality change.
The point of this flag is to avoid re-checking the whole SDK on every
build, since under many circumstances you can assume it hasn't
changed. That worked for Clang modules, but Swift cached modules also
end up with dependencies that shouldn't be updated.
rdar://problem/53279521
Replaces SearchPathOptions::RuntimeLibraryImportPath with an equivalent std::vector of paths. Also reimplements SearchPathOptions::SkipRuntimeLibraryImportPaths to cause the list of runtime library import paths to be empty, rather than exiting early from SerializedModuleLoader::findModule().
Adds the -vfsoverlay frontend option that enables the user to pass
VFS overlay YAML files to Swift. These files define a (potentially
many-layered) virtual mapping on which we predicate a VFS.
Switch all input-based memory buffer reads in the Frontend to the new
FileSystem-based approach.
- Add CompilerInvocation::getPCHHash
This will be used when creating a unique filename for a persistent
precompiled bridging header.
- Automatically generate and use a precompiled briding header
When we're given both -import-objc-header and -pch-output-dir
arguments, we will try to:
- Validate what we think the PCH filename should be for the bridging
header, based on the Swift PCH hash and the clang module hash.
- If we're successful, we'll just use it.
- If it's out of date or something else is wrong, we'll try to
emit it.
- This gives us a single filename which we can `stat` to check for the
validity of our code completion cache, which is keyed off of module
name, module filename, and module file age.
- Cache code completion results from imported modules
If we just have a single .PCH file imported, we can use that file as
part of the key used to cache declarations in a module. Because
multiple files can contribute to the __ObjC module, we've always given
it the phony filename "<imports>", which never exists, so `stat`-ing it
always fails and we never cache declarations in it.
This is extremely problematic for projects with huge bridging headers.
In the case where we have a single PCH import, this can bring warm code
completion times down to about 500ms from over 2-3s, so it can provide a
nice performance win for IDEs.
- Add a new test that performs two code-completion requests with a bridging header.
- Add some -pch-output-dir flags to existing SourceKit tests that import a bridging
header.
rdar://problem/31198982
This has the effect of propagating the search path to the clang importer as '-iframework'.
It doesn't affect whether a swift module is treated as system or not, this can be done as follow-up enhancement.
Now that we can pick up search paths from frameworks (necessary to debug
them properly), we can end up with exponential explosions leading to the
same search path coming up thousands of times, which destroys compilation
time /and/ debugger responsiveness. This is already hitting people with
frameworks compiled for app extensions (due to a mistaken approximation
of whether or not something is a framework), but we're turning this on for
all frameworks in the immediate future.
rdar://problem/20291720
Swift SVN r27087
...and then honor them.
While here, make -l a little more flexible (see interpret_with_options test).
rdar://problem/17830826, which unblocks the LLDB feature for the same.
Swift SVN r24033
This keeps us from having to deal with fat swiftmodules for now.
In the long run we're hoping to solve this problem with build configurations,
so that a single module file can support multiple architectures.
(See <rdar://problem/15056323>)
<rdar://problem/15204953>
Swift SVN r13135
