This executable is intended to be installed in the toolchain and act as
an executable compiler plugin just like other 'macro' plugins.
This plugin server has an optional method 'loadPluginLibrary' that
dynamically loads dylib plugins.
The compiler has a newly added option '-external-plugin-path'. This
option receives a pair of the plugin library search path (just like
'-plugin-path') and the corresponding "plugin server" path, separated
by '#'. i.e.
-external-plugin-path
<plugin library search path>#<plugin server executable path>
For exmaple, when there's a macro decl:
@freestanding(expression)
macro stringify<T>(T) -> (T, String) =
#externalMacro(module: "BasicMacro", type: "StringifyMacro")
The compiler look for 'libBasicMacro.dylib' in '-plugin-path' paths,
if not found, it falls back to '-external-plugin-path' and tries to find
'libBasicMacro.dylib' in them. If it's found, the "plugin server" path
is launched just like an executable plugin, then 'loadPluginLibrary'
method is invoked via IPC, which 'dlopen' the library path in the plugin
server. At the actual macro expansion, the mangled name for
'BasicMacro.StringifyMacro' is used to resolve the macro just like
dylib plugins in the compiler.
This is useful for
* Isolating the plugin process, so the plugin crashes doesn't result
the compiler crash
* Being able to use library plugins linked with other `swift-syntax`
versions
rdar://105104850
* Move `add_pure_swift_host_library()` from lib/CMakeLists.txt to
AddPureSwift.cmake so that code outside `lib` can use it
* Add `add_pure_swift_host_tool()` function to make a pure Swift
host executable target (for future usages)
* Specify depending `SwiftSyntax` modules explicitly because not all
Swift libraries uses all SwiftSyntax libraries
CMake has a misfeature wherein it tries to de-duplicate command line options.
This caused it to omit an `-Xfrontend` option when using
`add_pure_swift_host_library`.
rdar://106547267
CMake has a misfeature wherein it tries to de-duplicate command line
options. This caused it to omit an `-Xfrontend` option when using
`add_pure_swift_host_library`.
rdar://106547267
For future usage from other host libraries written in Swift
For CMake:
* Explicitly specify LINKER_LANGAGE to CXX in existing components so
that 'swiftc' is not used when linking with 'swiftASTGen'
* Add 'EMIT_MODULE' argument to 'add_pure_swift_host_library' to emit
.swiftmodule usable from other Swift libraries.
Once the API has gone through Swift Evolution, we will want to implicitly
import the _Backtracing module. Add code to do that, but set it to off
by default for now.
rdar://105394140
The dependency of the Observation library on the observation macro
plugin introduces a direct dependency of a target library on a host
library, (lib -> stdlib) that we'd like to avoid. Instead, make the
Swift frontend binary depend on the macro plugin libraries that we
build, so that it's the complete host-side stack.
Refactor the build support for ASTGen and ObservationMacros. Both have
nearly-identical copies of a bunch of messy, workaround-laden CMake
code to build "pure" Swift host libraries using CMake's Swift support.
Instead, introduce `add_pure_swift_host_library` to centralize all of
the hacks to build a host library that's all Swift, using CMake's
support directly (rather than custom commands), and link against the
swift-syntax stack. Switch ASTGen directly over to this.
Add `add_swift_macro_library` on top of this, to make it easy to
create a macro library and install it into the appopriate plugin
directory. Switch ObservationMacros over to this.
IDE/Refactoring had dependencies to libswiftIndex, but libswiftIndex
also depends on libswiftIDE (SourceEntityWalker, etc.)
To break libswiftIndex <-> libswiftIDE dependency cycle, move
"refactoring" related files to a new library 'libswiftRefactoring'
rdar://101692282
Allow user-defined macros to be loaded from dynamic libraries and evaluated.
- Introduce a _CompilerPluginSupport module installed into the toolchain. Its `_CompilerPlugin` protocol acts as a stable interface between the compiler and user-defined macros.
- Introduce a `-load-plugin-library <path>` attribute which allows users to specify dynamic libraries to be loaded into the compiler.
A macro library must declare a public top-level computed property `public var allMacros: [Any.Type]` and be compiled to a dynamic library. The compiler will call the getter of this property to obtain and register all macros.
Known issues:
- We current do not have a way to strip out unnecessary symbols from the plugin dylib, i.e. produce a plugin library that does not contain SwiftSyntax symbols that will collide with the compiler itself.
- `MacroExpansionExpr`'s type is hard-coded as `(Int, String)`. It should instead be specified by the macro via protocol requirements such as `signature` and `genericSignature`. We need more protocol requirements in `_CompilerPlugin` to handle this.
- `dlopen` is not secure and is only for prototyping use here.
Friend PR: apple/swift-syntax#1022
These libraries formed a strongly connected component in the CMake build graph. The weakest link I could find was from IDE to FrontendTool and Frontend, which was necessitated by the `CompileInstance` class (https://github.com/apple/swift/pull/40645). I moved a few files out of IDE into a new IDETools library to break the cycle.
The relationship between the code in these two libraries was fundamentally circular, indicating that they should not have been split. With other changes that I'm making to remove circular dependencies from the CMake build graph I eventually uncovered that these two libraries were required to link each other circularly, but that had been hidden by other cycles in the build graph previously.
Replace Attr.def with a gyb file that reads from swift-syntax
to automatically generate the attribute nodes.
For now, only the Swift attributes have been migrated. SIL attributes
can be defined manually in Attr.def.gyb. To add new Swift attributes,
a paired pull request to swift-syntax is now necessary.
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
Moved all the threading code to one place. Added explicit support for
Darwin, Linux, Pthreads, C11 threads and Win32 threads, including new
implementations of Once for Linux, Pthreads, C11 and Win32.
rdar://90776105
Create a new diagnostics serialization library, splitting out of
swiftAST to reduce the overheads for building the tooling for
cross-compiling the toolchain. This should reduce the build time for
swift-serialize-diagnostics to enable cross-compilation.
Move the source files from ParserSIL into the SIL library and remove the
ParserSIL library. ParsersSIL doesn't need to be its own library and this change will
simplify our builds.
Adds a tool `swift-symbolgraph-extract` that reads an existing Swift
module and prints a platform- and language-agnostic JSON description of
the module, primarly for documentation.
Adds a small sub-library `SymbolGraphGen` which houses the core
implementation for collecting relevant information about declarations.
The main entry point is integrated directly into the driver as a mode:
the tool is meant to be run outside of the normal edit-compile-run/test
workflow to avoid impacting build times.
Along with common options for other tools, unique options include
`pretty-print` for debugging, and a `minimum-access-level` options for
including internal documentation.
A symbol graph is a directed graph where the nodes are symbols in a
module and the edges are relationships between them. For example, a
`struct S` may have a member `var x`. The graph would have two nodes for
`S` and `x`, and one "member-of" relationship edge. Other relationship
kinds include "inherits-from" or "conforms to". The data format for a
symbol graph is still under development and may change without notice
until a specificiation and versioning scheme is published.
Various aspects about a symbol are recorded in the nodes, such as
availability, documentation comments, or data needed for printing the
shapes of declarations without having to understand specifics about the
langauge.
Implicit and public-underscored stdlib declarations are not included by
default.
rdar://problem/55346798
Separate out the host build of SwiftRemoteMirror from the "target" build
(the host for the standard library may be different from the host for
the compiler). Restructure the build to ensure that we use the correct
compiler for building the SwiftRemoteMirror for the host. This fixes a
build issue when building for Linux AArch64.
We would build two copies of swiftReflection, one for the host and one
for the target. However, the compiler configuration cannot be swapped
out in the middle as we were attempting to do. This would result in a
failure to build sometimes due to the missing dependency on the wanted
compiler. More importantly, it would also use the wrong compiler when
building the library. Although this duplicates the source paths,
correctness is preserved.
This refactors DWARFImporter to become a part of ClangImporter, since
it needs access to many of its implementation details anyway. The
DWARFImporterDelegate is just another mechanism for deserializing
Clang ASTs and once we have a Clang AST, the processing is effectively
the same.
Since the generated headers are in 'include/swift/Parse' they are globally accessible so make sure they are created early on.
If these get moved inside 'lib/Parse' then we can remove this dependency setting.
Instead of creating syntax nodes directly, modify the parser to invoke an abstract interface 'SyntaxParseActions' while it is parsing the source code.
This decouples the act of parsing from the act of forming a syntax tree representation.
'SyntaxTreeCreator' is an implementation of SyntaxParseActions that handles the logic of creating a syntax tree.
To enforce the layering separation of parsing and syntax tree creation, a static library swiftSyntaxParse is introduced to compose the two.
This decoupling is important for introducing a syntax parser library for SwiftSyntax to directly access parsing.
When debugging Objective-C or C++ code on Darwin, the debug info
collected by dsymutil in the .dSYM bundle is entirely
self-contained. It is possible to debug a program, set breakpoints and
print variables even without having the complete original source code
or a matching SDK available. With Swift, this is currently not the
case. Even though .dSYM bundles contain the binary .swiftmodule for
all Swift modules, any Clang modules that the Swift modules depend on,
still need to be imported from source to even get basic LLDB
functionality to work. If ClangImporter fails to import a Clang
module, effectively the entire Swift module depending on it gets
poisoned.
This patch is addressing this issue by introducing a ModuleLoader that
can ask queries about Clang Decls to LLDB, since LLDB knows how to
reconstruct Clang decls from DWARF and clang -gmodules producxes full
debug info for Clang modules that is embedded into the .dSYM budle.
This initial version does not contain any advanced functionality at
all, it merely produces an empty ModuleDecl. Intertestingly, even this
is a considerable improvement over the status quo. LLDB can now print
Swift-only variables in modules with failing Clang depenecies, and
becuase of fallback mechanisms that were implemented earlier, it can
even display the contents of pure Objective-C objects that are
imported into Swift. C structs obviously don't work yet.
rdar://problem/36032653
...finally breaking the dependency of Parse on Sema.
There are still some unfortunate dependencies here -- Xi's working on
getting /AST/ not dependent on Sema -- but this is a step forward.
It is a little strange that parseIntoSourceFile is in ParseSIL, and
therefore that that's still a dependency for anyone trying to, well,
parse. However, nearly all clients that parse want to type-check as
well, and that requires Sema, Serialization, and the ClangImporter...
and Serialization and SIL currently require each other as well
(another circular dependency). So it's not actively causing us trouble
right now.
