Now that ASTGen should be able to generate most Swift code. Let's
remove "legacy parser" call-in, and remove the unhealthy cyclic
dependency between lib/Parse and ASTGen.
Separate swift-syntax libs for the compiler and for the library plugins.
Compiler communicates with library plugins using serialized messages
just like executable plugins.
* `lib/swift/host/compiler/lib_Compiler*.dylib`(`lib/CompilerSwiftSyntax`):
swift-syntax libraries for compiler. Library evolution is disabled.
* Compiler (`ASTGen` and `swiftIDEUtilsBridging`) only depends on
`lib/swift/host/compiler` libraries.
* `SwiftInProcPluginServer`: In-process plugin server shared library.
This has one `swift_inproc_plugins_handle_message` entry point that
receives a message and return the response.
* In the compiler
* Add `-in-process-plugin-server-path` front-end option, which specifies
the `SwiftInProcPluginServer` shared library path.
* Remove `LoadedLibraryPlugin`, because all library plugins are managed
by `SwiftInProcPluginServer`
* Introduce abstract `CompilerPlugin` class that has 2 subclasses:
* `LoadedExecutablePlugin` existing class that represents an
executable plugin
* `InProcessPlugins` wraps `dlopen`ed `SwiftInProcPluginServer`
* Unified the code path in `TypeCheckMacros.cpp` and `ASTGen`, the
difference between executable plugins and library plugins are now
abstracted by `CompilerPlugin`
This library uses GenericMetadataBuilder with a ReaderWriter that can read data and resolve pointers from MachO files, and emit a JSON representation of a dylib containing the built metadata.
We use LLVM's binary file readers to parse the MachO files and resolve fixups so we can follow pointers. This code is somewhat MachO specific, but could be generalized to other formats that LLVM supports.
rdar://116592577
Previously, swift-syntax libraries were built in '/CMakeLists.txt', and
were installed in '/lib/CMakeLists.txt'. Consolidate them into
'/lib/SwiftSyntax/CMakeLists.txt'. Now the swift-syntax libraries look
like any other tool libraries outside the 'lib/SwiftSyntax', as they
should.
Clang and LLVM always provide those tablegen targets, so we don't need
to check their existence. Also adding `intrinsic_gen` as a dependency.
See https://github.com/apple/swift/pull/12147
`HeaderDependencies.cpp` cannot be compiled with
`SWIFT_INCLUDE_TOOLS=OFF` because it depends on `Config.h` which is only
generated while building tools. Exclude it from the build to fix `all`
target.
Follow-up to b39dba32f3
Add SwiftIDEUtils to the modules that we distribute with the toolchain.
The motivation here is to share the build of swift-syntax across the
compilers and the LSP. Using the shared linkage this way shaves ~6MiB
off of the LSP binary.
Use FetchContent to include swift-syntax directly in swift. This can be
thought of as an `add_subdirectory` for a directory outside the root.
The default build directory will be `_deps/swiftsyntax-subbuild/`, though
the modules and shared libraries will be built in `lib/swift/host` by
passing down `SWIFT_HOST_LIBRARIES_DEST_DIR` to avoid copying them as we
were doing previously.
This fixes the test suite when performing a clean build. The libraries
were being copied to the wrong location after the generalisation changes
made earlier.
Account for import libraries and the associated layout difference on
platforms (e.g. DLLs are placed in `bin`). This is required to enable
building the macro path on Windows.
`CMAKE_SHARED_LIBRARY_PREFIX` is empty on Windows. This accounts for
that and quotes the parameter which we otherwise will see as an error
due to insufficient parameters being passed.
"Support Macros in Linux" patches was inconsistent with checking
platforms. Some only checked 'LINUX' but some matches
'LINUX|ANDROID|OPENBSD|FREEBSD'. Although I don't have tested other
platoforms than Linux at all, there's no reason to limit it to Linux.
So use the consistent check to match 'LINUX|ANDROID|OPENBSD|FREEBSD'
In Linux. Instead of setting temporary "fallback" RUNPATH, Set
LD_LIBRARY_PATH to builder's runtime when building standard library.
So we don't need to strip the temporary RUNPATH when installing.
For compiling codes required for macro support, we now need swiftc
compiler in the build machine.
Unlike Darwin OSes, where swiftCore runtime is guaranteed to be present
in /usr/lib, Linux doesn't have ABI stability and the stdlib of the
build machine is not at the specific location. So the built compiler
cannot relies on the shared object in the toolchain.
This executable is intended to be installed in the toolchain and act as
an executable compiler plugin just like other 'macro' plugins.
This plugin server has an optional method 'loadPluginLibrary' that
dynamically loads dylib plugins.
The compiler has a newly added option '-external-plugin-path'. This
option receives a pair of the plugin library search path (just like
'-plugin-path') and the corresponding "plugin server" path, separated
by '#'. i.e.
-external-plugin-path
<plugin library search path>#<plugin server executable path>
For exmaple, when there's a macro decl:
@freestanding(expression)
macro stringify<T>(T) -> (T, String) =
#externalMacro(module: "BasicMacro", type: "StringifyMacro")
The compiler look for 'libBasicMacro.dylib' in '-plugin-path' paths,
if not found, it falls back to '-external-plugin-path' and tries to find
'libBasicMacro.dylib' in them. If it's found, the "plugin server" path
is launched just like an executable plugin, then 'loadPluginLibrary'
method is invoked via IPC, which 'dlopen' the library path in the plugin
server. At the actual macro expansion, the mangled name for
'BasicMacro.StringifyMacro' is used to resolve the macro just like
dylib plugins in the compiler.
This is useful for
* Isolating the plugin process, so the plugin crashes doesn't result
the compiler crash
* Being able to use library plugins linked with other `swift-syntax`
versions
rdar://105104850
* Move `add_pure_swift_host_library()` from lib/CMakeLists.txt to
AddPureSwift.cmake so that code outside `lib` can use it
* Add `add_pure_swift_host_tool()` function to make a pure Swift
host executable target (for future usages)
* Specify depending `SwiftSyntax` modules explicitly because not all
Swift libraries uses all SwiftSyntax libraries
CMake has a misfeature wherein it tries to de-duplicate command line options.
This caused it to omit an `-Xfrontend` option when using
`add_pure_swift_host_library`.
rdar://106547267
CMake has a misfeature wherein it tries to de-duplicate command line
options. This caused it to omit an `-Xfrontend` option when using
`add_pure_swift_host_library`.
rdar://106547267
For future usage from other host libraries written in Swift
For CMake:
* Explicitly specify LINKER_LANGAGE to CXX in existing components so
that 'swiftc' is not used when linking with 'swiftASTGen'
* Add 'EMIT_MODULE' argument to 'add_pure_swift_host_library' to emit
.swiftmodule usable from other Swift libraries.
Once the API has gone through Swift Evolution, we will want to implicitly
import the _Backtracing module. Add code to do that, but set it to off
by default for now.
rdar://105394140
The dependency of the Observation library on the observation macro
plugin introduces a direct dependency of a target library on a host
library, (lib -> stdlib) that we'd like to avoid. Instead, make the
Swift frontend binary depend on the macro plugin libraries that we
build, so that it's the complete host-side stack.
Refactor the build support for ASTGen and ObservationMacros. Both have
nearly-identical copies of a bunch of messy, workaround-laden CMake
code to build "pure" Swift host libraries using CMake's Swift support.
Instead, introduce `add_pure_swift_host_library` to centralize all of
the hacks to build a host library that's all Swift, using CMake's
support directly (rather than custom commands), and link against the
swift-syntax stack. Switch ASTGen directly over to this.
Add `add_swift_macro_library` on top of this, to make it easy to
create a macro library and install it into the appopriate plugin
directory. Switch ObservationMacros over to this.
IDE/Refactoring had dependencies to libswiftIndex, but libswiftIndex
also depends on libswiftIDE (SourceEntityWalker, etc.)
To break libswiftIndex <-> libswiftIDE dependency cycle, move
"refactoring" related files to a new library 'libswiftRefactoring'
rdar://101692282
Allow user-defined macros to be loaded from dynamic libraries and evaluated.
- Introduce a _CompilerPluginSupport module installed into the toolchain. Its `_CompilerPlugin` protocol acts as a stable interface between the compiler and user-defined macros.
- Introduce a `-load-plugin-library <path>` attribute which allows users to specify dynamic libraries to be loaded into the compiler.
A macro library must declare a public top-level computed property `public var allMacros: [Any.Type]` and be compiled to a dynamic library. The compiler will call the getter of this property to obtain and register all macros.
Known issues:
- We current do not have a way to strip out unnecessary symbols from the plugin dylib, i.e. produce a plugin library that does not contain SwiftSyntax symbols that will collide with the compiler itself.
- `MacroExpansionExpr`'s type is hard-coded as `(Int, String)`. It should instead be specified by the macro via protocol requirements such as `signature` and `genericSignature`. We need more protocol requirements in `_CompilerPlugin` to handle this.
- `dlopen` is not secure and is only for prototyping use here.
Friend PR: apple/swift-syntax#1022
These libraries formed a strongly connected component in the CMake build graph. The weakest link I could find was from IDE to FrontendTool and Frontend, which was necessitated by the `CompileInstance` class (https://github.com/apple/swift/pull/40645). I moved a few files out of IDE into a new IDETools library to break the cycle.
The relationship between the code in these two libraries was fundamentally circular, indicating that they should not have been split. With other changes that I'm making to remove circular dependencies from the CMake build graph I eventually uncovered that these two libraries were required to link each other circularly, but that had been hidden by other cycles in the build graph previously.
Replace Attr.def with a gyb file that reads from swift-syntax
to automatically generate the attribute nodes.
For now, only the Swift attributes have been migrated. SIL attributes
can be defined manually in Attr.def.gyb. To add new Swift attributes,
a paired pull request to swift-syntax is now necessary.
