The response file expansion function doesn't return all the arguments
allocated using StringSaver produced. It only use the StringSaver to
save the newly expanded arguments. Need to make sure all the original
arguments are saved so they can be used later during replay.
rdar://121808135
Build system can pass response file to libSwiftScan as the command-line
arguments. libSwiftScan needs to expand the response file before
performing tasks.
rdar://121291342
`llvm::cl::ParseCommandLineOptions()` is not thread-safe to be called
from libSwiftScan so replaying multiple commands in parallel can crash
the build system. Since the replay instance is not used for compilation
and only need information from command-line arguments to create outputs
and diagnostics, ignore LLVMArgs when replaying.
rdar://120423882
Avoid path encoding difference (for example, real_path vs. path from
symlink) by eliminating the path from cache key. Cache key is now
encoded with the index of the input file from all the input files from
the command-line, reguardless if those inputs will produce output or
not. This is to ensure stable ordering even the batching is different.
Add a new cache computation API that is preferred for using input index
directly. Old API for cache key is deprecated but still updated to
fallback to real_path comparsion if needed.
As a result of swift scan API change, rename the feature in JSON file to
avoid version confusion between swift-driver and libSwiftScan.
rdar://119387650
Add a new async download API from CASID. This helps build system better
schedule the job/remove duplicated downloads, than using an opaque
cached_output handle.
ASTGen always builds with the host Swift compiler, without requiring
bootstrapping, and is enabled in more places. Move the regex literal
parsing logic there so it is enabled in more host environments, and
makes use of CMake's Swift support. Enable all of the regex literal
tests when ASTGen is built, to ensure everything is working.
Remove the "AST" and "Parse" Swift modules from SwiftCompilerSources,
because they are no longer needed.
Add new APIs libSwiftScan that can be used for cache query and cache
replay. This enables swift-driver or build system to query the cache and
replay the compilation results without invocation swift-frontend for
better scheduling.
Conflicts:
- `lib/AST/TypeCheckRequests.cpp` renamed `isMoveOnly` which requires
a static_cast on rebranch because `Optional` is now a `std::optional`.
The code, previously, only properly handled such dependencies being a distinct category for Swift source and Swift textual dependency infos. Swift binary module dependencies must handle this similarly and this change adds the missing support for them. Recent refactor of the scanner also means that now Swift binary dependencies with Swift overlay dependencies may crash the scanner, and this change resolves this as well.
Resolves rdar://117088840
Update swift cache key computation mechanism from one cache key per
output, to one cache key per primary input file (for all outputs that
associated with that input).
The new schema allows fewer cache lookups while still preserving most of
the flexibility for batch mode and incremental mode.
Conflicts:
- `CMakeLists.txt` caused by the extra `-D` added in rebranch to
reduce the number of deprecation warnings.
- `lib/Frontend/PrintingDiagnosticConsumer.cpp` caused by the removal
of one of the `#if SWIFT_SWIFT_PARSER` on rebranch (probably should
have been done on main).
Use FetchContent to include swift-syntax directly in swift. This can be
thought of as an `add_subdirectory` for a directory outside the root.
The default build directory will be `_deps/swiftsyntax-subbuild/`, though
the modules and shared libraries will be built in `lib/swift/host` by
passing down `SWIFT_HOST_LIBRARIES_DEST_DIR` to avoid copying them as we
were doing previously.
libSwiftScan is built in 'lib' but installed in 'lib/swift/host' RUNPATH
should have correct '$ORIGIN/../{platform}' to load 'swiftCore' runtime
library.
For compiling codes required for macro support, we now need swiftc
compiler in the build machine.
Unlike Darwin OSes, where swiftCore runtime is guaranteed to be present
in /usr/lib, Linux doesn't have ABI stability and the stdlib of the
build machine is not at the specific location. So the built compiler
cannot relies on the shared object in the toolchain.
LLVM install_symlink takes a new argument on whether to create a symlink
or copy binaries when run. -- https://reviews.llvm.org/D145443
The variable that controls this in LLVM is `LLVM_USE_SYMLINKS`, which
defaults to `ON` on Unix-y hosts, but otherwise is false so that Windows
works. This is a configurable option, so Windows configs that can
support symlinks can take advantage of symlinks and save some space.
`LLVM_USE_SYMLINKS` is not exported from LLVM though, so we can't see
it to use. Instead, we have `SWIFT_USE_SYMLINKS`.
As of CMake 3.25, there are now global variables `LINUX=1`, `ANDROID=1`,
etc. These conflict with expressions that used these names as unquoted
strings in positions where CMake accepts 'variable|string', for example:
- `if(sdk STREQUAL LINUX)` would fail, because `LINUX` is now defined and
expands to 1, where it would previously coerce to a string.
- `if(${sdk} STREQUAL "LINUX")` would fail if `sdk=LINUX`, because the
left-hand side expands twice.
In this patch, I looked for a number of patterns to fix up, sometimes a
little defensively:
- Quoted right-hand side of `STREQUAL` where I was confident it was
intended to be a string literal.
- Removed manual variable expansion on left-hand side of `STREQUAL`,
`MATCHES` and `IN_LIST` where I was confident it was unintended.
Fixes#65028.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
Instead of being a part of 'directDependencies' on a module dependency info, make them a separate array of dependency IDs for Swift Source and Textual modules.
This will allow clients to still distinguish direct module dependencies imported from a given module, versus dependencies added because direct/transitive Clang module dependencies have Swift overlays.
This change does *not* remove overlay dependencies from 'directDependencies' yet, just adds them as a separate field on the module details info. A followup change will remove overlay and bridging header dependencies from 'directDependencies' once the clients have had a chance to adopt to this change.
This new version takes the path to the compiler executable as a parameter, in order for libSwiftScan to compute compiler-relative portions of runtimeLibraryPaths, runtimeResourcePath. V1, without knowing the path to the compiler executable, produced incomplete sets of these paths.
Previously no API returned a raw 'swiftscan_string_ref_t' directly so this was not needed.
However, 'swiftscan_compiler_target_info_query' returns a string directly, which the clients need to clean up.
This separates it from `libSwiftScan` and allows us to build this library without building much of the rest of the compiler.
Also refactor `utils/build-parser-lib` into `utils/build-tooling-libs` which builds both SwiftSyntaxParser and SwiftStaticMirror.
This commit adds new entry-points to `libSwiftScan` that operate on the new BinaryScanningTool, which reads out Swift type information from object files, starting with a query of all protocol conformances.
This cleans up 90 instances of this warning and reduces the build spew
when building on Linux. This helps identify actual issues when
building which can get lost in the stream of warning messages. It also
helps restore the ability to build the compiler with gcc.
Fixes a linux build error.
It doesn’t make sense to let add HAS_LIBSWIFT to add_swift_host_library(). This was added to work around a linker bug (d22b348adb). Instead do the workaround in libSwiftScan/CMakeLists.txt.
Adding build modes for libswift: off, hosttools, bootstrapping, bootstrapping-with-hostlibs
The two bootstrapping modes are new. For details see libswift/README.md
Doing so will allow clients to know which Swift-specific PCM arguments are already captured from the scan that first discovered this module.
SwiftDriver, in particular, will be able to use this information to avoid re-scanning a given Clang module if the initial scan was sufficient for all possible sets of PCM arguments on Swift modules that depend on said Clang module.
