Add ability to automatically chaining the bridging headers discovered from all
dependencies module when doing swift caching build. This will eliminate all
implicit bridging header imports from the build and make the bridging header
importing behavior much more reliable, while keep the compatibility at maximum.
For example, if the current module A depends on module B and C, and both B and
C are binary modules that uses bridging header, when building module A,
dependency scanner will construct a new header that chains three bridging
headers together with the option to build a PCH from it. This will make all
importing errors more obvious while improving the performance.
Batch dependency scanning was added as a mechanism to support multiple compilation contexts within a single module dependency graph.
The Swift compiler and the Explicitly-built modules model has long since abandoned this approach and this code has long been stale. It is time to remove it and its associated C API.
Instead, each scan's 'ModuleDependenciesCache' will hold all of the data corresponding to discovered module dependencies.
The initial design presumed the possibility of sharing a global scanning cache amongs different scanner invocations, possibly even different concurrent scanner invocations.
This change also deprecates two libSwiftScan entry-points: 'swiftscan_scanner_cache_load' and 'swiftscan_scanner_cache_serialize'. They never ended up getting used, and since this code has been largely stale, we are confident they have not otherwise had users, and they do not fit with this design.
A follow-up change will re-introduce moduele dependency cache serialization on a per-query basis and bring the binary format up-to-date.
Fix few issues from previous implementation from explicit module build
with macros and accurate macro dependency scanning in
https://github.com/swiftlang/swift/pull/73421.
First, there is a crash when propagating the macro dependencies. It
turns out that the current macro plugin implementation doesn't need the
downstream users to know about the plugin search path from the upstream
dependencies.
Secondly, fix a bug that the swiftinterface that has macro usage won't
build because the build command doesn't inherit the plugin search path
option.
Finally, add JSON output for macro dependencies so it is easier to
debug the macro dependencies.
rdar://131214106
This change modifies the dependency scanner to keep track of source locations of each encountered 'import' statement, in order to be able to emit diagnostics with source locations if an import failed to resolve.
- Keep track of each 'import' statement's source buffer, line number, and column number when adding it. The dependency scanner utilizes separate compilation instances, and therefore separate Source Managers for scanning `import` statements of user sources and textual interfaces of Swift dependencies. Since import resolution may happen in the main scanner compilation instance while the `import` itself was found by an interface-scanning sub-instance, we cannot simply hold on to the import's `SourceLoc`.
- Add libSwiftScan API for diagnostics to carry above source locations to clients.
Otherwise they may have module dependencies of their own which will not be detected by the scanner and included in the list of explicit inputs for compilation.
Avoid path encoding difference (for example, real_path vs. path from
symlink) by eliminating the path from cache key. Cache key is now
encoded with the index of the input file from all the input files from
the command-line, reguardless if those inputs will produce output or
not. This is to ensure stable ordering even the batching is different.
Add a new cache computation API that is preferred for using input index
directly. Old API for cache key is deprecated but still updated to
fallback to real_path comparsion if needed.
As a result of swift scan API change, rename the feature in JSON file to
avoid version confusion between swift-driver and libSwiftScan.
rdar://119387650
Add a new async download API from CASID. This helps build system better
schedule the job/remove duplicated downloads, than using an opaque
cached_output handle.
Add new APIs libSwiftScan that can be used for cache query and cache
replay. This enables swift-driver or build system to query the cache and
replay the compilation results without invocation swift-frontend for
better scheduling.
The code, previously, only properly handled such dependencies being a distinct category for Swift source and Swift textual dependency infos. Swift binary module dependencies must handle this similarly and this change adds the missing support for them. Recent refactor of the scanner also means that now Swift binary dependencies with Swift overlay dependencies may crash the scanner, and this change resolves this as well.
Resolves rdar://117088840
Update swift cache key computation mechanism from one cache key per
output, to one cache key per primary input file (for all outputs that
associated with that input).
The new schema allows fewer cache lookups while still preserving most of
the flexibility for batch mode and incremental mode.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
Instead of being a part of 'directDependencies' on a module dependency info, make them a separate array of dependency IDs for Swift Source and Textual modules.
This will allow clients to still distinguish direct module dependencies imported from a given module, versus dependencies added because direct/transitive Clang module dependencies have Swift overlays.
This change does *not* remove overlay dependencies from 'directDependencies' yet, just adds them as a separate field on the module details info. A followup change will remove overlay and bridging header dependencies from 'directDependencies' once the clients have had a chance to adopt to this change.
This new version takes the path to the compiler executable as a parameter, in order for libSwiftScan to compute compiler-relative portions of runtimeLibraryPaths, runtimeResourcePath. V1, without knowing the path to the compiler executable, produced incomplete sets of these paths.
Previously no API returned a raw 'swiftscan_string_ref_t' directly so this was not needed.
However, 'swiftscan_compiler_target_info_query' returns a string directly, which the clients need to clean up.
This is the start of the removal of the C++ implementation of libSyntax
in favor of the new Swift Parser and Swift Syntax libraries. Now that
the Swift Parser has switched the SwiftSyntaxParser library over to
being a thin wrapper around the Swift Parser, there is no longer any
reason we need to retain any libSyntax infrastructure in the swift
compiler.
As a first step, delete the infrastructure that builds
lib_InternalSwiftSyntaxParser and convert any scripts that mention
it to instead mention the static mirror libraries. The --swiftsyntax
build-script flag has been retained and will now just execute the
SwiftSyntax and Swift Parser builds with the just-built tools.
This adds C functions that can be called from SwiftSyntax to enable or disable bare slash regex parsing.
It also adds a function to set the Swift language version - while this is not of top priority at the moment it will become important once we introduce a new language mode again.
rdar://93750821
This separates it from `libSwiftScan` and allows us to build this library without building much of the rest of the compiler.
Also refactor `utils/build-parser-lib` into `utils/build-tooling-libs` which builds both SwiftSyntaxParser and SwiftStaticMirror.
This commit adds new entry-points to `libSwiftScan` that operate on the new BinaryScanningTool, which reads out Swift type information from object files, starting with a query of all protocol conformances.
To split the SwiftSyntax library into two modules – `SwiftSyntax` with no dependency on `_InternalSwiftSyntaxParser.dylib` and `SwiftSyntaxParser` which generates a `SwiftSyntax` tree using `_InternalSwiftSyntaxParser.dylib` – we need to copy the definition of the C types into the SwiftSyntax repository. To make sure the two match, move the duplicated types into their own file. That way, we can compare them in SwiftSyntax’s build script and thus make sure their definitions are the same.
Doing so will allow clients to know which Swift-specific PCM arguments are already captured from the scan that first discovered this module.
SwiftDriver, in particular, will be able to use this information to avoid re-scanning a given Clang module if the initial scan was sufficient for all possible sets of PCM arguments on Swift modules that depend on said Clang module.
These kinds of modules differ from `SwiftTextual` modules in that they do not have an interface and have source-files.
It is cleaner to enforce this distinction with types, instead of checking for interface optionality everywhere.
Using the serialization format added in https://github.com/apple/swift/pull/37585.
- Add load/save code for the `-scan-dependencies` code-path.
- Add `libSwiftDriver` entry-points to load/store the cache of a given scanner instance.
We don't actually need the range for layout nodes, so just store it
for token nodes. This will also make deferred node handling easier
later on, because we don't need to keep track of layout node ranges.
Previously, the passed in source file was implicitly terminated by the
first null character. The source file might, however, contain a null
character in the middle and we shouldn't stop parsing at it.
It is not a distinct result returned by `swiftscan_compiler_supported_arguments_query` and `swiftscan_compiler_supported_features_query`, not owned by any other type, so it must be disposed of by the clients.
