When caching build is enabled, teach dependency scanner to report
command-lines with `-direct-clang-cc1-module-build` so the later
compilation can instantiate clang importer with cc1 args directly. This
avoids running clang driver code, which might involve file system
lookups, which are the file deps that are not captured and might result
in different compilation mode.
rdar://119275464
Otherwise they may have module dependencies of their own which will not be detected by the scanner and included in the list of explicit inputs for compilation.
ClangImporter’s SwiftLookupTables map Swift names to their corresponding Clang declarations. These tables are built into a module’s clang .pcm file and missing or inaccurate entries can cause name lookup to fail to find an imported declaration.
Swift has always included a helper function that would dump these tables, and swift-ide-test has a command-line switch that would invoke it, but these tools are clumsy to use in many debugging scenarios. Add a frontend flag that dumps the tables at the end of the frontend job, making it a lot easier to get at this information in the context of a specific compilation.
When prefix mapping paths that are used in clang, ensure we are
consistently using the same prefix mapper from clang. This prevents
mismatches that could cause modules to fail to load.
rdar://123324072
This reduces the memory overhead of objcImpl from one word per Decl to one bit per Decl, at the cost of making cache lookups slightly slower (but it will only be consulted once for non-objcImpl decls, which is by far the most common case).
This allows calling a C++ function with default arguments from Swift without having to explicitly specify the values of all arguments.
rdar://103975014
Re-write and clean up how clang-importer is created from clang
arguments. Previously, it is unclear if `getClangArguments` will return
CC1 args or driver args and the logic is unnecessarily compilicated when
creating clang invocation. Now clang invocation is always created from
cc1 arguments, which can be directly provided via direct-cc1-mode or
converted from driver args.
There is no functional changes in this patch, other than
`-dump-clang-diagnostics` now will always print cc1 args, and also
driver args if that is applicable.
libc++ recently split the `std` module into many top-level modules: 571178a21a
Previously if a C++ module had `#include <iosfwd>`, importing that module in Swift would make the entire C++ stdlib visible from Swift, since it was a single top-level Clang module. After libc++ got split it doesn't automatically do so, but we need to preserve the current behavior for Swift users.
rdar://119270491
Swift has some module maps it overlays on Linux and Windows that groups all of the C standard library headers into a single module. This doesn’t allow clang and C++ headers to layer properly with the OS/SDK modules. clang will set -fbuiltin-headers-in-system-modules as necessary for Apple SDKs, but Swift will need to pass that flag itself when required by its module maps.
This is a simple work around to avoid importing virtual functions when symbolic
imports are turned on. Test cases that were failing before this WA are in
test/Interop/Cxx/symbolic-imports.
Thanks to Alex Lorenz for providing this WA to me (@hyp).
Allow DependencyScanner to canonicalize path using a prefix map. When
option `-scanner-prefix-map` option is used, dependency scanner will
remap all the input paths in following:
* all the paths in the CAS file system or clang include tree
* all the paths related to input on the command-line returned by scanner
This allows all the input paths to be canonicalized so cache key can be
computed reguardless of the exact on disk path.
The sourceFile field is not remapped so build system can track the exact
file as on the local file system.
From being a scattered collection of 'static' methods in ScanDependencies.cpp
and member methods of ASTContext. This makes 'ScanDependencies.cpp' much easier
to read, and abstracts the actual scanning logic away to a place with common
state which will make it easier to reason about in the future.
This fixes the automatic `std::unordered_map` conformance to CxxDictionary on Linux. Previously `std::unordered_map::const_iterator` was not auto-conformed to UnsafeCxxInputIterator because its `operator==` is defined on a templated base class of `const_iterator`.
rdar://105220600
This prevents users from calling functions with unsupported or unavailable return types. This ensures that users don't for example call a function that returns a non-copyable and non-movable type
Fixes https://github.com/apple/swift/issues/64401
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
Teach swift dependency scanner to use CAS to capture the full dependencies for a build and construct build commands with immutable inputs from CAS.
This allows swift compilation caching using CAS.
When swift-frontend is explicitly passed the pch file as bridging header
on command-line through `-import-objc-header`, it needs to print the
original source file name if needed to the generated objc header.
rdar://109411245
The Clang importer's Clang instance may be configured with a different (higher)
OS version than the compilation target itself in order to be able to load
pre-compiled Clang modules that are aligned with the broader SDK, and match the
SDK deployment target against which Swift modules are also built. In this case,
we must use the Swift compiler's OS version triple in order to generate the
binary as-requested.
This change makes 'ClangImporter' 'Implementation' keep track of a distinct
'TargetInfo' and 'CodeGenOpts' containers that are meant to be used by clients
in IRGen. When '-clang-target' is not set, they are defined to be copies of the
'ClangImporter's built-in module-loading Clang instance. When '-clang-target' is
set, they are configured with the Swift compilation's target triple and OS
version (but otherwise identical) instead. To distinguish IRGen clients from
module loading clients, 'getModuleAvailabilityTarget' is added for module
loading clients of 'ClangImporter'.
The notion of using a different triple for loading Clang modules arises for the
following reason:
- Swift is able to load Swift modules built against a different target triple
than the source module that is being compiled. Swift relies on availability
annotations on the API within the loaded modules to ensure that compilation
for the current target only uses appropriately-available API from its
dependencies.
- Clang, in contrast, requires that compilation only ever load modules (.pcm)
that are precisely aligned to the current source compilation. Because the
target triple (OS version in particular) between Swift source compilation and
Swift dependency module compilation may differ, this would otherwise result in
builtin multiple copies of the same Clang module, against different OS
versions, once for each different triple in the build graph.
Instead, with Explicitly-Built Modules, Swift sets a '-clang-target' argument
that ensures that all Clang modules participating in the build are built against
the SDK deployment target, matching the Swift modules in the SDK, which allows
them to expose a maximally-available API surface as required by
potentially-depending Swift modules' target OS version.
--------------------------------------------
For example:
Suppose we are building a source module 'Foo', targeting 'macosx10.0', using an
SDK with a deployment target of 'macosx12.0'. Swift modules in said SDK will be
built for 'macosx12.0' (as hard-coded in their textual interfaces), meaning they
may reference symbols expected to be present in dependency Clang modules at that
target OS version.
Suppose the source module 'Foo' depends on Swift module 'Bar', which then
depends on Clang module `Baz`. 'Bar' must be built targeting 'macosx12.0'
(SDK-matching deployment target is hard-coded into its textual interface). Which
means that 'Bar' expects 'Baz' to expose symbols that may only be available when
targeting at least 'macosx12.0'. e.g. 'Baz' may have symbols guarded with
'__MAC_OS_X_VERSION_MIN_REQUIRED >= __MAC_12_0'. For this reason, we use
'-clang-target' to ensure 'Baz' is built targeting 'macosx12.0', and can be
loaded by both 'Foo' and 'Bar'.
As a result, we cannot direclty use the Clang instance's target triple here and
must check if we need to instead use the triple of the Swift compiler instance.
Resolves rdar://109228963
For a `@Testable` import in program source, if a Swift interface dependency is discovered, and has an adjacent binary `.swiftmodule`, open up the module, and pull in its optional dependencies. If an optional dependency cannot be resolved on the filesystem, fail silently without raising a diagnostic.
Using a virutal output backend to capture all the outputs from
swift-frontend invocation. This allows redirecting and/or mirroring
compiler outputs to multiple location using different OutputBackend.
As an example usage for the virtual outputs, teach swift compiler to
check its output determinism by running the compiler invocation
twice and compare the hash of all its outputs.
Virtual output will be used to enable caching in the future.
CF_OPTIONS is defined differently in the SDK based on
a __cplusplus preprocessor branch. As a result, declarations
referencing CF_OPTIONS are mangled differently depending
on if C++ interop is enabled.
This meant a module compiled with cxx interop on could
not be linked with a module compiled without and vice versa.
This patch modifies the mangler such that the mangled names
are consistent. This is achieved by feeding the mangler a modified
AST node that looks like the Objective-C definition of CF_OPTIONS,
even when we have cxx interop enabled.
This changes the scanner's behavior to "resolve" a discovered module's dependencies to a set of Module IDs: module name + module kind (swift textual, swift binary, clang, etc.).
The 'ModuleDependencyInfo' objects that are stored in the dependency scanner's cache now carry a set of kind-qualified ModuleIDs for their dependencies, in addition to unqualified imported module names of their dependencies.
Previously, the scanner's internal state would cache a module dependnecy as having its own set of dependencies which were stored as names of imported modules. This led to a design where any time we needed to process the dependency downstream from its discovery (e.g. cycle detection, graph construction), we had to query the ASTContext to resolve this dependency's imports, which shouldn't be necessary. Now, upon discovery, we "resolve" a discovered dependency by executing a lookup for each of its imported module names (this operation happens regardless of this patch) and store a fully-resolved set of dependencies in the dependency module info.
Moreover, looking up a given module dependency by name (via `ASTContext`'s `getModuleDependencies`) would result in iterating over the scanner's module "loaders" and querying each for the module name. The corresponding modules would then check the scanner's cache for a respective discovered module, and if no such module is found the "loader" would search the filesystem.
This meant that in practice, we searched the filesystem on many occasions where we actually had cached the required dependency, as follows:
Suppose we had previously discovered a Clang module "foo" and cached its dependency info.
-> ASTContext.getModuleDependencies("foo")
--> (1) Swift Module "Loader" checks caches for a Swift module "foo" and doesn't find one, so it searches the filesystem for "foo" and fails to find one.
--> (2) Clang Module "Loader" checks caches for a Clang module "foo", finds one and returns it to the client.
This means that we were always searching the filesystem in (1) even if we knew that to be futile.
With this change, queries to `ASTContext`'s `getModuleDependencies` will always check all the caches first, and only delegate to the scanner "loaders" if no cached dependency is found. The loaders are then no longer in the business of checking the cached contents.
To handle cases in the scanner where we must only lookup either a Swift-only module or a Clang-only module, this patch splits 'getModuleDependencies' into an alrady-existing 'getSwiftModuleDependencies' and a newly-added 'getClangModuleDependencies'.
