While this made sense in the distant past where the scanning service provided backing storage for the dependency cache, it no longer does so and now makes for awkard layering where clients get at the service via the cache. Now the cache is a simple data structure while all the clients that need access to the scanning service will get it explicitly.
- 'SwiftModuleScanner' will now be owned directly by the 'ModuleDependencyScanningWorker' and will contain all the necessary custom logic, instead of being instantiated by the module interface loader for each query
- Moves ownership over module output path and sdk module output path directly into the scanning worker, instead of the cache
On creation, 'ClangImporter' adds overlay modulemap files for non-modular platform libraries (e.g. glibc, libstdc++), which allows Swift code to import and use those libraries.
This change adds the same filesystem overlay to dependency scanning queries by applying them to the filesystem instantiated for each depndency scanning worker. Without these overlays EBM builds cannot discover and use non-modular system libraries on non-Darwin platforms.
Resolves rdar://151780437
After removing the CASFS implementation for clang modules, there is no
need to capture clang extra file that sets up the VFS for the clang
modules since all content imported by ClangImporter is dependency
scanned and available via include-tree. This saves more ClangImporter
instance when caching is enabled.
Update the test to check that clang content found via `-Xcc` VFS options
can currently work without capture the headermaps and vfs overlays.
In Swift 6.1, we introduced `SWIFT_RETURNS_RETAINED` and
`SWIFT_RETURNS_UNRETAINED` annotations for C++ APIs to explicitly
specify the ownership convention of `SWIFT_SHARED_REFERENCE` type return
values.
Currently the Swift compiler emits warnings for unannotated C++ APIs
returning `SWIFT_SHARED_REFERENCE` types. We've received some feedback
that people are finding these warnings useful to get a reminder to
annotate their APIs. While this improves correctness , it also imposes a
high annotation burden on adopters — especially in large C++ codebases.
This patch addresses that burden by introducing two new type-level
annotations:
- `SWIFT_RETURNED_AS_RETAINED_BY_DEFAULT`
- `SWIFT_RETURNED_AS_UNRETAINED_BY_DEFAULT`
These annotations allow developers to specify a default ownership
convention for all C++ APIs returning a given
`SWIFT_SHARED_REFERENCE`-annotated type, unless explicitly overridden at
the API by using `SWIFT_RETURNS_RETAINED` or `SWIFT_RETURNS_UNRETAINED`.
If a C++ class inherits from a base class annotated with
`SWIFT_RETURNED_AS_RETAINED_BY_DEFAULT` or
`SWIFT_RETURNED_AS_UNRETAINED_BY_DEFAULT`, the derived class
automatically inherits the default ownership convention unless it is
explicitly overridden. This strikes a balance between safety/correctness
and usability:
- It avoids the need to annotate every API individually.
- It retains the ability to opt out of the default at the API level when
needed.
- To verify correctness, the user can just remove the
`SWIFT_RETURNED_AS_(UN)RETAINED_BY_DEFAULT` annotation from that type
and they will start seeing the warnings on all the unannotated C++ APIs
returning that `SWIFT_SHARED_REFERENCE` type. They can add
`SWIFT_RETURNS_(UN)RETAINED` annotation at each API in which they want a
different behaviour than the default. Then they can reintroduce the
`SWIFT_RETURNED_AS_(UN)RETAINED_BY_DEFAULT` at the type level to
suppress the warnings on remaining unannotated APIs.
A global default ownership convention (like always return
`unretained`/`unowned`) was considered but it would weaken the
diagnostic signal and remove valuable guardrails that help detect
use-after-free bugs and memory leaks in absence of
`SWIFT_RETURNS_(UN)RETAINED` annotations. In the absence of these
annotations when Swift emits the unannotated API warning, the current
fallback behavior (e.g. relying on heuristics based on API name such as
`"create"`, `"copy"`, `"get"`) is derived from Objective-C interop but
is ill-suited for C++, which has no consistent naming patterns for
ownership semantics.
Several codebases are expected to have project-specific conventions,
such as defaulting to unretained except for factory methods and
constructors. A type-level default seems like the most precise and
scalable mechanism to support such patterns. It integrates cleanly with
existing `SWIFT_SHARED_REFERENCE` usage and provides a per-type opt-in
mechanism without global silencing of ownership diagnostics.
This addition improves ergonomics while preserving the safety benefits
of explicit annotations and diagnostics.
rdar://145453509
Importing C++ class templates in symbolic mode has proven to be problematic in interaction with other compiler features, and it isn't used widely. This change removes the feature.
rdar://150528798
It is possible for a C++ class template to inherit from a specialization
of itself. Normally, these are imported to Swift as separate (unrelated)
types, but when symbolic import is enabled, unspecialized templates are
imported in place of their specializations, leading to circularly
inheriting classes to seemingly inherit from themselves.
This patch adds a check to guard against the most common case of
circular inheritance, when a class template directly inherits from
itself. This pattern appears in a recent version of libc++,
necessitating this patch. However, the solution here is imperfect as it
does not handle more complex/contrived circular inheritance patterns.
This patch also adds a test case exercising this pattern. The
-index-store-path flag causes swift-frontend to index the C++ module
with symbolic import enabled, without the fix in this patch, that test
triggers an assertion failure due to the circular reference (and can
infinitely recurse in the StorageVisitor when assertions are disabled).
rdar://148026461
To trigger this error one needs to import a nested type from C++, use it
in a generic context in Swift, and export it back to C++. We were
inconsisent in what namespace did we declare the functions to get the
type metadata for types. It was in the swift namespace for foreign types
and in the module namespace for Swift types. This PR standardizes on how
the metadata function is declared and called to fix the issue.
Fixes#80538.
rdar://148597079
When loading a module with embedded bridging header, bind the bridging
header module in the context when bridging header auto chaining is used.
This is because all the bridging header contents are chained into a PCH
file so binary module with bridging header should reference the PCH file
for all declarations.
rdar://148538787
This patch fixes the access check for nested private C++ enums to look for the SWIFT_PRIVATE_FILEID of the enclosing C++ class, if any. Previously, the check was looking at for SWIFT_PRIVATE_FILEID on the enum decl itself (which is meaningless); that prevented nested private enum members from being accessible in Swift.
This patch also specializes the type signature of getPrivateFileIDAttrs to clarify the fact that SWIFT_PRIVATE_FILEID is not a meaningful annotation on anything other than CXXRecordDecl, because that is the only kind of decl that can assign access specifiers to its members.
rdar://148081340
The Error enum synthesized declarations, e.g. the struct and its static accessors, should generally appear to be identical to the underlying Clang definitions. There are some specific use cases where the synthesized declarations are necessary though.
I've added an option for USR generation to override the Clang node and emit the USR of the synthesized Swift declaration. This is used by SwiftDocSupport so that the USRs of the synthesized declarations are emitted.
Fixes 79912
With '-sdk-module-cache-path', Swift textual interfaces found in the SDK will be built into a separate SDK-specific module cache.
Clang modules are not yet affected by this change, pending addition of the required API.
It is possible for a module interface (e.g., ModuleA) to be generated
with C++ interop disabled, and then rebuilt with C++ interop enabled
(e.g., because ModuleB, which imports ModuleA, has C++ interop enabled).
This circumstance can lead to various issues when name lookup behaves
differently depending on whether C++ interop is enabled, e.g., when
a module name is shadowed by a namespace of the same name---this only
happens in C++ because namespaces do not exist in C. Unfortunately,
naming namespaces the same as a module is a common C++ convention,
leading to many textual interfaces whose fully-qualified identifiers
(e.g., c_module.c_member) cannot be correctly resolved when C++ interop
is enabled (because c_module is shadowed by a namespace of the same
name).
This patch does two things. First, it introduces a new frontend flag,
-formal-cxx-interoperability-mode, which records the C++ interop mode
a module interface was originally compiled with. Doing so allows
subsequent consumers of that interface to interpret it according to the
formal C++ interop mode. Note that the actual "versioning" used by this
flag is very crude: "off" means disabled, and "swift-6" means enabled.
This is done to be compatible with C++ interop compat versioning scheme,
which seems to produce some invalid (but unused) version numbers. The
versioning scheme for both the formal and actual C++ interop modes
should be clarified and fixed in a subsequent patch.
The second thing this patch does is fix the module/namespace collision
issue in module interface files. It uses the formal C++ interop mode to
determine whether it should resolve C++-only decls during name lookup.
For now, the fix is very minimal and conservative: it only filters out
C++ namespaces during unqualified name lookup in an interface that was
originally generated without C++ interop. Doing so should fix the issue
while minimizing the chance for collateral breakge. More cases other
than C++ namespaces should be added in subsequent patches, with
sufficient testing and careful consideration.
rdar://144566922
This patch is follow-up work from #78942 and imports non-public members,
which were previously not being imported. Those members can be accessed
in a Swift file blessed by the SWIFT_PRIVATE_FILEID annotation.
As a consequence of this patch, we are also now importing inherited members
that are inaccessible from the derived classes, because they were declared
private, or because they were inherited via nested private inheritance. We
import them anyway but mark them unavailable, for better diagnostics and to
(somewhat) simplify the import logic for inheritance.
Because non-public base class members are now imported too, this patch
inflames an existing issue where a 'using' declaration on an inherited member
with a synthesized name (e.g., operators) produces duplicate members, leading
to miscompilation (resulting in a runtime crash). This was not previously noticed
because a 'using' declaration on a public inherited member is not usually
necessary, but is a common way to expose otherwise non-public members.
This patch puts in a workaround to prevent this from affecting the behavior
of MSVC's std::optional implementation, which uses this pattern of 'using'
a private inherited member. That will be fixed in a follow-up patch.
Follow-up work is also needed to correctly diagnose ambiguous overloads
in cases of multiple inheritance, and to account for virtual inheritance.
rdar://137764620
This patch introduces an a C++ class annotation, SWIFT_PRIVATE_FILEID,
which will specify where Swift extensions of that class will be allowed
to access its non-public members, e.g.:
class SWIFT_PRIVATE_FILEID("MyModule/MyFile.swift") Foo { ... };
The goal of this feature is to help C++ developers incrementally migrate
the implementation of their C++ classes to Swift, without breaking
encapsulation and indiscriminately exposing those classes' private and
protected fields.
As an implementation detail of this feature, this patch introduces an
abstraction for file ID strings, FileIDStr, which represent a parsed pair
of module name/file name.
rdar://137764620
Add ability to automatically chaining the bridging headers discovered from all
dependencies module when doing swift caching build. This will eliminate all
implicit bridging header imports from the build and make the bridging header
importing behavior much more reliable, while keep the compatibility at maximum.
For example, if the current module A depends on module B and C, and both B and
C are binary modules that uses bridging header, when building module A,
dependency scanner will construct a new header that chains three bridging
headers together with the option to build a PCH from it. This will make all
importing errors more obvious while improving the performance.
This commit removes the guardrails in ImportDecl.cpp:SwiftDeclConverter
that prevent it from importing non-public C++ members. It also
accordingly adjusts all code that assumes generated Swift decls should
be public. This commit does not import non-public inherited members;
that needs its own follow-up patch.
Note that Swift enforces stricter invariants about access levels than C++.
For instance, public typealiases cannot be assigned private underlying types,
and public functions cannot take or return private types. Meanwhile,
both of these patterns are supported in C++, where exposing private types
from a class's public interface is considered feature. As far as I am aware,
Swift was already importing such private-containing public decls from C++
already, but I added a test suite, access inversion, that checks and
documents this scenario, to ensure that it doesn't trip any assertions.
This change refactors the top-level dependency scanning flow to follow the following procedure:
Scan():
1. From the source target under scan, query all imported module identifiers for a *Swift* module. Leave unresolved identifiers unresolved. Proceed transitively to build a *Swift* module dependency graph.
2. Take every unresolved import identifier in the graph from (1) and, assuming that it must be a Clang module, dispatch all of them to be queried in-parallel by the scanner's worker pool.
3. Resolve bridging header Clang module dpendencies
4. Resolve all Swift overlay dependencies, relying on all Clang modules collected in (2) and (3)
5. For the source target under scan, use all of the above discovered module dependencies to resolve all cross-import overlay dependencies
Use IncludeTreeFileList instead of full feature CASFS for swift
dependency filesystem. This allows smaller CAS based VFS that is smaller
and faster. This is enabled by the CAS enabled compilation does not
need to iterate file system.
rdar://136787368
We should still try adding the overlays, even if we're asked not to
generate a diagnostic while doing so. That's slightly safer because
it means that we're less likely to find ourselves in a situation
where `swift-modulewrap` wants to use types from the C/C++ library
and can't.
rdar://115918181
`swift-modulewrap` uses the `ClangImporter` to obtain a module loader,
but it doesn't take an SDK argument (nor does anything bother to pass
one), which means that when cross-compiling you get warnings about not
being able to find the C library.
Suppress the warning by telling the `ClangImporter` that we don't care
about the C library here.
rdar://115918181
This makes sure that Swift respects `-Xcc -stdlib=libc++` flags.
Clang already has existing logic to discover the system-wide libc++ installation on Linux. We rely on that logic here.
Importing a Swift module that was built with a different C++ stdlib is not supported and emits an error.
The Cxx module can be imported when compiling with any C++ stdlib. The synthesized conformances, e.g. to CxxRandomAccessCollection also work. However, CxxStdlib currently cannot be imported when compiling with libc++, since on Linux it refers to symbols from libstdc++ which have different mangled names in libc++.
rdar://118357548 / https://github.com/swiftlang/swift/issues/69825
The decision to exclude `-Xcc -D` options from swift module hash
actually doesn't help to solve the problem. It wouldn't reduce the
module variants (or the number of swiftmodule build commands) because
the command-line also encodes all the clang PCM dependencies that do get
affected by `-Xcc` flags.
To avoid the false sharing and the nondeterministic build products,
include most of the `-Xcc` flags, except include search path, into swift
module hash.
rdar://132046247
When the swiftmodule is built with different clang importer arguments,
they can have the same module hash, causing them to be wrongly re-used even
they contains different interfaces. Add ReducedExtraArgs to the module hash to
disambiguate them.
However, some Xcc arguments, most commonly `-D` options do not affect the
swiftmodule being generated. Do not pass `-Xcc -DARGS` to swift
interface compilation to reduce the amount of module variants in the
build.
rdar://131408266
These x-refs might not be resolvable using regular lookup from the 'std' module as they could be instantiated/synthesized
by the clang importer. Augment the lookup logic in that case to try clang importer lookup logic that is used during
the conformance to the C++ iterator protocol.
Fix the problem that when the only module can be found is an
invalid/out-of-date swift binary module, canImport and import statement
can have different view for if the module can be imported or not.
Now canImport will evaluate to false if the only module can be found for
name is an invalid swiftmodule, with a warning with the path to the
module so users will not be surprised by such behavior.
rdar://128876895
The clang nodes associated with Swift's Core Foundation types can already be
represented by a pointer. The interop code does not need to add an extra
layer of indirection in those cases.
rdar://119840281
Make sure the `-Xcc` options to the scanner are correctly considered
when creating ClangImporterCC1 arguments for constructed swift interface
compilation job. Under directcc1 mode, `-Xcc` options should be used to
constructed sub-invocation but should not be added to GenericArgs for
constructing interface compilation jobs.
rdar://128873665
When caching is enabled with include-tree, the bridging header PCH is
created from the include tree directly. Setup the rewriter correctly
when embedding the bridging header into swift binary module.
rdar://125719747
When caching build is enabled, teach dependency scanner to report
command-lines with `-direct-clang-cc1-module-build` so the later
compilation can instantiate clang importer with cc1 args directly. This
avoids running clang driver code, which might involve file system
lookups, which are the file deps that are not captured and might result
in different compilation mode.
rdar://119275464
