CAS needs to determine if an output is needed very early (when computing supplementary outputs) in the pipeline so we can do caching and replays.
Resolves: rdar://141850408
This patch adds support for serialization and deserialization of
debug scopes.
Debug scopes are serialized in post order and enablement is
controlled through the experimental-serialize-debug-info flag which
is turned off by default. Functions only referred to by these debug
scopes are deserialized as zombie functions directly.
This is something that I have wanted to add for a while and have never had the
need to. I need it now to fix a bug in the bots where I am forced to use IRGen
output to test ThunkLowering which causes platform level differences to show up
in the FileCheck output. With this, I can just emit the actual lowered SIL
output and just test it at that level. There are other cases like this where we
are unable to test lowered SIL so we use IRGen creating this brittleness.
Hopefully this stops this problem from showing up in the future.
rdar://138845396
This change refactors the top-level dependency scanning flow to follow the following procedure:
Scan():
1. From the source target under scan, query all imported module identifiers for a *Swift* module. Leave unresolved identifiers unresolved. Proceed transitively to build a *Swift* module dependency graph.
2. Take every unresolved import identifier in the graph from (1) and, assuming that it must be a Clang module, dispatch all of them to be queried in-parallel by the scanner's worker pool.
3. Resolve bridging header Clang module dpendencies
4. Resolve all Swift overlay dependencies, relying on all Clang modules collected in (2) and (3)
5. For the source target under scan, use all of the above discovered module dependencies to resolve all cross-import overlay dependencies
It might be unexpected to future users that `-swift-compiler-version`
would produce a version aligned to .swiftinterface instead of one used
to build the .swiftmodule file. To avoid this possible confusion, let's
scope down the version to `-interface-compiler-version` flag and
`SWIFT_INTERFACE_COMPILER_VERSION` option in the module.
If swiftinterface doesn't have `-swift-compiler-version` flag
it means that it was generated with an older version of the
Swift compiler. In such cases it would be incorrect to use
"current" compiler version because the field is intended to
indicate the compiler the swiftinterface was built with.
to verify ExportedSourceFileRequest == 0.
In release mode only non-zero stats are printed by default now.
Fix diagnostic when compiler is built without statistics support.
Introduce the concept of public facing module name designed to hide support module from external clients. This name should be set on a support module that is an implementation detail of a public module, the public module name of the support module should be the name of the public module. For example, BigKitCore’s public module name can be set to BigKit for diagnostics to refer to both modules as BigKit.
This name is used to refer to the support module in diagnostics for external clients. We determine if a client is external if they also import the module identified by the public module name. If a client doesn’t import the public module, diagnostics will always use the real name of the module.
The public module name is set with the flag -public-module-name and it's preserved in textual swiftinterfaces and binary swiftmodules.
rdar://134527933
Rather than trying to re-compute the cache path manually for the
default, use the clang provided interface to home the module cache. This
ensures that we do not write the files into a top-level directory on
Windows.
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
When building a module from its interface, do not diagnose whether or not a
feature is available in production compilers. This is important since older
compilers may be expected to build .swiftinterfaces that were produced by newer
compilers where the feature has been enabled by default.
Resolves rdar://125500318
allow a more standard way to pass experimental features
from build systems. Also moved other flags relevant to
diagnostics from Frontend options to Lang options.
Ref: rdar://124648653
we only check if the loaded module is built from a package interface. This is
not enough as a binary module could just contain exportable decls if built with
experimental-skip-non-exportable-decls, essentially resulting in content equivalent
to interface content. This might be made a default behavior so this PR requires
a module to opt in to allow non-resilient access by a participating client in the
same package.
Since it affects module format, SWIFTMODULE_VERSION_MINOR is updated.
rdar://123651270
ClangImporter’s SwiftLookupTables map Swift names to their corresponding Clang declarations. These tables are built into a module’s clang .pcm file and missing or inaccurate entries can cause name lookup to fail to find an imported declaration.
Swift has always included a helper function that would dump these tables, and swift-ide-test has a command-line switch that would invoke it, but these tools are clumsy to use in many debugging scenarios. Add a frontend flag that dumps the tables at the end of the frontend job, making it a lot easier to get at this information in the context of a specific compilation.
The `-experimental-lazy-typecheck` and `-experimental-skip-non-exportable-decls`
flags are not safe to use when emitting a non-resilient module because the
clients of non-resilient modules expect to have access to all the members of a
type in order to e.g. compute the size the type. The
`-experimental-skip-non-exportable-decls` flag skips serialization of
non-public members and would therefore cause mis-compilation. The
`-experimental-lazy-typecheck` is theoretically safe for non-resilient modules
but more requestification work is needed before it can be used successfully.
Resolves rdar://122272758
Remove `-allow-unstable-cache-key-for-testing` frontend flag. It is a
test only flag when the infrastructure is not ready to write tests for
fully cachable tasks. It is no longer needed after all the related tests
are rewritten to use dependency scanner.
This option causes the -experimental-lazy-typecheck and
-experimental-skip-non-exportable-decls options to be inferred from the
presense of -experimental-skip-non-inlinable-function-bodies. This new option
is meant to be a temporary testing aid that allows lazy typechecking to be
tested on projects without full build system support for passing the other
flags to the right jobs.
Resolves rdar://118938251
`-disable-sandbox` to disable sandboxing when invoking subprocess from
from the frontend. Since `sandbox(7)` in macOS doesn't support nested
sandbox, complation used to fail when the parent build process is sandboxed.
It has an extension .package.swiftinterface and contains package decls
as well as SPIs and public/inlinable decls. When a module is loaded
from interface, it now looks up the package-name in the interface
and checks if the importer is in the same package. If so, it uses
that package interface found to load the module. If not, uses the existing
logic to load modules.
Resolves rdar://104617854
Until a swiftc with https://github.com/apple/swift-driver/pull/1465 is
available widely, enable -experimental-skip-non-exportable-decls whenever
-experimental-lazy-typecheck is specified.
Resolves rdar://117020908
Rename -experimental-serialize-external-decls only to
-experimental-skip-non-exportable-decls in preparation for the flag being used
to influence more than just serialization.
Resolves rdar://116771543
An "API descriptor" file is JSON describing the externally accessible symbols
of a module and metadata associated with those symbols like availability and
SPI status. This output was previously only generated by the
`swift-api-extract` alias of `swift-frontend`, which is desgined to take an
already built module as input. Post-processing a built module to extract this
information is inefficient because the module and the module's dependencies
need to be deserialized in order to visit the entire AST. We can generate this
output more efficiently as a supplementary output of the -emit-module job that
originally produced the module (since the AST is already available in-memory).
The -emit-api-descriptor flag can be used to request this output.
This change lays the groundwork by introducing frontend flags. Follow up
changes are needed to make API descriptor emission during -emit-module
functional.
Part of rdar://110916764.
Conflict in CAS options when
`std::vector<std::string> CacheReplayPrefixMap;` was added.
Conflicts:
include/swift/Frontend/FrontendOptions.h
Resolution: Take both
Teach CachedDiagnosticsProcessor to replay diagnostics for all consumers
with a different path using a path prefix map. This allows diagnostics
to be replayed for the actual path on disk after the scanner
canonicalized the path for compilation.
From being a scattered collection of 'static' methods in ScanDependencies.cpp
and member methods of ASTContext. This makes 'ScanDependencies.cpp' much easier
to read, and abstracts the actual scanning logic away to a place with common
state which will make it easier to reason about in the future.
