We noticed some Swift clients rely on the serialized search paths in the module to
find dependencies and droping these paths altogether can lead to build failures like
rdar://85840921.
This change teaches the serialization to obfuscate the search paths and the deserialization
to recover them. This allows clients to keep accessing these paths without exposing
them when shipping the module to other users.
We've recently added the -experimental-hermetic-seal-at-link compiler flag,
which turns on aggressive dead-stripping optimizations and assumes that library
code can be optimized against client code because all users of the library
code/types are present at link/LTO time. This means that any module that's
built with -experimental-hermetic-seal-at-link requires all clients of this
module to also use -experimental-hermetic-seal-at-link. This PR enforces that
by storing a bit in the serialized module, and checking the bit when importing
modules.
This commit adds a new frontend flag that applies debug path prefixing to the
paths serialized in swiftmodule files. This makes it possible to use swiftmodule
files that have been built on different machines by applying the inverse map
when debugging, in a similar fashion to source path prefixing.
The inverse mapping in LLDB will be handled in a follow up PR.
Second pass at #39138
Tests updated to handle windows path separators.
This reverts commit f5aa95b381.
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag
This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.
* Updated tests
This commit adds the `-prefix-serialized-debugging-options` flag,
which is used to apply the debug prefix map to serialized debugging
options embedded in the swiftmodule files.
Serialize the canonical name of the SDK used when building a swiftmodule
file and use it to ensure that the swiftmodule file is loaded only with
the same SDK. The SDK name must be passed down from the frontend.
This will report unsupported configurations like:
- Installing roots between incompatible SDKs without deleting the
swiftmodule files.
- Having multiple targets in the same project using different SDKs.
- Loading a swiftmodule created with a newer SDK (and stdlib) with an
older SDK.
All of these lead to hard to investigate deserialization failures and
this change should detect them early, before reaching a deserialization
failure.
rdar://78048939
Foundation imports CoreFoundation with `@_implementationOnly`,
so CoreFoundation's modulemap won't be read, and the dependent libraries
of CoreFoundation will not be automatically linked when using static
linking.
For example, CoreFoundation depends on libicui18n and it's modulemap has
`link "icui18n"` statement. If Foundation imports CoreFoundation with
`@_implementationOnly` as a private dependency, the toolchain doesn't have
CoreFoundation's modulemap and Foundation's swiftmodule doesn't import
CoreFoundation. So the swiftc can't know that libicui18n is required.
This new option will add LINK_LIBRARY entry in swiftmodule to
specify dependent libraries (in the example case, Foundation's
swiftmodule should have LINK_LIBRARY entry of libicui18n)
See also: [Autolinking behavior of @_implementationOnly with static linking](https://forums.swift.org/t/autolinking-behavior-of-implementationonly-with-static-linking/44393)
If the `-static` option is specified, store that in the generated
swiftmodule file. When de-serializing, recover this information in the
representative SILModule.
This will be used for code generation on Windows. It is the missing
piece to allow static linking to function properly. It additionally
opens the path to additional optimization on ELF-ish targets - GOT, PLT
references can be avoided when the linked module is known to be static.
Co-authored by: Saleem Abdulrasool <compnerd@compnerd.org>
This allows library authors to pass down a project version number so that library users can conditionally
import that library based on the available version in the search paths.
Needed for rdar://73992299
In the legacy driver, these flags will merely be propagated to the
frontends to indicate that they should disable serialization of
incremental information in swift module files.
In the new driver, these flags control whether the Swift driver performs
an incremental build that is aware of metadata embedded in the module.
Kudos to David for coming up with our new marketing name: Incremental
Imports.
rdar://74363450
Remove the option to switch off nested types tables. In a world where
re-entrant direct lookup will cause deserialization to fail (or worse),
disabling these tables will only lead to further instability in the
compiler.
When a Swift module built with debug info imports a library without
debug info from a textual interface, the textual interface is
necessary to reconstruct types defined in the library's interface. By
recording the Swift interface files in DWARF dsymutil can collect them
and LLDB can find them.
rdar://problem/49751363
Add a bit to the module to determine whether the dependency’s stored bit pattern is a hash or an mtime.
Prebuilt modules store a hash of their dependencies because we can’t be sure their dependencies will have the same modtime as when they were built.
Previously, we included the PCH hash components in the cache key. While they didn’t do any harm, they didn’t contribute any unique information about the module in question.
Additionally, passing the effective language version in means that each dependency that uses a different -swift-version would re-compile all of its dependencies. This is unfortunate, as that means the standard library is recompiled potentially several times.
Hashing the contents of the interface files is overkill. In practice, size and last modification time are enough to determine if a file has changed on disk, and therefore should be rebuilt.
There's a class of errors in Serialization called "circularity
issues", where declaration A in file A.swift depends on declaration B
in file B.swift, and B also depends on A. In some cases we can manage
to type-check each of these files individually due to the laziness of
'validateDecl', but then fail to merge the "partial modules" generated
from A.swift and B.swift to form a single swiftmodule for the library
(because deserialization is a little less lazy for some things). A
common case of this is when at least one of the declarations is
nested, in which case a lookup to find that declaration needs to load
all the members of the parent type. This gets even worse when the
nested type is defined in an extension.
This commit sidesteps that issue specifically for nested types by
creating a top-level, per-file table of nested types in the "partial
modules". When a type is in the same module, we can then look it up
/without/ importing all other members of the parent type.
The long-term solution is to allow accessing any members of a type
without having to load them all, something we should support not just
for module-merging while building a single target but when reading
from imported modules as well. This should improve both compile time
and memory usage, though I'm not sure to what extent. (Unfortunately,
too many things still depend on the whole members list being loaded.)
Because this is a new code path, I put in a switch to turn it off:
frontend flag -disable-serialization-nested-type-lookup-table
https://bugs.swift.org/browse/SR-3707 (and possibly others)
That's how everything behaved anyway. Might as well make it explicit and
stop special-casing it.
I've left in compatibility for modules built with older compilers so that
people using the OS toolchains aren't immediately unable to debug their apps.
As soon as we change the module format in a more significant way, I can take
this out.
Groundwork for rdar://problem/21254367; see next commit.
Swift SVN r29437
...so that the debugger has the best possible chance of being able to load
the app properly.
We don't do this for frameworks today because we don't want to leak this
information into the public module; once we have a distinction between
"the module that ships with the framework" and "the module that goes into
the debug info" we can do this for frameworks as well.
Part of rdar://problem/17670778
Swift SVN r25204
There's also a testing option, -serialize-debugging-options, to force this
extra info to be serialized even for library targets. In the long run we'll
probably write out this information for all targets, but strip it out of
the "public module" when a framework is built. (That way it ends up in the
debug info's copy of the module.)
Incidentally, this commit includes the ability to add search paths to the
Clang importer on the fly, which is most of rdar://problem/16347147.
Unfortunately there's no centralized way to add search paths to both Clang
/and/ Swift at the moment.
Part of rdar://problem/17670778
Swift SVN r24545
Previously, we depended on whether or not a serialized module was located
within a framework bundle to consider whether or not it may have a "Clang
half". However, LLDB loads serialized modules from dSYM bundles. Rather
than try to figure out if such a module is "really" a framework, just track
whether the original module was built with -import-underlying-module. If so,
consider the underlying Clang module to be re-exported.
rdar://problem/18099523
Swift SVN r21544
