Commit Graph

70 Commits

Author SHA1 Message Date
Alexis Laferrière
f7f69c6ae1 [Serialization] Load non-public transitive dependencies on @testable imports
A @testable import allows a client to call internal decls which may
refer to non-public dependencies. To support such a use case, load
non-public transitive dependencies of a module when it's imported
@testable from the main module.

This replaces the previous behavior where we loaded those dependencies
for any modules built for testing. This was risky as we would load more
module for any debug build, opening the door to a different behavior
between debug and release builds. In contrast, applying this logic to
@testable clients will only change the behavior of test targets.

rdar://107329303
2023-03-29 13:59:28 -07:00
Alexis Laferrière
599346885e [Serialization] Differentiate module loading behavior for non-public imports
Differentiate `internal` and `fileprivate` imports from
implementation-only imports at the module-wide level to offer a
different module loading strategy. The main difference is for non-public
imports from a module with testing enabled to be loaded by transitive
clients.

Ideally, we would only load transitive non-public dependencies on
testable imports of the middle module. The current module loading logic
doesn't allow for this behavior easily as a module may be first loaded
for a normal import and extra dependencies would have to be loaded on
later imports. We may want to refactor the module loading logic to allow
this if needed.

rdar://106514965
2023-03-21 16:46:53 -07:00
Alexis Laferrière
d7cd65cdd6 [Serialization] Move some package-only import logic from writer to reader
Let's centralize the logic deciding if we load a transitive dependency
on the client side and have the producer write the truth in the
swiftmodule.
2023-03-21 16:45:49 -07:00
Alexis Laferrière
8aadcf464f [Serialization] Do not accept packages without a name
Realign the module loading behavior with the one of the package
access-level. If the package name is an empty string, don't accept other
modules with an empty package name as being part of the same module and
don't load package dependencies in such a case.
2023-03-20 14:03:45 -07:00
Alexis Laferrière
56e013edb9 [Serialization] Try to load transitive package dependencies in the debugger 2023-03-20 14:03:45 -07:00
Alexis Laferrière
61c0827427 [Serialization] Refactor logic deciding transitive module loading logic
Refactor and centralize the logic about how implementation-only and
package-only dependencies should be loaded.
2023-03-20 13:53:10 -07:00
Alexis Laferrière
1b86261a74 [Serialization] Exit right way on any invalid swiftmodule file
The previous behavior kept going even after we reported an invalid
swiftmodule. As such it ended up losing the precise invalid reason and
returned Malformed later on.
2023-03-07 09:55:50 -08:00
Alexis Laferrière
6980cf211b [Serialization] Intro package-only module wide information and serialize it 2023-03-03 11:43:20 -08:00
swift-ci
b08054dcc3 Merge remote-tracking branch 'origin/main' into rebranch 2023-02-14 09:38:50 -08:00
Alexis Laferrière
a5ccbf3264 [Serialization] Only remark if the last digit mismatches in precise tag check
Weaken the precise tag check at loading swiftmodule to accept binary
modules build by a compiler with a tag where only the last digit is
different. We assume that the other digit in the version should ensure
compiler and stdlib compatibility. If the last digit doesn't match,
still raise a remark.

rdar://105158258
2023-02-13 14:28:10 -08:00
Alexis Laferrière
e079db233f [Serialization] Update env var to disable precise tag check
Rename the env var to SWIFT_IGNORE_SWIFTMODULE_REVISION as it's
recommended for use outside of testing.
2023-02-13 13:51:30 -08:00
Alexis Laferrière
4579e37e61 [Serialization] Intro distinct compiler revision string for serialization 2023-02-13 13:51:30 -08:00
swift-ci
0db0a3e7a5 Merge remote-tracking branch 'origin/main' into rebranch 2023-01-27 21:35:06 -08:00
Alexis Laferrière
39fb1c5f55 [ModuleInterface] Intro export-as for Swift modules
Introduce a new flag `-export-as` to specify a name used to identify the
target module in swiftinterfaces. This provides an analoguous feature
for Swift module as Clang's `export_as` feature.

In practice it should be used when a lower level module `MyKitCore` is
desired to be shown publicly as a downstream module `MyKit`. This should
be used in conjunction with `@_exported import MyKitCore` from `MyKit`
that allows clients to refer to all services as being part of `MyKit`,
while the new `-export-as MyKit` from `MyKitCore` will ensure that the
clients swiftinterfaces also use the `MyKit` name for all services.

In the current implementation, the export-as name is used in the
module's clients and not in the declarer's swiftinterface (e.g.
`MyKitCore`'s swiftinterface still uses the `MyKitCore` module name).
This way the module swiftinterface can be verified. In the future, we
may want a similar behavior for other modules in between `MyKitCore` and
`MyKit` as verifying a swiftinterface referencing `MyKit` without it
being imported would fail.

rdar://103888618
2023-01-26 14:27:31 -08:00
swift-ci
50fff10c87 Merge remote-tracking branch 'origin/main' into rebranch 2022-12-19 23:54:19 -08:00
Ellie Shin
72ee150982 Add -package-name flag
De/serialize package name in module binary
Resoles rdar://103531218, rdar://103531208
2022-12-19 14:33:44 -08:00
swift-ci
b6ccc365d8 Merge remote-tracking branch 'origin/main' into rebranch 2022-12-09 17:33:36 -08:00
Adrian Prantl
006ac8fadc Don't require a strict revision match in LLDB for tagged compilers.
This is a follow-up to 3cc2831608 because the
relaxed checking introduced was being bypassed by the code block that checks for
a tagged compiler.
2022-12-09 12:05:25 -08:00
swift-ci
9638cd3de8 Merge remote-tracking branch 'origin/main' into rebranch 2022-12-02 12:37:21 -08:00
Adrian Prantl
a5e1786a99 Expose Swift search paths in validateSerializedAST
This is for the benefit of LLDB, which currently does an expensive import of all
modules to get to the same information.

rdar://40097459
2022-12-01 13:14:08 -08:00
swift-ci
713223eb4c Merge remote-tracking branch 'origin/main' into rebranch 2022-11-28 18:14:43 -08:00
Xi Ge
5987654b3a Merge branch 'main' into allowable-serialization 2022-11-28 09:36:04 -08:00
swift-ci
efef52a86e Merge remote-tracking branch 'origin/main' into rebranch 2022-11-28 09:33:40 -08:00
Xi Ge
67bbab7e02 serialization: encode allowable client names in binary module format 2022-11-25 18:43:40 -08:00
Erik Eckstein
ab1b343dad use new llvm::Optional API
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`

The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.

rdar://102362022
2022-11-21 19:44:24 +01:00
Erik Eckstein
e292d17672 llvm::array_lengthof -> std::size 2022-11-04 20:44:19 +01:00
Alexis Laferrière
41ec454129 [Serialization] Update error message on deserialization failure
This adds information about whether the module was built from source or
from a swiftinterface, whether it's resilient or not, and tweaks the
format.
2022-10-27 18:51:28 -07:00
Alexis Laferrière
2854c1b3cb [Serialization] Write in the swiftmodule if it's built from a swiftinterface
This information will allow us to distinguish swiftmodule built from
source vs swiftinterface.
2022-10-27 18:51:28 -07:00
Adrian Prantl
3cc2831608 Don't require a strict revision match in LLDB.
For release-management purposes during development, LLDB's embedded Swift
compiler's version number can sometimes be off-by-one in the last digit
compared to the Swift compiler.

This patch restores the old behavior from before 17183629e4.

rdar://101299168
2022-10-19 09:03:13 -07:00
Becca Royal-Gordon
d20e00e531 [NFC] Improve deserialization errors for wrong record kind
There are a bunch of places where we expect to find a record with a particular record ID/kind in the decls_block and have to bail out if we see something we don’t expect. Add an `InvalidRecordKindError` we can use in this situation to produce a useful error message and adopt it in various places.

This change also makes deserialization errors print the path to the invalid file.
2022-09-12 13:35:59 -07:00
Robert Widmann
18b79ffcfd Resolve a Layering Violation in libBasic
Basic should not be allowed to link Parse, yet it was doing so
to allow Version to provide a constructor that would conveniently
parse a StringRef. This entrypoint also emitted diagnostics, so it
pulled in libAST.

Sink the version parser entrypoint down into Parse where it belongs
and point all the clients to the right place.
2022-09-09 00:21:30 -07:00
Alexis Laferrière
e17590e0c0 Merge pull request #60610 from xymus/serial-root-message
[Serialization] Suggest a solution path for modified swiftmodules causing deserialization failures
2022-08-18 14:49:26 -07:00
Alexis Laferrière
9c840a7156 [Serialization] Suggest solution for SDK mods causing deserialization failures
Many deserialization failures at this points are caused by archives
installed over the SDK. Let's extend the current error message with a
solution for such a case.

rdar://86280699
2022-08-18 12:30:49 -07:00
Slava Pestov
b36f37cc72 Serialization: Preserve identity of opened generic environments
We used to create a new environment for each opened archetype, which is
incorrect when deserializing a nested type of another opened archetype.
2022-08-15 13:59:12 -04:00
Alexis Laferrière
36b87d3c5c [Serialization] Use a more reliable macro to get the compiler tag
Use only the SWIFT_COMPILER_VERSION macro to check for swiftmodules
being written by the same compiler that reads it. In practice, it's the
macro used for release builds of the compiler.

rdar://96868333
2022-07-18 13:20:20 -07:00
Alexis Laferrière
c341010fbd [Basic][Serialization] Intro and use isCurrentCompilerTagged 2022-05-17 16:45:34 -07:00
Alexis Laferrière
3523adbb53 [Serialization] Limit per-SDK swiftmodule restriction to tagged compilers
Only production compilers should apply the per-SDK restriction on
loading swiftmodules. Use the "is the compiler tagged" information over
a release build to align with the other main swiftmodule loading
restriction accepting only swiftmodules built by the same tag.

Also use an env var SWIFT_DEBUG_FORCE_SWIFTMODULE_PER_SDK to enable
testing this feature in any compilers.
2022-05-16 13:22:35 -07:00
Alexis Laferrière
c8059a09e9 [Serialization] Soft-reject swiftmodules built against a different SDK
Change the way swiftmodules built against a different SDK than their
clients are rejected. This makes them silently ignored when the module
can be rebuilt from their swiftinterface, instead of reporting a hard
error.

rdar://93257769
2022-05-16 13:22:35 -07:00
Alexis Laferrière
17183629e4 [Serialization] Harden the compiler tag check on swiftmodules
When two different serialization formats share a version number but are
different enough, it can defeat the check to restrict loading
swiftmodules built by the same compiler. Add a backup check in case the
REVISION block is unseen, for swiftmodules only not swiftdoc or
sourceinfo.

rdar://93188070
2022-05-12 13:12:48 -07:00
John McCall
713bb43a94 Rework the serialization schema for conformances and requirements.
We now schedule conformance emissions in basically the same way
we do for types and declarations, which means that we'll emit them
uniquely in the module file instead of redundantly at every use.
This should produce substantially smaller module files overall,
especially for modules that heavily use generics.  It also means
that we can remove all the unfortunate code to support using
different abbrev codes for them in different bitcode blocks.

Requirement lists are now emitted inline in the records that need
them instead of as trailing records.  I think this will improve
space usage, but mostly it assists in eliminating the problem
where abbrev codes are shared between blocks.
2022-02-28 17:32:00 -05:00
Saleem Abdulrasool
317c5167d3 Serialization: remove force linked symbols for static libs
This adjusts the IRGen to avoid the force load symbol for static
linking.  When static linking, we can elide the force load symbol as it
exists to ensure that the shared library is loaded at runtime
unconditionally.  However, the symbol will not preserve the library and
it will be DCE'd appropriately.  This resolves the unresolved force load
symbol when statically linking on Windows.
2022-02-15 16:44:28 -08:00
Alexis Laferrière
39c3fc01a8 [Serialization] Restrict loading all swiftmodules by compiler tag
Swiftmodule loading was previously restricted by compiler tag only for
resilient modules. This left room for resilient modules with a corrupted
control block to pass as non-resilient modules.

Apply the same check for non-resilient modules (so all modules) when
read from a tagged compiler.

rdar://88081456
2022-01-27 11:13:57 -08:00
Xi Ge
6377c3a742 Revert "Revert "serialization: obfuscate the serialized search paths"" 2021-12-02 13:21:04 -08:00
Saleem Abdulrasool
11d5d6d4ca Revert "serialization: obfuscate the serialized search paths" 2021-12-02 08:18:23 -08:00
Xi Ge
0047d81f9a serialization: obfuscate the serialized search paths
We noticed some Swift clients rely on the serialized search paths in the module to
find dependencies and droping these paths altogether can lead to build failures like
rdar://85840921.

This change teaches the serialization to obfuscate the search paths and the deserialization
to recover them. This allows clients to keep accessing these paths without exposing
them when shipping the module to other users.
2021-12-01 11:47:41 -08:00
Kuba (Brecka) Mracek
c89eca6c34 Enforce consistent usage of -experimental-hermetic-seat-at-link flag (#39986)
We've recently added the -experimental-hermetic-seal-at-link compiler flag,
which turns on aggressive dead-stripping optimizations and assumes that library
code can be optimized against client code because all users of the library
code/types are present at link/LTO time. This means that any module that's
built with -experimental-hermetic-seal-at-link requires all clients of this
module to also use -experimental-hermetic-seal-at-link. This PR enforces that
by storing a bit in the serialized module, and checking the bit when importing
modules.
2021-11-30 10:44:58 -08:00
Alexis Laferrière
8e23a3815f Merge pull request #39579 from xymus/fix-minus
[Serialization] Fix merge character artifact
2021-10-05 09:33:12 -07:00
Meghana Gupta
f458d9b490 Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag (#39516)
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag

This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.

* Updated tests
2021-10-04 18:46:40 -07:00
Alexis Laferrière
8873988299 [Serialization] Remove - character artifact 2021-10-04 17:07:55 -07:00
Alexis Laferrière
362cacf5a1 [Basic] Update getSwiftRevision to return a StringRef 2021-09-21 10:02:40 -07:00