Commit Graph

64 Commits

Author SHA1 Message Date
Alexis Laferrière
1b86261a74 [Serialization] Exit right way on any invalid swiftmodule file
The previous behavior kept going even after we reported an invalid
swiftmodule. As such it ended up losing the precise invalid reason and
returned Malformed later on.
2023-03-07 09:55:50 -08:00
Alexis Laferrière
6980cf211b [Serialization] Intro package-only module wide information and serialize it 2023-03-03 11:43:20 -08:00
swift-ci
b08054dcc3 Merge remote-tracking branch 'origin/main' into rebranch 2023-02-14 09:38:50 -08:00
Alexis Laferrière
a5ccbf3264 [Serialization] Only remark if the last digit mismatches in precise tag check
Weaken the precise tag check at loading swiftmodule to accept binary
modules build by a compiler with a tag where only the last digit is
different. We assume that the other digit in the version should ensure
compiler and stdlib compatibility. If the last digit doesn't match,
still raise a remark.

rdar://105158258
2023-02-13 14:28:10 -08:00
Alexis Laferrière
e079db233f [Serialization] Update env var to disable precise tag check
Rename the env var to SWIFT_IGNORE_SWIFTMODULE_REVISION as it's
recommended for use outside of testing.
2023-02-13 13:51:30 -08:00
Alexis Laferrière
4579e37e61 [Serialization] Intro distinct compiler revision string for serialization 2023-02-13 13:51:30 -08:00
swift-ci
0db0a3e7a5 Merge remote-tracking branch 'origin/main' into rebranch 2023-01-27 21:35:06 -08:00
Alexis Laferrière
39fb1c5f55 [ModuleInterface] Intro export-as for Swift modules
Introduce a new flag `-export-as` to specify a name used to identify the
target module in swiftinterfaces. This provides an analoguous feature
for Swift module as Clang's `export_as` feature.

In practice it should be used when a lower level module `MyKitCore` is
desired to be shown publicly as a downstream module `MyKit`. This should
be used in conjunction with `@_exported import MyKitCore` from `MyKit`
that allows clients to refer to all services as being part of `MyKit`,
while the new `-export-as MyKit` from `MyKitCore` will ensure that the
clients swiftinterfaces also use the `MyKit` name for all services.

In the current implementation, the export-as name is used in the
module's clients and not in the declarer's swiftinterface (e.g.
`MyKitCore`'s swiftinterface still uses the `MyKitCore` module name).
This way the module swiftinterface can be verified. In the future, we
may want a similar behavior for other modules in between `MyKitCore` and
`MyKit` as verifying a swiftinterface referencing `MyKit` without it
being imported would fail.

rdar://103888618
2023-01-26 14:27:31 -08:00
swift-ci
50fff10c87 Merge remote-tracking branch 'origin/main' into rebranch 2022-12-19 23:54:19 -08:00
Ellie Shin
72ee150982 Add -package-name flag
De/serialize package name in module binary
Resoles rdar://103531218, rdar://103531208
2022-12-19 14:33:44 -08:00
swift-ci
b6ccc365d8 Merge remote-tracking branch 'origin/main' into rebranch 2022-12-09 17:33:36 -08:00
Adrian Prantl
006ac8fadc Don't require a strict revision match in LLDB for tagged compilers.
This is a follow-up to 3cc2831608 because the
relaxed checking introduced was being bypassed by the code block that checks for
a tagged compiler.
2022-12-09 12:05:25 -08:00
swift-ci
9638cd3de8 Merge remote-tracking branch 'origin/main' into rebranch 2022-12-02 12:37:21 -08:00
Adrian Prantl
a5e1786a99 Expose Swift search paths in validateSerializedAST
This is for the benefit of LLDB, which currently does an expensive import of all
modules to get to the same information.

rdar://40097459
2022-12-01 13:14:08 -08:00
swift-ci
713223eb4c Merge remote-tracking branch 'origin/main' into rebranch 2022-11-28 18:14:43 -08:00
Xi Ge
5987654b3a Merge branch 'main' into allowable-serialization 2022-11-28 09:36:04 -08:00
swift-ci
efef52a86e Merge remote-tracking branch 'origin/main' into rebranch 2022-11-28 09:33:40 -08:00
Xi Ge
67bbab7e02 serialization: encode allowable client names in binary module format 2022-11-25 18:43:40 -08:00
Erik Eckstein
ab1b343dad use new llvm::Optional API
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`

The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.

rdar://102362022
2022-11-21 19:44:24 +01:00
Erik Eckstein
e292d17672 llvm::array_lengthof -> std::size 2022-11-04 20:44:19 +01:00
Alexis Laferrière
41ec454129 [Serialization] Update error message on deserialization failure
This adds information about whether the module was built from source or
from a swiftinterface, whether it's resilient or not, and tweaks the
format.
2022-10-27 18:51:28 -07:00
Alexis Laferrière
2854c1b3cb [Serialization] Write in the swiftmodule if it's built from a swiftinterface
This information will allow us to distinguish swiftmodule built from
source vs swiftinterface.
2022-10-27 18:51:28 -07:00
Adrian Prantl
3cc2831608 Don't require a strict revision match in LLDB.
For release-management purposes during development, LLDB's embedded Swift
compiler's version number can sometimes be off-by-one in the last digit
compared to the Swift compiler.

This patch restores the old behavior from before 17183629e4.

rdar://101299168
2022-10-19 09:03:13 -07:00
Becca Royal-Gordon
d20e00e531 [NFC] Improve deserialization errors for wrong record kind
There are a bunch of places where we expect to find a record with a particular record ID/kind in the decls_block and have to bail out if we see something we don’t expect. Add an `InvalidRecordKindError` we can use in this situation to produce a useful error message and adopt it in various places.

This change also makes deserialization errors print the path to the invalid file.
2022-09-12 13:35:59 -07:00
Robert Widmann
18b79ffcfd Resolve a Layering Violation in libBasic
Basic should not be allowed to link Parse, yet it was doing so
to allow Version to provide a constructor that would conveniently
parse a StringRef. This entrypoint also emitted diagnostics, so it
pulled in libAST.

Sink the version parser entrypoint down into Parse where it belongs
and point all the clients to the right place.
2022-09-09 00:21:30 -07:00
Alexis Laferrière
e17590e0c0 Merge pull request #60610 from xymus/serial-root-message
[Serialization] Suggest a solution path for modified swiftmodules causing deserialization failures
2022-08-18 14:49:26 -07:00
Alexis Laferrière
9c840a7156 [Serialization] Suggest solution for SDK mods causing deserialization failures
Many deserialization failures at this points are caused by archives
installed over the SDK. Let's extend the current error message with a
solution for such a case.

rdar://86280699
2022-08-18 12:30:49 -07:00
Slava Pestov
b36f37cc72 Serialization: Preserve identity of opened generic environments
We used to create a new environment for each opened archetype, which is
incorrect when deserializing a nested type of another opened archetype.
2022-08-15 13:59:12 -04:00
Alexis Laferrière
36b87d3c5c [Serialization] Use a more reliable macro to get the compiler tag
Use only the SWIFT_COMPILER_VERSION macro to check for swiftmodules
being written by the same compiler that reads it. In practice, it's the
macro used for release builds of the compiler.

rdar://96868333
2022-07-18 13:20:20 -07:00
Alexis Laferrière
c341010fbd [Basic][Serialization] Intro and use isCurrentCompilerTagged 2022-05-17 16:45:34 -07:00
Alexis Laferrière
3523adbb53 [Serialization] Limit per-SDK swiftmodule restriction to tagged compilers
Only production compilers should apply the per-SDK restriction on
loading swiftmodules. Use the "is the compiler tagged" information over
a release build to align with the other main swiftmodule loading
restriction accepting only swiftmodules built by the same tag.

Also use an env var SWIFT_DEBUG_FORCE_SWIFTMODULE_PER_SDK to enable
testing this feature in any compilers.
2022-05-16 13:22:35 -07:00
Alexis Laferrière
c8059a09e9 [Serialization] Soft-reject swiftmodules built against a different SDK
Change the way swiftmodules built against a different SDK than their
clients are rejected. This makes them silently ignored when the module
can be rebuilt from their swiftinterface, instead of reporting a hard
error.

rdar://93257769
2022-05-16 13:22:35 -07:00
Alexis Laferrière
17183629e4 [Serialization] Harden the compiler tag check on swiftmodules
When two different serialization formats share a version number but are
different enough, it can defeat the check to restrict loading
swiftmodules built by the same compiler. Add a backup check in case the
REVISION block is unseen, for swiftmodules only not swiftdoc or
sourceinfo.

rdar://93188070
2022-05-12 13:12:48 -07:00
John McCall
713bb43a94 Rework the serialization schema for conformances and requirements.
We now schedule conformance emissions in basically the same way
we do for types and declarations, which means that we'll emit them
uniquely in the module file instead of redundantly at every use.
This should produce substantially smaller module files overall,
especially for modules that heavily use generics.  It also means
that we can remove all the unfortunate code to support using
different abbrev codes for them in different bitcode blocks.

Requirement lists are now emitted inline in the records that need
them instead of as trailing records.  I think this will improve
space usage, but mostly it assists in eliminating the problem
where abbrev codes are shared between blocks.
2022-02-28 17:32:00 -05:00
Saleem Abdulrasool
317c5167d3 Serialization: remove force linked symbols for static libs
This adjusts the IRGen to avoid the force load symbol for static
linking.  When static linking, we can elide the force load symbol as it
exists to ensure that the shared library is loaded at runtime
unconditionally.  However, the symbol will not preserve the library and
it will be DCE'd appropriately.  This resolves the unresolved force load
symbol when statically linking on Windows.
2022-02-15 16:44:28 -08:00
Alexis Laferrière
39c3fc01a8 [Serialization] Restrict loading all swiftmodules by compiler tag
Swiftmodule loading was previously restricted by compiler tag only for
resilient modules. This left room for resilient modules with a corrupted
control block to pass as non-resilient modules.

Apply the same check for non-resilient modules (so all modules) when
read from a tagged compiler.

rdar://88081456
2022-01-27 11:13:57 -08:00
Xi Ge
6377c3a742 Revert "Revert "serialization: obfuscate the serialized search paths"" 2021-12-02 13:21:04 -08:00
Saleem Abdulrasool
11d5d6d4ca Revert "serialization: obfuscate the serialized search paths" 2021-12-02 08:18:23 -08:00
Xi Ge
0047d81f9a serialization: obfuscate the serialized search paths
We noticed some Swift clients rely on the serialized search paths in the module to
find dependencies and droping these paths altogether can lead to build failures like
rdar://85840921.

This change teaches the serialization to obfuscate the search paths and the deserialization
to recover them. This allows clients to keep accessing these paths without exposing
them when shipping the module to other users.
2021-12-01 11:47:41 -08:00
Kuba (Brecka) Mracek
c89eca6c34 Enforce consistent usage of -experimental-hermetic-seat-at-link flag (#39986)
We've recently added the -experimental-hermetic-seal-at-link compiler flag,
which turns on aggressive dead-stripping optimizations and assumes that library
code can be optimized against client code because all users of the library
code/types are present at link/LTO time. This means that any module that's
built with -experimental-hermetic-seal-at-link requires all clients of this
module to also use -experimental-hermetic-seal-at-link. This PR enforces that
by storing a bit in the serialized module, and checking the bit when importing
modules.
2021-11-30 10:44:58 -08:00
Alexis Laferrière
8e23a3815f Merge pull request #39579 from xymus/fix-minus
[Serialization] Fix merge character artifact
2021-10-05 09:33:12 -07:00
Meghana Gupta
f458d9b490 Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag (#39516)
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag

This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.

* Updated tests
2021-10-04 18:46:40 -07:00
Alexis Laferrière
8873988299 [Serialization] Remove - character artifact 2021-10-04 17:07:55 -07:00
Alexis Laferrière
362cacf5a1 [Basic] Update getSwiftRevision to return a StringRef 2021-09-21 10:02:40 -07:00
Alexis Laferrière
5c2122a08c [Serialization] Restrict resilient swiftmodules to the compiler that built them
Introduce a new loading restriction that is more strict than the serialization
version check on swiftmodules. Tagged compilers will only load
library-evolution enabled swiftmodules that are produced by a compiler with the
exact same revision id. This will be more reliable in production
environments than using the serialization version which we forgot to
update from time to time. This shouldn't affect development compilers that
will still load any module with a compatible serialization version.

rdar://83105234
2021-09-20 18:06:35 -07:00
Alexis Laferrière
c38d1773d2 [Serialization] Restrict loading swiftmodule files to the builder's SDK
Serialize the canonical name of the SDK used when building a swiftmodule
file and use it to ensure that the swiftmodule file is loaded only with
the same SDK. The SDK name must be passed down from the frontend.

This will report unsupported configurations like:

- Installing roots between incompatible SDKs without deleting the
swiftmodule files.
- Having multiple targets in the same project using different SDKs.
- Loading a swiftmodule created with a newer SDK (and stdlib) with an
older SDK.

All of these lead to hard to investigate deserialization failures and
this change should detect them early, before reaching a deserialization
failure.

rdar://78048939
2021-09-13 16:44:08 -07:00
Doug Gregor
d54abea922 Implement customizable Sendable conformance diagnostics.
Rework Sendable checking to be completely based on "missing"
conformances, so that we can individually diagnose missing Sendable
conformances based on both the module in which the conformance check
happened as well as where the type was declared. The basic rules here
are to only diagnose if either the module where the non-Sendable type
was declared or the module where it was checked was compiled with a
mode that consistently diagnoses `Sendable`, either by virtue of
being Swift 6 or because `-warn-concurrency` was provided on the
command line. And have that diagnostic be an error in Swift 6 or
warning in Swift 5.x.

There is much tuning to be done here.
2021-08-14 08:13:10 -07:00
Ben Barham
599ba8bdbd Add all deserialization fatal output to the pretty stack trace
Rather than outputting diagnostics and to stderr, output all the extra
information added when deserialization fatally fails to the pretty stack
trace instead. Since the pretty stack trace is added to crash logs, this
should avoid the dance of requesting the compiler output

  - Moves the previous "**** DESERIALIZATION FAILURE ..." output to the
    last pretty stack trace line

  - Removes the module and compiler version notes added to the fatal
    diagnostic

  - Adds a new effective compiler version line for all frontend failure.
    Somewhat duplicates the line from the driver, but adds in the
    effective version

  - Adds a new line for the full misc version of the module that failed.
    May double up with previous "While reading from ..." lines that are
    added in various deserialization methods, but better to have it
    twice than not at all
2021-06-04 08:32:07 +10:00
Xi Ge
5c5715f626 Serialization: embed subminor and build version in the user module version entry 2021-05-17 10:15:47 -07:00
Saleem Abdulrasool
0771f1ec86 Merge pull request #37211 from compnerd/windows-static-linking
IRGen: support static linking on Windows
2021-05-05 07:26:36 -07:00