Commit Graph

54 Commits

Author SHA1 Message Date
swift-ci
b6ccc365d8 Merge remote-tracking branch 'origin/main' into rebranch 2022-12-09 17:33:36 -08:00
Adrian Prantl
006ac8fadc Don't require a strict revision match in LLDB for tagged compilers.
This is a follow-up to 3cc2831608 because the
relaxed checking introduced was being bypassed by the code block that checks for
a tagged compiler.
2022-12-09 12:05:25 -08:00
swift-ci
9638cd3de8 Merge remote-tracking branch 'origin/main' into rebranch 2022-12-02 12:37:21 -08:00
Adrian Prantl
a5e1786a99 Expose Swift search paths in validateSerializedAST
This is for the benefit of LLDB, which currently does an expensive import of all
modules to get to the same information.

rdar://40097459
2022-12-01 13:14:08 -08:00
swift-ci
713223eb4c Merge remote-tracking branch 'origin/main' into rebranch 2022-11-28 18:14:43 -08:00
Xi Ge
5987654b3a Merge branch 'main' into allowable-serialization 2022-11-28 09:36:04 -08:00
swift-ci
efef52a86e Merge remote-tracking branch 'origin/main' into rebranch 2022-11-28 09:33:40 -08:00
Xi Ge
67bbab7e02 serialization: encode allowable client names in binary module format 2022-11-25 18:43:40 -08:00
Erik Eckstein
ab1b343dad use new llvm::Optional API
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`

The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.

rdar://102362022
2022-11-21 19:44:24 +01:00
Erik Eckstein
e292d17672 llvm::array_lengthof -> std::size 2022-11-04 20:44:19 +01:00
Alexis Laferrière
41ec454129 [Serialization] Update error message on deserialization failure
This adds information about whether the module was built from source or
from a swiftinterface, whether it's resilient or not, and tweaks the
format.
2022-10-27 18:51:28 -07:00
Alexis Laferrière
2854c1b3cb [Serialization] Write in the swiftmodule if it's built from a swiftinterface
This information will allow us to distinguish swiftmodule built from
source vs swiftinterface.
2022-10-27 18:51:28 -07:00
Adrian Prantl
3cc2831608 Don't require a strict revision match in LLDB.
For release-management purposes during development, LLDB's embedded Swift
compiler's version number can sometimes be off-by-one in the last digit
compared to the Swift compiler.

This patch restores the old behavior from before 17183629e4.

rdar://101299168
2022-10-19 09:03:13 -07:00
Becca Royal-Gordon
d20e00e531 [NFC] Improve deserialization errors for wrong record kind
There are a bunch of places where we expect to find a record with a particular record ID/kind in the decls_block and have to bail out if we see something we don’t expect. Add an `InvalidRecordKindError` we can use in this situation to produce a useful error message and adopt it in various places.

This change also makes deserialization errors print the path to the invalid file.
2022-09-12 13:35:59 -07:00
Robert Widmann
18b79ffcfd Resolve a Layering Violation in libBasic
Basic should not be allowed to link Parse, yet it was doing so
to allow Version to provide a constructor that would conveniently
parse a StringRef. This entrypoint also emitted diagnostics, so it
pulled in libAST.

Sink the version parser entrypoint down into Parse where it belongs
and point all the clients to the right place.
2022-09-09 00:21:30 -07:00
Alexis Laferrière
e17590e0c0 Merge pull request #60610 from xymus/serial-root-message
[Serialization] Suggest a solution path for modified swiftmodules causing deserialization failures
2022-08-18 14:49:26 -07:00
Alexis Laferrière
9c840a7156 [Serialization] Suggest solution for SDK mods causing deserialization failures
Many deserialization failures at this points are caused by archives
installed over the SDK. Let's extend the current error message with a
solution for such a case.

rdar://86280699
2022-08-18 12:30:49 -07:00
Slava Pestov
b36f37cc72 Serialization: Preserve identity of opened generic environments
We used to create a new environment for each opened archetype, which is
incorrect when deserializing a nested type of another opened archetype.
2022-08-15 13:59:12 -04:00
Alexis Laferrière
36b87d3c5c [Serialization] Use a more reliable macro to get the compiler tag
Use only the SWIFT_COMPILER_VERSION macro to check for swiftmodules
being written by the same compiler that reads it. In practice, it's the
macro used for release builds of the compiler.

rdar://96868333
2022-07-18 13:20:20 -07:00
Alexis Laferrière
c341010fbd [Basic][Serialization] Intro and use isCurrentCompilerTagged 2022-05-17 16:45:34 -07:00
Alexis Laferrière
3523adbb53 [Serialization] Limit per-SDK swiftmodule restriction to tagged compilers
Only production compilers should apply the per-SDK restriction on
loading swiftmodules. Use the "is the compiler tagged" information over
a release build to align with the other main swiftmodule loading
restriction accepting only swiftmodules built by the same tag.

Also use an env var SWIFT_DEBUG_FORCE_SWIFTMODULE_PER_SDK to enable
testing this feature in any compilers.
2022-05-16 13:22:35 -07:00
Alexis Laferrière
c8059a09e9 [Serialization] Soft-reject swiftmodules built against a different SDK
Change the way swiftmodules built against a different SDK than their
clients are rejected. This makes them silently ignored when the module
can be rebuilt from their swiftinterface, instead of reporting a hard
error.

rdar://93257769
2022-05-16 13:22:35 -07:00
Alexis Laferrière
17183629e4 [Serialization] Harden the compiler tag check on swiftmodules
When two different serialization formats share a version number but are
different enough, it can defeat the check to restrict loading
swiftmodules built by the same compiler. Add a backup check in case the
REVISION block is unseen, for swiftmodules only not swiftdoc or
sourceinfo.

rdar://93188070
2022-05-12 13:12:48 -07:00
John McCall
713bb43a94 Rework the serialization schema for conformances and requirements.
We now schedule conformance emissions in basically the same way
we do for types and declarations, which means that we'll emit them
uniquely in the module file instead of redundantly at every use.
This should produce substantially smaller module files overall,
especially for modules that heavily use generics.  It also means
that we can remove all the unfortunate code to support using
different abbrev codes for them in different bitcode blocks.

Requirement lists are now emitted inline in the records that need
them instead of as trailing records.  I think this will improve
space usage, but mostly it assists in eliminating the problem
where abbrev codes are shared between blocks.
2022-02-28 17:32:00 -05:00
Saleem Abdulrasool
317c5167d3 Serialization: remove force linked symbols for static libs
This adjusts the IRGen to avoid the force load symbol for static
linking.  When static linking, we can elide the force load symbol as it
exists to ensure that the shared library is loaded at runtime
unconditionally.  However, the symbol will not preserve the library and
it will be DCE'd appropriately.  This resolves the unresolved force load
symbol when statically linking on Windows.
2022-02-15 16:44:28 -08:00
Alexis Laferrière
39c3fc01a8 [Serialization] Restrict loading all swiftmodules by compiler tag
Swiftmodule loading was previously restricted by compiler tag only for
resilient modules. This left room for resilient modules with a corrupted
control block to pass as non-resilient modules.

Apply the same check for non-resilient modules (so all modules) when
read from a tagged compiler.

rdar://88081456
2022-01-27 11:13:57 -08:00
Xi Ge
6377c3a742 Revert "Revert "serialization: obfuscate the serialized search paths"" 2021-12-02 13:21:04 -08:00
Saleem Abdulrasool
11d5d6d4ca Revert "serialization: obfuscate the serialized search paths" 2021-12-02 08:18:23 -08:00
Xi Ge
0047d81f9a serialization: obfuscate the serialized search paths
We noticed some Swift clients rely on the serialized search paths in the module to
find dependencies and droping these paths altogether can lead to build failures like
rdar://85840921.

This change teaches the serialization to obfuscate the search paths and the deserialization
to recover them. This allows clients to keep accessing these paths without exposing
them when shipping the module to other users.
2021-12-01 11:47:41 -08:00
Kuba (Brecka) Mracek
c89eca6c34 Enforce consistent usage of -experimental-hermetic-seat-at-link flag (#39986)
We've recently added the -experimental-hermetic-seal-at-link compiler flag,
which turns on aggressive dead-stripping optimizations and assumes that library
code can be optimized against client code because all users of the library
code/types are present at link/LTO time. This means that any module that's
built with -experimental-hermetic-seal-at-link requires all clients of this
module to also use -experimental-hermetic-seal-at-link. This PR enforces that
by storing a bit in the serialized module, and checking the bit when importing
modules.
2021-11-30 10:44:58 -08:00
Alexis Laferrière
8e23a3815f Merge pull request #39579 from xymus/fix-minus
[Serialization] Fix merge character artifact
2021-10-05 09:33:12 -07:00
Meghana Gupta
f458d9b490 Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag (#39516)
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag

This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.

* Updated tests
2021-10-04 18:46:40 -07:00
Alexis Laferrière
8873988299 [Serialization] Remove - character artifact 2021-10-04 17:07:55 -07:00
Alexis Laferrière
362cacf5a1 [Basic] Update getSwiftRevision to return a StringRef 2021-09-21 10:02:40 -07:00
Alexis Laferrière
5c2122a08c [Serialization] Restrict resilient swiftmodules to the compiler that built them
Introduce a new loading restriction that is more strict than the serialization
version check on swiftmodules. Tagged compilers will only load
library-evolution enabled swiftmodules that are produced by a compiler with the
exact same revision id. This will be more reliable in production
environments than using the serialization version which we forgot to
update from time to time. This shouldn't affect development compilers that
will still load any module with a compatible serialization version.

rdar://83105234
2021-09-20 18:06:35 -07:00
Alexis Laferrière
c38d1773d2 [Serialization] Restrict loading swiftmodule files to the builder's SDK
Serialize the canonical name of the SDK used when building a swiftmodule
file and use it to ensure that the swiftmodule file is loaded only with
the same SDK. The SDK name must be passed down from the frontend.

This will report unsupported configurations like:

- Installing roots between incompatible SDKs without deleting the
swiftmodule files.
- Having multiple targets in the same project using different SDKs.
- Loading a swiftmodule created with a newer SDK (and stdlib) with an
older SDK.

All of these lead to hard to investigate deserialization failures and
this change should detect them early, before reaching a deserialization
failure.

rdar://78048939
2021-09-13 16:44:08 -07:00
Doug Gregor
d54abea922 Implement customizable Sendable conformance diagnostics.
Rework Sendable checking to be completely based on "missing"
conformances, so that we can individually diagnose missing Sendable
conformances based on both the module in which the conformance check
happened as well as where the type was declared. The basic rules here
are to only diagnose if either the module where the non-Sendable type
was declared or the module where it was checked was compiled with a
mode that consistently diagnoses `Sendable`, either by virtue of
being Swift 6 or because `-warn-concurrency` was provided on the
command line. And have that diagnostic be an error in Swift 6 or
warning in Swift 5.x.

There is much tuning to be done here.
2021-08-14 08:13:10 -07:00
Ben Barham
599ba8bdbd Add all deserialization fatal output to the pretty stack trace
Rather than outputting diagnostics and to stderr, output all the extra
information added when deserialization fatally fails to the pretty stack
trace instead. Since the pretty stack trace is added to crash logs, this
should avoid the dance of requesting the compiler output

  - Moves the previous "**** DESERIALIZATION FAILURE ..." output to the
    last pretty stack trace line

  - Removes the module and compiler version notes added to the fatal
    diagnostic

  - Adds a new effective compiler version line for all frontend failure.
    Somewhat duplicates the line from the driver, but adds in the
    effective version

  - Adds a new line for the full misc version of the module that failed.
    May double up with previous "While reading from ..." lines that are
    added in various deserialization methods, but better to have it
    twice than not at all
2021-06-04 08:32:07 +10:00
Xi Ge
5c5715f626 Serialization: embed subminor and build version in the user module version entry 2021-05-17 10:15:47 -07:00
Saleem Abdulrasool
0771f1ec86 Merge pull request #37211 from compnerd/windows-static-linking
IRGen: support static linking on Windows
2021-05-05 07:26:36 -07:00
Ben Barham
f9859a545f Merge pull request #37196 from bnbarham/warn-on-invalid-sourceinfo
[Serialization] Add warning when .swiftsourceinfo is malformed
2021-05-04 16:16:49 +10:00
Erik Eckstein
762e9c7882 Serialization: serialize if the module is a static library
If the `-static` option is specified, store that in the generated
swiftmodule file.  When de-serializing, recover this information in the
representative SILModule.

This will be used for code generation on Windows.  It is the missing
piece to allow static linking to function properly.  It additionally
opens the path to additional optimization on ELF-ish targets - GOT, PLT
references can be avoided when the linked module is known to be static.

Co-authored by: Saleem Abdulrasool <compnerd@compnerd.org>
2021-05-02 09:34:52 -07:00
Ben Barham
bfb9205c7c [Serialization] Add warning when .swiftsourceinfo is malformed
Output a warning rather than ignoring a malformed .swiftsourceinfo
completely.

Resolves rdar://77350048
2021-05-01 11:31:24 +10:00
Xi Ge
fe5c7ef995 ModuleInterface/Serialization: allow library authors to define a custom module version number
This allows library authors to pass down a project version number so that library users can conditionally
import that library based on the available version in the search paths.

Needed for rdar://73992299
2021-04-30 10:00:45 -07:00
Doug Gregor
2a28fed34d Allow one to separately specify the ABI name of a module.
Introduce a new compiler flag `-module-abi-name <name>` that uses the
given name as the ABI name for the module (rather than the module's
name in source code). The ABI name impacts name mangling and metadata.
2021-03-12 07:42:07 -08:00
Rintaro Ishizaki
1b6e7857c2 [Serialization] Serialize/deserialize source file list 2021-01-13 11:49:52 -08:00
Arnold Schwaighofer
2b6ba2830c Serialize decls with _specialize(exported:true, target: somefunc(_:), ...) in a list for fast lookup
rdar://71430641
2020-12-11 09:58:32 -08:00
Robert Widmann
9f79fd5df1 Serialize Fingerprints in Swift Modules 2020-11-17 16:23:07 -08:00
Ben Barham
241559dc88 [Serialization] Add an option to output modules regardless of errors
Adds a new frontend option
"-experimental-allow-module-with-compiler-errors". If any compilation
errors occur while generating the .swiftmodule, this mode will skip SIL
entirely and only serialize the (likey invalid) AST.

This existence of this option during generation is serialized into the
resulting .swiftmodule. Errors found in deserialization are only allowed
if it is set.

Primarily intended for IDE requests (eg. indexing and code completion)
to ensure robust cross-module results, despite possible errors.

Resolves rdar://69815975
2020-11-10 14:47:22 +10:00
Robert Widmann
d2e7bdcfab Teach SwiftModules To Embed Incremental Information
Take advantage of the binary swiftdeps serialization utliities built during #32131. Add a new optional information block to swiftdeps files. For now, don't actually serialize swiftdeps information.

Frontends will use this information to determine whether to write incremental dependencies across modules into their swiftdeps files. We will then teach the driver to deserialize the data from this section and integrate it into its incremental decision making.
2020-09-24 20:07:01 -06:00