Commit Graph

63 Commits

Author SHA1 Message Date
Xi Ge
5987654b3a Merge branch 'main' into allowable-serialization 2022-11-28 09:36:04 -08:00
Xi Ge
67bbab7e02 serialization: encode allowable client names in binary module format 2022-11-25 18:43:40 -08:00
Erik Eckstein
ab1b343dad use new llvm::Optional API
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`

The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.

rdar://102362022
2022-11-21 19:44:24 +01:00
Alexis Laferrière
4a582806dc [ModuleInterface] Fix silencing errors mode in swiftinterface rebuild
Make sure we disable forwarding diagnostics from the underlying instance
when building a swiftinterface in silencing errors mode.
2022-11-08 09:31:17 -08:00
Alexis Laferrière
5d59a8fe16 [ModuleInterface] Implement silencing errors on rebuild with a new engine 2022-10-31 10:59:20 -07:00
Alexis Laferrière
2854c1b3cb [Serialization] Write in the swiftmodule if it's built from a swiftinterface
This information will allow us to distinguish swiftmodule built from
source vs swiftinterface.
2022-10-27 18:51:28 -07:00
Artem Chikin
a90d3e2de1 Factor out reading flags/version from an interface from setting up a Compiler sub-instance 2022-08-16 08:36:58 -07:00
Artem Chikin
7fd2a29fb7 Refactor 'ModuleInterfaceBuilder' to separate CompilerInstance setup logic from compilation logic (moved to ExplicitModuleInterfaceBuilder). 2022-08-16 08:36:55 -07:00
Ben Barham
22d62bb4e0 Merge remote-tracking branch 'origin/main' into rebranch
Trivial conflict caused by the line above the
`IGM.constructInitialFnAttributes` change in `lib/IRGen/GenDecl.cpp`
having an extra argument passed in rebranch (due to the new LLVM API).
2022-08-03 19:12:06 -07:00
Artem Chikin
7bdec998b1 Add flag that allows ignoring compiler flags specified in an interface file when running a '-compile-module-from-interface' frontend action. 2022-08-02 10:54:52 -07:00
Ben Barham
427a689a96 [next] Convert const char * fields to StringRef
llvm/llvm-project d0262c2394f46bb7da2a75529413d625c70908e5 added a new
default bool param to the two constructors in `SmallVectorMemoryBuffer`.
Since `options.OutputPath` is a `const char *` and that can be promoted
to a `bool`, the constructor being called was changed to the first
constructor (with a default buffer name) - promotion is preferred over
conversion.

Convert the various output paths to a `StringRef` - all their uses
converted to `StringRef` anyway. Also specify the default parameter in
order to maintain the old behaviour, which didn't require a null
terminator.
2022-05-11 17:06:29 -07:00
Alex Hoppen
669e3f34a6 Merge pull request #40155 from ahoppen/pr/improve-module-search-path-lookup
[Serialization] Improve module loading performance
2021-12-20 18:09:17 +01:00
Alex Hoppen
fe7878ecce [Serialization] Improve module loading performance
When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.

To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
  Module1.framework

searchPath2/
  Module1.framework
  Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
2021-12-14 12:44:13 +01:00
Alex Hoppen
63c31033fc [Frontend] Load standard libarary in CompilerInstance::setup
Instead of checking that the stdlib can be loaded in a variety of places, check it when setting up the compiler instance. This required a couple more checks to avoid loading the stdlib in cases where it’s not needed.

To be able to differentiate stdlib loading failures from other setup errors, make `CompilerInstance::setup` return an error message on failure via an inout parameter. Consume that error on the call side, replacing a previous, more generic error message, adding error handling where appropriate or ignoring the error message, depending on the context.
2021-12-13 15:32:08 +01:00
Xi Ge
f77ca2e2ae Frontend: add a flag to downgrade all module interface verification errors to warnings
Ideally, module interface verification should fail the build when fatal error occurs when
type checking emitted module interfaces. However, we found it's hard to stage this phase in
because the ideal case requires all Swift adopters to have valid interfaces. This new front-end flag allows
driver to downgrade all interface verification errors to warnings as an intermediate step.
2021-11-09 15:58:42 -08:00
Meghana Gupta
f458d9b490 Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag (#39516)
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag

This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.

* Updated tests
2021-10-04 18:46:40 -07:00
Xi Ge
f97653ef37 Frontend: teach -emit-module and -merge-modules to emit ABI descriptor files 2021-09-21 16:51:52 -07:00
Alexis Laferrière
c38d1773d2 [Serialization] Restrict loading swiftmodule files to the builder's SDK
Serialize the canonical name of the SDK used when building a swiftmodule
file and use it to ensure that the swiftmodule file is loaded only with
the same SDK. The SDK name must be passed down from the frontend.

This will report unsupported configurations like:

- Installing roots between incompatible SDKs without deleting the
swiftmodule files.
- Having multiple targets in the same project using different SDKs.
- Loading a swiftmodule created with a newer SDK (and stdlib) with an
older SDK.

All of these lead to hard to investigate deserialization failures and
this change should detect them early, before reaching a deserialization
failure.

rdar://78048939
2021-09-13 16:44:08 -07:00
Xi Ge
1a660c08ca Frontend: teach -compile-module-from-interface action to emit ABI descriptor as byproduct
We have implemented a libSwiftDriver-based tool to generate prebuilt module cache for
entire SDKs. Anchored on the same infrastructure, we could also generate ABI baselines
for entire SDKs.
2021-08-20 15:47:10 -07:00
Xi Ge
35cb97b8ad ABIChecker: minor refactoring to move code to APIDigester lib. NFC 2021-08-19 22:27:14 -07:00
Xi Ge
b6cd513534 Frontend: teach the compiler to use a backup directory to find .swiftinterface files to compile
This mechanism allows the compiler to use a backup interface file to build into a binary module when
a corresponding interface file from the SDK is failing for whatever reasons. This mechansim should be entirely opaque
to end users except several diagnostic messages communicating backup interfaces are used.

Part of rdar://77676064
2021-05-13 09:11:45 -07:00
Xi Ge
fe5c7ef995 ModuleInterface/Serialization: allow library authors to define a custom module version number
This allows library authors to pass down a project version number so that library users can conditionally
import that library based on the available version in the search paths.

Needed for rdar://73992299
2021-04-30 10:00:45 -07:00
eeckstein
f53f80ff04 Merge pull request #36928 from eeckstein/module-build-error-message
ModuleInterfaceBuilder: give a more specific error message in case of a compiler mismatch
2021-04-16 08:55:32 +02:00
Erik Eckstein
d49de1a982 ModuleInterfaceBuilder: give a more specific error message in case of a compiler mismatch 2021-04-15 14:06:17 +02:00
Becca Royal-Gordon
02c747609b Preload standard library in ModuleInterfaceBuilder
Previously, when the standard library module interface was broken, Swift would try to rebuild it repeatedly during -compile-module-from-interface jobs because `ASTContext::getStdlibModule()` would try to load the standard library again each time it was called. This led to extremely slow compilations that repeatedly emitted the same errors.

To avoid this, we make ModuleInterfaceBuilder try to load the standard library right away and bail out if it can’t.

Fixes rdar://75669548.
2021-03-25 18:48:00 -07:00
Alexis Laferrière
b209f72ff4 Merge pull request #36185 from xymus/move-lock-files
[Frontend] Lock the swiftmodule when rebuilding from the swiftinterface
2021-03-02 16:29:00 -08:00
Robert Widmann
d946df7895 Include Incremental Dependencies in Module Trace 2021-03-02 12:52:52 -08:00
Alexis Laferrière
2a1e5ed08f [Frontend] Lock the swiftmodule when rebuilding from the swiftinterface
When rebuilding a module interface from the textual interface, lock the
destination path of the created swiftmodule instead of the source
swiftinterface. The swiftinterface files are likely to be in the SDK and
may be on a read-only filesystem.

rdar://60247977
2021-03-02 11:04:00 -08:00
Xi Ge
2e0fb76654 ModuleInterface: rephrase remark message when acquiring lock file failed
The original remark message “could not acquire lock file for module interface” sounds too severe.
It may confuse users that this is a serious issue. We should rephrase it to a less obtrusive one.

rdar://70055223
2020-10-07 10:43:35 -07:00
Xi Ge
8ccee27db7 ModuleInterface: refactor ModuleInterfaceChecker out of ModuleInterfaceLoader
This refactoring allows us to drop ModuleInterfaceLoader when explicit modules
are enabled. Before this change, the dependencies scanner needs the loader to be
present to access functionalities like collecting prebuilt module candidates.
2020-10-01 10:30:48 -07:00
Kuba Mracek
a20118b54d When building modules on secondary threads via RunSafelyOnThread, request 8 MB of stack 2020-09-24 17:23:07 -07:00
Robert Widmann
a8766cce5b [NFC] Refactor InputFile's Accessors 2020-09-11 22:28:58 -06:00
Brent Royal-Gordon
8fbd449501 Merge pull request #33114 from brentdax/check-your-interfaces-at-the-door
Verify that just-emitted module interfaces parse and typecheck
2020-08-24 20:33:58 -07:00
Xi Ge
5c9b737c89 DependenciesScanner: prefer private Swift module interfaces if present
rdar://67257185
2020-08-18 13:09:07 -07:00
Brent Royal-Gordon
22e3cff700 Tweak diagnostic for failed module interface verification 2020-08-07 17:45:04 -07:00
Brent Royal-Gordon
9bc787ef5f Add a “typecheck module interface” mode 2020-08-07 16:00:15 -07:00
Xi Ge
f9396f2812 ModuleInterface: teach -compile-module-from-interface to emit forwarding module
-compile-module-from-interface action now takes arguments of -candidate-module-file.
If one of the candidate module files is up-to-date, the action emits a forwarding
module pointing to the candidate module instead of building a binary module.
2020-07-18 19:13:47 -07:00
Robert Widmann
e2a7dd0802 Merge pull request #32052 from CodaFi/is-uniquely-referenced
[NFC] Remove Extra Deduplication of External Depends
2020-05-28 10:34:28 -07:00
Robert Widmann
4e6b68fbcc [NFC] Remove Extra Deduplication of External Depends
Clang's DependencyCollector already deduplicates entries by the provided
file path.
2020-05-27 15:58:46 -07:00
Hamish Knight
bccdc0e062 NFC: Rename performSILGeneration -> performASTLowering
And also rename the underlying request and
descriptor.

This rename is motivated by the fact that the
operation may instead perform parsing of SIL files
and/or deserialization of SIB files.
2020-05-27 09:36:11 -07:00
Xi Ge
7d08a24161 ModuleInterface: reconstruct command-line arguments for building Swift module from interface explicitly 2020-05-15 10:49:45 -07:00
Xi Ge
3952fd5bf7 ModuleInterface: refactor compiler instance configuration to a standalone delegate class. NFC
Module interface builder used to maintain a separate compiler instance for
building Swift modules. The configuration of this compiler instance is also
useful for dependencies scanner because it needs to emit front-end compiler invocation
for building Swift modules explicitly.

This patch refactor the configuration out to a delegate class, and the
delegate class is also used by the dependency scanner.
2020-05-12 16:19:27 -07:00
Xi Ge
3cb8d8e814 DependencyScanner: inherit ASTContext options when configuring a sub compiler instance for interface scanner. NFC 2020-05-05 11:29:12 -07:00
Xi Ge
9bc036c050 DependencyScanner: honor additional compiler flags in interfaces files when collecting imports
Additional flags in interface files may change parsing behavior like #if
statements. We should use a fresh ASTContext with these additional
flags when parsing interface files to collect imports.

rdar://62612027
2020-05-04 22:18:11 -07:00
Brent Royal-Gordon
cd7bc73a0a Merge pull request #30462 from brentdax/futures-end-part-two
Provide fallback SourceLoc for swiftinterface build errors
2020-03-18 16:23:49 -07:00
Brent Royal-Gordon
a27fdad4e5 Provide fallback SourceLoc for swiftinterface build errors
When a swiftinterface fails to build for any of various reasons, we try to diagnose the failure at the site of the `import` declaration. But if the import is implicitly added—which happens for many SDK modules, like the standard library and ClangImporter overlays—there is no source location for the import, so the error ends up being diagnosed at <unknown>:0. This causes a number of issues; most notably, Xcode doesn’t display the diagnostic as prominently as others.

This change falls back to diagnosing the error at line 1, column 1 of the swiftinterface file itself. This is perhaps not an ideal location, and it won’t help with I/O errors where we can’t open the swiftinterface file (and therefore can’t diagnose an error in it), but it should improve the way we display most module interface building errors.
2020-03-17 18:44:31 -07:00
Dmitri Gribenko
841eeb05b0 Merge pull request #30403 from MForster/forster/string-fixes
Cherry-pick StringRef->std::string conversion fixes into `master`
2020-03-17 12:09:36 +01:00
Brent Royal-Gordon
a7a5e340aa Improve diagnostic for broken module interfaces
Currently, when a swiftinterface file fails to load, we emit the specific diagnostics for the failures, followed by a generic “failed to load module ‘Foo’” message. This PR improves that final diagnostic, particularly when the cause may be that the interface was emitted by a newer compiler using backwards-incompatible syntax.
2020-03-13 20:31:55 -07:00
Fred Riss
259d78a350 Adapt to llvm.org StringRef API change 2020-03-13 19:08:22 +01:00
John McCall
eea07b318d Preserve PrettyStackTrace context in the interface worker thread.
Because we block the calling thread, this should be safe for the existing
implementation of PrettyStackTrace; however, it would be much nicer if
LLVM provided direct support for this.

The most important thing from the trace that we want to print is the
original command line, so an alternative would be to just make a
PrettyStackTrace that printed all of the CompilerInvocation context;
I think that's accessible somewhere.  But this is a nice, simple
improvement.
2020-03-09 22:13:36 -04:00