Commit Graph

499 Commits

Author SHA1 Message Date
Ben Barham
f9859a545f Merge pull request #37196 from bnbarham/warn-on-invalid-sourceinfo
[Serialization] Add warning when .swiftsourceinfo is malformed
2021-05-04 16:16:49 +10:00
Xi Ge
8fa2e8a03d ModuleLoader: teach canImport to check Swift user module versions
For config condition `canImport(Foo, version: N)`, this patch teaches the compiler to check N
against the version of the Swift module Foo on disk. It returns true if the module version on
disk is greater or equal to N and returns false otherwise.

Part of rdar://73992299
2021-05-03 14:04:01 -07:00
Xi Ge
3d6662f382 ModuleInterface: refactor out a utility to extract compiler flags from a module interface file. NFC 2021-05-02 22:46:32 -07:00
Xi Ge
bbe5b83de9 Parser: teach canImport to take an additional parameter indicating the minimum module version
canImport should be able to take an additional parameter labeled by either version or
underlyingVersion. We need underlyingVersion for clang modules with Swift overlays because they
have separate version numbers. The library users are usually interested in checking the importability
of the underlying clang module instead of its Swift overlay.

Part of rdar://73992299
2021-05-02 17:47:44 -07:00
Erik Eckstein
762e9c7882 Serialization: serialize if the module is a static library
If the `-static` option is specified, store that in the generated
swiftmodule file.  When de-serializing, recover this information in the
representative SILModule.

This will be used for code generation on Windows.  It is the missing
piece to allow static linking to function properly.  It additionally
opens the path to additional optimization on ELF-ish targets - GOT, PLT
references can be avoided when the linked module is known to be static.

Co-authored by: Saleem Abdulrasool <compnerd@compnerd.org>
2021-05-02 09:34:52 -07:00
Ben Barham
bfb9205c7c [Serialization] Add warning when .swiftsourceinfo is malformed
Output a warning rather than ignoring a malformed .swiftsourceinfo
completely.

Resolves rdar://77350048
2021-05-01 11:31:24 +10:00
Xi Ge
fe5c7ef995 ModuleInterface/Serialization: allow library authors to define a custom module version number
This allows library authors to pass down a project version number so that library users can conditionally
import that library based on the available version in the search paths.

Needed for rdar://73992299
2021-04-30 10:00:45 -07:00
Ben Barham
c763ab5d1e [Serialization] Store offset of decls in .swiftsourceinfo
The locations stored in .swiftsourceinfo included the presumed file,
line, and column. When a location is requested it would read these, open
the external file, create a line map, and find the offset corresponding
to that line/column.

The offset is known during serialization though, so output it as well to
avoid having to read the file and generate the line map.

Since the serialized location is returned from `Decl::getLoc()`, it
should not be the presumed location. Instead, also output the line
directives so that the presumed location can be built as per normal
locations.

Finally, move the cache out of `Decl` and into `ASTContext`, since very
few declarations will actually have their locations deserialized. Make
sure to actually write to that cache so it's used - the old cache was
never written to.
2021-04-29 12:33:05 +10:00
Artem Chikin
0445336adf Add location to the sema_no_import_target diagnostic
Thsi diagnostic currently emits, for example:
```
could not find module Foo for target arm64; found: x86_64
```
It is sometimes very useful to know where exactly the `found` module is located, so this PR changes this diagnostic to emit:
```
could not find module Foo for target arm64; found: x86_64, at:
<Path where Foo.swiftmodule/x86_64.swiftmodule is located>
```
2021-04-13 11:02:58 -07:00
Ben Barham
3ea9bed415 [SourceKit/CursorInfo] Add constructor to call results
Cursor info for a constructor would previously give the cursor info for
the containing type only. It now also adds cursor info for the
constructor itself in a "secondary_symbols" field.

Refactor `passCursorInfoForDecl` to use a single allocator rather than
keeping track of positions in a buffer and assigning everything at the
end of the function.

Refactor the various available refactoring gathering functions to take a
SmallVectorImpl and to not copy strings where they don't need to.

Resolves rdar://75385556
2021-03-30 13:23:59 +10:00
Doug Gregor
2a28fed34d Allow one to separately specify the ABI name of a module.
Introduce a new compiler flag `-module-abi-name <name>` that uses the
given name as the ABI name for the module (rather than the module's
name in source code). The ABI name impacts name mangling and metadata.
2021-03-12 07:42:07 -08:00
Ben Barham
f410958c9c [Serialization] Import incompatible targets when allowing errors
If allowing modules to be output with compile errors
(-experimental-allow-module-with-errors), import targets regardless of
whether they are compatible or not, and still output the module. The
error diagnostic will still be output (preventing SILGen), but the AST
will be available for various editor functionality.
2021-02-25 09:14:19 +10:00
Robert Widmann
108bd50ab0 Teach DependencyTracker to track Fingerprints of Incremental Dependencies 2021-02-02 09:58:28 -08:00
Rintaro Ishizaki
a9a044c6e3 [Serialization] Serialize fingerprint, mtime, size of the source files 2021-01-13 11:49:52 -08:00
Rintaro Ishizaki
1b6e7857c2 [Serialization] Serialize/deserialize source file list 2021-01-13 11:49:52 -08:00
Arnold Schwaighofer
2b6ba2830c Serialize decls with _specialize(exported:true, target: somefunc(_:), ...) in a list for fast lookup
rdar://71430641
2020-12-11 09:58:32 -08:00
Robert Widmann
883902411b Migrate loadFingerprint onto ModuleDecl and Friends 2020-11-18 12:20:14 -08:00
Artem Chikin
3221734e7b Merge pull request #34244 from artemcm/RefactorScannerBinaryDependencies
[Dependency Scanner] Refactor ModuleDependencies to represent binary-only Swift modules explicitly
2020-10-12 17:03:25 -07:00
Artem Chikin
f9d6c6a619 [Dependency Scanner] Refactor ModuleDependencies to represent binary-only Swift modules explicitly
This matches the behavior of the current client (`swift-driver`) and reduces ambiguity in how the nodes in the graph are to be treated. Swift dependencies with a textual interface, for example, must be built into a binary module by clients. Swift dependencies without a textual interface, with only a binary module, are to be used directly, without any up-to-date checks.

Note, this is distinct from Swift dependencies that have a textual interface, for which we also detect potential pre-build binary module candidates. Those are still reported in the `details` field of textual Swift dependencies as `prebuiltModuleCandidates`.
2020-10-12 09:56:03 -07:00
Brent Royal-Gordon
b440ab7331 [NFC] Move several types/functions to Import.h
To help consolidate our various types describing imports, this commit moves the following types and methods to Import.h:

* ImplicitImports
* ImplicitStdlibKind
* ImplicitImportInfo
* ModuleDecl::ImportedModule
* ModuleDecl::OrderImportedModules (as ImportedModule::Order)
* ModuleDecl::removeDuplicateImports() (as ImportedModule::removeDuplicates())
* SourceFile::ImportFlags
* SourceFile::ImportOptions
* SourceFile::ImportedModuleDesc

This commit is large and intentionally kept mechanical—nothing interesting to see here.
2020-10-09 18:57:07 -07:00
Robert Widmann
a77f059c82 Turn on Cross-Module Incremental Dependencies!
Cross-Module incremental dependencies are a new experimental mode of the Swift driver and frontend. Through a tight partnership between the two, we enable the driver to have far greater visibility into the dependency structure of a Swift module.

Rather than invent a new model, we have chosen to extend the existing incremental compilation model that works for a single module to multiple modules. To do this, we need the frontend to emit Swift dependencies in a form the driver can consume. We could emit these metadata in the form of an extra supplementary output that summarizes the contents of a generated module. However, this approach comes with a number of downsides:

- It requires additional integration with the build system
- It assumes swiftmodule files will be consumed directly from the build directory; they are not
- It incorrectly assumes a swiftmodule has but one interface. Taken in aggregate, a swiftmodule directory has one interface *per triple*

Given this, the approach we take here is to encode these dependencies directly into the swiftmodule file itself. When frontends load these souped-up incremental swiftmodule files, they record in their own swiftdeps files that they depend on an incremental swiftmodule. Upon the next build, the driver is then able to read that module file, extract the swiftdeps information from it, and use it to influence the way it schedules jobs.

The sum total is that we neatly extend the intra-module case of incremental builds to the inter-module case by treating swiftmodule inputs not as opaque entities, but as "big ol' flat Swift files" that just export an interface like any other Swift file within the module. As a further optimization, and because clients literally cannot observe this aspect of the incremental build, we only serialize the provides (the "defs" side of a "use-def" edge) when emitting swiftdeps metadata into swiftmodule files.

rdar://69595010
2020-10-05 18:59:57 -07:00
Robert Widmann
8e73d213a0 [NFC] Add Incremental Info Bits to ModuleFileSharedCore 2020-09-24 20:07:01 -06:00
Robert Widmann
e0d67f4fb2 [NFC] Refine the Return Type of SerializedModuleLoaderBase::loadAST
This grants access to the name of the loaded file to clients, which will be important for the upcoming cross-module incremental build experiments.
2020-09-24 11:32:31 -06:00
Varun Gandhi
d3369f7711 [NFC] Rename ImportFilterKind cases to be clearer. 2020-09-23 12:50:20 -07:00
Saleem Abdulrasool
bdb3da805c Frontend: allow directory layout for Swift on non-Darwin platforms
Adjust the serialized module loader to allow directory layouts for the
Swift module on non-Darwin targets, unifying the layout across all the
platforms.  It also eases cross-architecture and cross-platform
development by having the same layout, which can enable more similar
flag usage.
2020-09-13 10:55:38 -07:00
Brent Royal-Gordon
cff4ddf13a [NFC] Adopt new ImportPath types and terminology
# Conflicts:
#	lib/IDE/CodeCompletion.cpp
2020-09-10 19:07:49 -07:00
Rintaro Ishizaki
6a0a448b01 [Serialization] Remove extInfo param from ModuleFileSharedCore::load()
Populated 'ExtendedValidationInfo' is not used at all.
2020-09-03 14:51:28 -07:00
Rintaro Ishizaki
b7c19656ab [Serialization] Minor ModuleFile/ModuleFileSharedCore improvements
* Add properties to ModuleFile which holds information from the control
  block.
* 'ExtendedValidationInfo' parameter for 'ModuleFileSharedCore::load()'
  cannot be 'nullptr'. Make it non-defaulted Rvalue reference.
2020-09-03 11:18:43 -07:00
Xi Ge
e107182f1a Merge remote-tracking branch 'apple/master' into master-rebranch 2020-09-02 18:36:37 -07:00
Rintaro Ishizaki
aa48494c74 [Serialization] Local allocate ModuleFileSharedCore.ModuleInterfacePath
'ModuleInterfacePath' is passed as 'StringRef' that memory used to
reside in ASTContext. But 'ModuleFileSharedCore' should be 'ASTContext'
independent. So copy it using its own allocator.
2020-09-02 12:52:14 -07:00
Nathan Hawes
f0c8b8e571 Manually merge remote-tracking branch 'upstream/master' into HEAD
Conflicts:
	include/swift/AST/PrettyStackTrace.h
2020-08-14 11:06:27 -07:00
Artem Chikin
cf773c3d0e [Explicit Module Loader] Refactor ESWL to use findModule instead of findModuleFilesInDirectory
And add isFramework to the Explicit Module Map and its parser.
2020-08-12 12:06:46 -07:00
Artem Chikin
40c1687fd2 [Fast Dependency Scanner] Produce information on whether an explicit module is a framework
In the fast dependency scanner, depending on whether a module intrface was found via the import search path or framework search path, encode into the dependency graph Swift module details, whether a given module is a framework.
2020-08-12 10:12:35 -07:00
Nathan Hawes
b0a2230860 Manually merge remote-tracking branch 'upstream/master' into merge-in-master 2020-07-27 09:25:01 -07:00
Xi Ge
056f00322d DependenciesScanner: refactor cross import overlay resolver to a standalone function. NFC
This change facilitates resolving cross-import overlays from transitive dependencies.
2020-07-23 17:47:47 -07:00
swift-ci
cdcf7e3c56 Merge remote-tracking branch 'origin/master' into master-rebranch 2020-07-22 17:04:32 -07:00
Alexis Laferrière
d9a88b948b Merge pull request #33052 from xymus/swiftmodule-files-are-volatile-opt
[Serialization] Move loading swiftmodule files as volatile behind a flag
2020-07-22 16:48:25 -07:00
swift-ci
627ed18e79 Merge remote-tracking branch 'origin/master' into master-rebranch 2020-07-22 12:04:54 -07:00
Alexis Laferrière
2c66f0c75d [Serialization] Move loading swiftmodule files as volatile behind a flag
Introduce a new frontend flag -enable-volatile-modules to trigger
loading swiftmodule files as volatile and avoid using mmap. Revert the
default behavior to using mmap.
2020-07-22 12:04:44 -07:00
Artem Chikin
95c8391cd7 Merge pull request #32903 from artemcm/NoImplicitWhenExplicit
[Explicit Module Builds] Prevent SerializedModuleLoader from running in Explicit Module Build mode.
2020-07-22 11:49:08 -07:00
Artem Chikin
140fd73f83 [Explicit Module Builds] Prevent SerializedModuleLoader from running in Explicit Module Build mode.
In order to avoid accidentally implicitly loading modules that are expected but were not provided as explicit inputs.

- Use either SerializedModuleLoader or ExplicitSwiftModuleLoader for loading of partial modules, depending on whether we are in Explicit Module Build or Implicit Module Build mode.
2020-07-22 09:00:40 -07:00
swift-ci
c22b1f580b Merge remote-tracking branch 'origin/master' into master-rebranch 2020-07-21 16:24:01 -07:00
Alexis Laferrière
0ede7282bb [Serialization] Add more context to why we load swiftmodule files as volatile 2020-07-21 12:17:57 -07:00
Alexis Laferrière
a4becdac81 [Serialization] Load swiftmodule files as volatile to avoid mmap
Avoid mmaping swiftmodule files to hopefully fix issues seen when
building many Swift projects in parallel on NFS. This only affects
loading ModuleFile, it doesn't affect scanning swiftmodule for
dependecies which are still handled as non-volatile.

rdar://63755989
2020-07-21 10:32:16 -07:00
Nathan Hawes
9d4ed5f39c Manually merge remote-tracking branch 'upstream/master' into manually-merge-master-to-master-rebranch 2020-07-20 16:09:55 -07:00
Nathan Hawes
244dc4a768 [AST] Rename PlatformKind::OSX to PlatformKind::macOS
Because the names are coming from a .def file used for printing too, this
simplifies the printing logic as well.
2020-07-08 16:29:31 -07:00
Argyrios Kyrtzidis
99c29145b2 [Serialization] Refactor subset of ModuleFile into ModuleFileSharedCore
The difference with `ModuleFile` is that `ModuleFileSharedCore` provides immutable data and is independent of a particular ASTContext.
It is designed to be able to be shared across multiple `ModuleFile`s of different `ASTContext`s in a thread-safe manner.
2020-07-06 00:44:36 -07:00
swift_jenkins
d883300bc1 Merge remote-tracking branch 'origin/master' into master-next 2020-07-01 17:59:25 -07:00
Xi Ge
84dd4db0b1 Dependencies Scanner: report compiled Swift module paths if they are available
For the explicit module mode, swift-driver uses -compile-module-from-interface to
generate modules from interfaces found by the dependency scanner. However, we don't
need to build the binary module if up-to-date modules are available, either adjacent
to the interface file or in the prebuilt module cache directory. This patch teaches
dependencies scanner to report these ready-to-use binary modules.
2020-07-01 11:54:15 -07:00
swift_jenkins
75bea14185 Merge remote-tracking branch 'origin/master' into master-next 2020-06-23 10:39:14 -07:00