When serializing the module interface path of an interface that
is part of the SDK, we serialize relative to the SDK path. During
deserialization we need to know if a path was serialized relative
to the SDK or not. The existing logic assumes any relative path
has been serialized relative to the SDK, which makes it impossible
to compile modules from relative swiftinterface paths that are not
part of the SDK.
Update the swiftmodule file to include an attribute to show if the
path was serialized relative to the SDK or not, which is used
during deserialization to correctly reconstruct the interface path.
When compiling the swiftmodule from the textual swift interface, ensure
that we re-serialise the static or dynamic nature of the module. This is
required for proper code generation where the static and dynamic linking
is material to symbolic references. This also opens the possibility of
optimizations on other platforms via internalisation of the symbols.
Fixes: #77756
Serialize the `-public-autolink-library <name>` option to the
moduleinterface file because it can affect the LINK_LIBRARY entries in a
swiftmodule file. Without saving the option, the library won't be linked
when a module compiled from the moduleinterface is used.
This change marks the `-public-autolink-library` option as a module
interface option and reads it when building a swiftmodule by module
loader.
Although I don't plan to bring over new assertions wholesale
into the current qualification branch, it's entirely possible
that various minor changes in main will use the new assertions;
having this basic support in the release branch will simplify that.
(This is why I'm adding the includes as a separate pass from
rewriting the individual assertions)
LLVM is presumably moving towards `std::string_view` -
`StringRef::startswith` is deprecated on tip. `SmallString::startswith`
was just renamed there (maybe with some small deprecation inbetween, but
if so, we've missed it).
The `SmallString::startswith` references were moved to
`.str().starts_with()`, rather than adding the `starts_with` on
`stable/20230725` as we only had a few of them. Open to switching that
over if anyone feels strongly though.
The SDK build version is a decent heuristic for expected changes in the
SDK. Any change in SDK, to clang headers in particular, can break
references from cached swiftmodules.
Track the SDK build version as part of the swiftmodule cache hash. This
will ensure we rebuild from swiftinterfaces on SDK updates.
rdar://122655978
When index-while-building is enabled, system modules are rebuilt from
their interface with diagnostics silenced so failures are not propagated
to the build. This is enabled via a local diagnostic engine that has no
consumers.
Typechecking uses the lack of consumers to add
`ConstraintSystemFlags::SuppressDiagnostics`, which controls whether
salvaging (and output of diagnostics) is run. There are cases today
where salvaging can find a correct solution though, so we should ensure
that it's always run.
This is a quick workaround for the indexing case - we should instead
always run salvaging, regardless of whether diagnostics are suppressed
or not.
Resolves rdar://117133297.
When we run an interface verification tasks with Explicit module builds, we directly invoke a '-explicit-interface-module-build' instance with a '-typecheck-module-from-interface' action. So the builder needs to recognize this as a typechecking invocation. In implicit builds, this gets lowered into a separate compiler sub-instance with a '-typecheck' action, for some reason.
resolves rdar://115565571
Reformatting everything now that we have `llvm` namespaces. I've
separated this from the main commit to help manage merge-conflicts and
for making it a bit easier to read the mega-patch.
This is phase-1 of switching from llvm::Optional to std::optional in the
next rebranch. llvm::Optional was removed from upstream LLVM, so we need
to migrate off rather soon. On Darwin, std::optional, and llvm::Optional
have the same layout, so we don't need to be as concerned about ABI
beyond the name mangling. `llvm::Optional` is only returned from one
function in
```
getStandardTypeSubst(StringRef TypeName,
bool allowConcurrencyManglings);
```
It's the return value, so it should not impact the mangling of the
function, and the layout is the same as `std::optional`, so it should be
mostly okay. This function doesn't appear to have users, and the ABI was
already broken 2 years ago for concurrency and no one seemed to notice
so this should be "okay".
I'm doing the migration incrementally so that folks working on main can
cherry-pick back to the release/5.9 branch. Once 5.9 is done and locked
away, then we can go through and finish the replacement. Since `None`
and `Optional` show up in contexts where they are not `llvm::None` and
`llvm::Optional`, I'm preparing the work now by going through and
removing the namespace unwrapping and making the `llvm` namespace
explicit. This should make it fairly mechanical to go through and
replace llvm::Optional with std::optional, and llvm::None with
std::nullopt. It's also a change that can be brought onto the
release/5.9 with minimal impact. This should be an NFC change.
When serializing module dependencies, avoid look up the file if it is
not needed for serailization.
This also adds a proper diagnostics if the lookup failed so user can
understand which file is missing.
Using a virutal output backend to capture all the outputs from
swift-frontend invocation. This allows redirecting and/or mirroring
compiler outputs to multiple location using different OutputBackend.
As an example usage for the virtual outputs, teach swift compiler to
check its output determinism by running the compiler invocation
twice and compare the hash of all its outputs.
Virtual output will be used to enable caching in the future.
`getValue` -> `value`
`getValueOr` -> `value_or`
`hasValue` -> `has_value`
`map` -> `transform`
The old API will be deprecated in the rebranch.
To avoid merge conflicts, use the new API already in the main branch.
rdar://102362022
Trivial conflict caused by the line above the
`IGM.constructInitialFnAttributes` change in `lib/IRGen/GenDecl.cpp`
having an extra argument passed in rebranch (due to the new LLVM API).
llvm/llvm-project d0262c2394f46bb7da2a75529413d625c70908e5 added a new
default bool param to the two constructors in `SmallVectorMemoryBuffer`.
Since `options.OutputPath` is a `const char *` and that can be promoted
to a `bool`, the constructor being called was changed to the first
constructor (with a default buffer name) - promotion is preferred over
conversion.
Convert the various output paths to a `StringRef` - all their uses
converted to `StringRef` anyway. Also specify the default parameter in
order to maintain the old behaviour, which didn't require a null
terminator.
When looking for a Swift module on disk, we were scanning all module search paths if they contain the module we are searching for. In a setup where each module is contained in its own framework search path, this scaled quadratically with the number of modules being imported. E.g. a setup with 100 modules being imported form 100 module search paths could cause on the order of 10,000 checks of `FileSystem::exists`. While these checks are fairly fast (~10µs), they add up to ~100ms.
To improve this, perform a first scan of all module search paths and list the files they contain. From this, create a lookup map that maps filenames to the search paths they can be found in. E.g. for
```
searchPath1/
Module1.framework
searchPath2/
Module1.framework
Module2.swiftmodule
```
we create the following lookup table
```
Module1.framework -> [searchPath1, searchPath2]
Module2.swiftmodule -> [searchPath2]
```
Instead of checking that the stdlib can be loaded in a variety of places, check it when setting up the compiler instance. This required a couple more checks to avoid loading the stdlib in cases where it’s not needed.
To be able to differentiate stdlib loading failures from other setup errors, make `CompilerInstance::setup` return an error message on failure via an inout parameter. Consume that error on the call side, replacing a previous, more generic error message, adding error handling where appropriate or ignoring the error message, depending on the context.
Ideally, module interface verification should fail the build when fatal error occurs when
type checking emitted module interfaces. However, we found it's hard to stage this phase in
because the ideal case requires all Swift adopters to have valid interfaces. This new front-end flag allows
driver to downgrade all interface verification errors to warnings as an intermediate step.
* Fix unnecessary one-time recompile of stdlib with -enable-ossa-flag
This includes a bit in the module format to represent if the module was
compiled with -enable-ossa-modules flag. When compiling a client module
with -enable-ossa-modules flag, all dependent modules are checked for this bit,
if not on, recompilation is triggered with -enable-ossa-modules.
* Updated tests
Serialize the canonical name of the SDK used when building a swiftmodule
file and use it to ensure that the swiftmodule file is loaded only with
the same SDK. The SDK name must be passed down from the frontend.
This will report unsupported configurations like:
- Installing roots between incompatible SDKs without deleting the
swiftmodule files.
- Having multiple targets in the same project using different SDKs.
- Loading a swiftmodule created with a newer SDK (and stdlib) with an
older SDK.
All of these lead to hard to investigate deserialization failures and
this change should detect them early, before reaching a deserialization
failure.
rdar://78048939
We have implemented a libSwiftDriver-based tool to generate prebuilt module cache for
entire SDKs. Anchored on the same infrastructure, we could also generate ABI baselines
for entire SDKs.
This mechanism allows the compiler to use a backup interface file to build into a binary module when
a corresponding interface file from the SDK is failing for whatever reasons. This mechansim should be entirely opaque
to end users except several diagnostic messages communicating backup interfaces are used.
Part of rdar://77676064
This allows library authors to pass down a project version number so that library users can conditionally
import that library based on the available version in the search paths.
Needed for rdar://73992299
Previously, when the standard library module interface was broken, Swift would try to rebuild it repeatedly during -compile-module-from-interface jobs because `ASTContext::getStdlibModule()` would try to load the standard library again each time it was called. This led to extremely slow compilations that repeatedly emitted the same errors.
To avoid this, we make ModuleInterfaceBuilder try to load the standard library right away and bail out if it can’t.
Fixes rdar://75669548.
When rebuilding a module interface from the textual interface, lock the
destination path of the created swiftmodule instead of the source
swiftinterface. The swiftinterface files are likely to be in the SDK and
may be on a read-only filesystem.
rdar://60247977
The original remark message “could not acquire lock file for module interface” sounds too severe.
It may confuse users that this is a serious issue. We should rephrase it to a less obtrusive one.
rdar://70055223
