This scanning mode allows swift-driver to query module dependencies in a batch
and in a more granular way. In short term, it could help solve a problem that
clang module dependencies may vary if target triple changes. In a longer term,
we could break a holistic dependencies graph into smaller pieces for better caching
and reusing.
This change doesn't include the implementation of using the specified scanner
arguments to set up Clang dependencies scanner. It will come in later commits.
Previously, the availability was 5.3. Since
compareProtocolConformanceDescriptors was added in 5.4 and was used by
metadata accessors with baked-in checks for arguments which matched
prespecializations, 5.3 was incorrect. Moreover, now that the searching
for matches is done by getGenericMetadata, the metadata accessors no
longer contain the early exits, so running against a 5.3 runtime would
entail the metadata accessor failing to produce canonical prespecialized
records.
Here, the availability is bumped to 5.4 which includes the runtime
changes to support the metadata accessors not having early exits to
return prespecialized records.
Optimizes String operations with constant operands.
Specifically:
* Replaces x.append(y) with x = y if x is empty.
* Removes x.append("")
* Replaces x.append(y) with x = x + y if x and y are constant strings.
* Replaces _typeName(T.self) with a constant string if T is statically known.
With this optimization it's possible to constant fold string interpolations, like "the \(Int.self) type" -> "the Int type"
This new pass runs on high-level SIL, where semantic calls are still in place.
rdar://problem/65642843
The new function swift_getCanonicalSpecializedMetadata takes a metadata
request, a prespecialized non-canonical metadata, and a cache as its
arguments. The idea of the function is either to bless the provided
prespecialized metadata as canonical if there is not currently a
canonical metadata record for the type it describes or else to return
the actual canonical metadata.
When called, the metadata cache checks for a preexisting entry for this
metadata. If none is found, the passed-in prespecialized metadata is
added to the cache. Otherwise, the metadata record found in the cache
is returned.
rdar://problem/56995359
For the explicit module mode, swift-driver uses -compile-module-from-interface to
generate modules from interfaces found by the dependency scanner. However, we don't
need to build the binary module if up-to-date modules are available, either adjacent
to the interface file or in the prebuilt module cache directory. This patch teaches
dependencies scanner to report these ready-to-use binary modules.
The new function swift_compareProtocolConformanceDescriptors calls
through to the preexisting code in MetadataCacheKey which has been
extracted out from MetadataCacheKey::compareWitnessTables into a new
public static function
MetadataCacheKey::compareProtocolConformanceDescriptors.
The new function's availability is "future" for now.
The new function `swift_compareTypeContextDescriptors` is equivalent to
a call through to swift::equalContexts. The implementation it the same
as that of swift::equalContexts with the following removals:
- Handling of context descriptors of kind other outside of
ContextDescriptorKind::Type_First...ContextDescriptorKind::Type_Last.
Because the arguments are both TypeContextDescriptors, the kinds are
known to fall within that range.
- Casting to TypeContextDescriptor. The arguments are already of that
type.
For now, the new function has "future" availability.
There's no reason clients need to be able to access this data directly.
It obscures where module loading is actually happening, and makes it too
easy to accidentally register a module with the wrong identifier in the
context.
Hide the registration operations behind opaque accessors.
Currently when parsing a SourceFile, the parser
gets handed pointers so that it can write the
interface hash and collected tokens directly into
the file. It can also call `setSyntaxRoot` at
the end of parsing to set the syntax tree.
In preparation for the removal of
`performParseOnly`, this commit formalizes these
values as outputs of `ParseSourceFileRequest`,
ensuring that the file gets parsed when the
interface hash, collected tokens, or syntax tree
is queried.
The ClangImporter currently calls into
`ObjCSelector`'s `lookupDirect` in a couple of
places, stashing the selector in a DenseMap to try
and avoid re-entrancy problems.
However this will become a problem once
`ObjCSelector`'s `lookupDirect` is both
requestified and starts pulling in members from
the main module, so migrate the ClangImporter off
calling it.
Fortunately most of its uses only care about decls
with associated Clang nodes. For those cases, we
can use the existing member table, making sure to
populate it with any method we import.
In one case, the ClangImporter needs to check to
see if there's a deserialized Swift method with a
matching selector. Instead of calling through to
`lookupDirect`, let's just query the Swift module
loaders directly.
Module interface builder used to maintain a separate compiler instance for
building Swift modules. The configuration of this compiler instance is also
useful for dependencies scanner because it needs to emit front-end compiler invocation
for building Swift modules explicitly.
This patch refactor the configuration out to a delegate class, and the
delegate class is also used by the dependency scanner.
Additional flags in interface files may change parsing behavior like #if
statements. We should use a fresh ASTContext with these additional
flags when parsing interface files to collect imports.
rdar://62612027
Implement a new "fast" dependency scanning option,
`-scan-dependencies`, in the Swift frontend that determines all
of the source file and module dependencies for a given set of
Swift sources. It covers four forms of modules:
1) Swift (serialized) module files, by reading the module header
2) Swift interface files, by parsing the source code to find imports
3) Swift source modules, by parsing the source code to find imports
4) Clang modules, using Clang's fast dependency scanning tool
A single `-scan-dependencies` operation maps out the full
dependency graph for the given Swift source files, including all
of the Swift and Clang modules that may need to be built, such
that all of the work can be scheduled up front by the Swift
driver or any other build system that understands this
option. The dependency graph is emitted as JSON, which can be
consumed by these other tools.
Add a private scratch context to the ASTContext and allow IntrinsicInfo sole access to it so it can allocate attributes into it. This removes the final dependency on the global context.
Serialize derivative function configurations per module.
`@differentiable` and `@derivative` attributes register derivatives for
`AbstractFunctionDecl`s for a particular "derivative function configuration":
parameter indices and dervative generic signature.
To find `@derivative` functions registered in other Swift modules, derivative
function configurations must be serialized per module. When configurations for
a `AbstractFunctionDecl` are requested, all configurations from imported
modules are deserialized. This module serialization technique has precedent: it
is used for protocol conformances (e.g. extension declarations for a nominal
type) and Obj-C members for a class type.
Add `AbstractFunctionDecl::getDerivativeFunctionConfigurations` entry point
for accessing derivative function configurations.
In the differentiation transform: use
`AbstractFunctionDecl::getDerivativeFunctionConfigurations` to implement
`findMinimalDerivativeConfiguration` for canonical derivative function
configuration lookup, replacing `getMinimalASTDifferentiableAttr`.
Resolves TF-1100.
Replace it with the "legacy semantic queries" bit. The remaining client
of this bit is SourceKit, which appears to require this bit be set
conditionally so certain semantic property wrapper requests return
a sentinel value.
We should migrate these requests to a syntactic interface as soon as
possible.
rdar://60516325
There were a couple of methods in LangOptions and some related ones in
Availability and ASTContext that were added more recently.
Refactor the three older checks to the newer scheme.
Rather than registering individual IRGen passes
when we want to execute them, store function
pointers to all the pass constructors on the
ASTContext. This will make it easier to requestify
the execution of pass pipelines.
As part of this, we have to change the type export rules to
prevent `@convention(c)` function types from being used in
exported interfaces if they aren't serializable. This is a
more conservative version of the original rule I had, which
was to import such function-pointer types as opaque pointers.
That rule would've completely prevented importing function-pointer
types defined in bridging headers and so simply doesn't work,
so we're left trying to catch the unsupportable cases
retroactively. This has the unfortunate consequence that we
can't necessarily serialize the internal state of the compiler,
but that was already true due to normal type uses of aggregate
types from bridging headers; if we can teach the compiler to
reliably serialize such types, we should be able to use the
same mechanisms for function types.
This PR doesn't flip the switch to use Clang function types
by default, so many of the clang-function-type-serialization
FIXMEs are still in place.
The `@differentiable` attribute marks a function as differentiable.
Example:
```
@differentiable(wrt: x, jvp: derivativeFoo where T: Differentiable)
func id<T>(_ x: T) -> T { x }
```
The `@differentiable` attribute has an optional `wrt:` clause specifying the
parameters that are differentiated "with respect to", i.e. the differentiability
parameters. The differentiability parameters must conform to the
`Differentiable` protocol.
If the `wrt:` clause is unspecified, the differentiability parameters are
currently inferred to be all parameters that conform to `Differentiable`.
The `@differentiable` attribute also has optional `jvp:` and `vjp:` labels
for registering derivative functions. These labels are deprecated in favor of
the `@derivative` attribute and will be removed soon.
The `@differentiable` attribute also has an optional `where` clause, specifying
extra differentiability requirements for generic functions.
The `@differentiable` attribute is gated by the
`-enable-experimental-differentiable-programming` flag.
Code changes:
- Add `DifferentiableAttributeTypeCheckRequest`.
- Currently, the request returns differentiability parameter indices, while
also resolving `JVPFunction`, `VJPFunction`, and
`DerivativeGenericSignature` and mutating them in-place in
`DifferentiableAttr`. This was the simplest approach that worked without
introducing request cycles.
- Add "is type-checked" bit to `DifferentiableAttr`.
- Alternatively, I tried changing `DifferentiableAttributeTypeCheckRequest` to
use `CacheKind::Cache` instead of `CacheKind::SeparatelyCached`, but it did
not seem to work: `@differentiable` attributes in non-primary-files were
left unchecked.
Type-checking rules (summary):
- `@differentiable` attribute must be declared on a function-like "original"
declaration: `func`, `init`, `subscript`, `var` (computed properties only).
- Parsed differentiability parameters must be valid (if they exist).
- Parsed `where` clause must be valid (if it exists).
- Differentiability parameters must all conform to `Differentiable`.
- Original result must all conform to `Differentiable`.
- If JVP/VJP functions are specified, they must match the expected type.
- `@differentiable(jvp:vjp:)` for derivative registration is deprecated in
favor of `@derivative` attribute, and will be removed soon.
- Duplicate `@differentiable` attributes with the same differentiability
parameters are invalid.
- For protocol requirements and class members with `@differentiable` attribute,
conforming types and subclasses must have the same `@differentiable` attribute
(or one with a superset of differentiability parameter indices) on
implementing/overriding declarations.
`TangentSpace` represents the tangent space of a type.
- For `Differentiable`-conforming types:
- The tangent space is the `TangentVector` associated type.
- For tuple types:
- The tangent space is a tuple of the elements' tangent space types, for the
elements that have a tangent space.
- Other types have no tangent space.
`TypeBase::getAutoDiffTangentSpace` gets the tangent space of a type.
`TangentSpace` is used to:
- Compute the derivative function type of a given original function type.
- Compute the type of tangent/adjoint values during automatic differentiation.
Progress towards TF-828: upstream `@differentiable` attribute type-checking.
Compatibility with earlier swift runtimes would require modifying the
runtime compatibility libraries to adjust the behavior of
checkMetadataState by way of typeForMangledNode or even
typeForMangledName. For now, simply require that a version of swift
whose runtime knows about prespecialized metadata is being targeted.
