swift-mirror/docs/ModulesBuild.rst

:orphan:

.. @raise litre.TestsAreMissing
.. default-role:: term
.. title:: Modules Build Model

.. Import substitutions for Unicode arrows.
.. include:: <isoamsa.txt>


This document describes the process used to build a module from multiple Swift
source files.

.. contents:: :local:

See also the :doc:`Modules User Model <Modules>`.


Build Setups
============

Building a single source file
-----------------------------

A single Swift source file can be compiled to an object file using ``swift -c``.

.. sidebar:: Why is executable mode the default?

  By making executable mode the default, a single-file script can go from being
  interpreted to being compiled without any extra work. 

.. _executable-mode:
.. _library-mode:

By default, a file is compiled in "executable" mode, which

- permits top-level code, to be run via ``main`` when the program is launched,
  and
- does not generate global symbols for top-level variable declarations.

Files can also be compiled in "library" mode, which reverses these restrictions
(forbidding top-level code but allowing the declaration of module-scoped global
variables). This is accomplished by passing the ``-parse-as-library`` flag to
the compiler.

.. admonition:: TODO

  Not allowing true globals in the main source file seems like an awkward 
  limitation. The reason for this is that top-level variables should follow
  local variable definite initialization rules, but it seems like you might
  want to define true globals as well.


Building an entire module
-------------------------

Passing multiple source files to the Swift compiler driver will compile them
all to object files, then link the resulting object files into a binary. Each
file will be individually parsed and contains its own context, but non-private
declarations in each file are :ref:`implicitly visible <implicit-visibility>`
to every other file at the global scope. The compiler will save the
intermediate outputs in order to avoid recompiling every file in the module for
subsequent builds.

Passing multiple files to the compiler implies :ref:`library mode
<library-mode>`.

.. admonition:: TODO

  The compiler ought to be smart enough to figure out if there is one main
  source file and several library files. This would help the command line
  experience going from an interpreted script, spread over multiple files, to
  a compiled binary.


Interpreted Mode
----------------

The Swift compiler also supports being used as an interpreter through use of
the ``-i`` flag. In this mode, the single input file is parsed in
:ref:`executable mode <executable-mode>`. As with compilation, files can be
explicitly imported within the source using ``import``. However, in interpreter
mode *all imported source files* are processed for code generation, since
there's no chance to link separate object files later.

Swift source files support the Unix convention of a `shebang`__ line at the
top of the file; this line will be recognized and skipped in the compiler's
interpreter mode.

__ http://goto.apple.com/?http://en.wikipedia.org/wiki/Shebang_(Unix)

.. admonition:: TODO

  Is there a way to have "several files in the same module" for interpreted
  scripts? The -module-source-list option no longer makes sense to expose to
  users.


Build Details
=============

.. admonition:: FIXME

  How does Xcode tell Swift about the other targets in the project, which should
  be available for import, but may not have been built yet? (Alternately, how
  does Xcode discover that they are dependencies?)


Dependency Analysis
-------------------

The build system's first duty is to decide what needs to be rebuilt, based on
output files generated by the last build. There are two ways a file might need
to be rebuilt:

1. The user has changed that file since the last build.
2. The user changed a decl the file depended on (directly or indirectly).

The build system makes a list of all files that have been changed since the
last build. Each file is parsed and its public decls are checked against the
public decls of the last build. If the decls are the same, dependent files do
not need to be recompiled.

Depending on how the declaration is used elsewhere, some subset of the entire
module may need to be recompiled. The most conservative answer is to recompile
the entire module; here are some possible refinements to that rule:

- If a declaration is only used in a function body, the function's interface
  has not been affected, and users of the function do not need to be recompiled.
- Adding an overload requires all users of a function to be recompiled, since
  it may change the overload resolution.
- Adding or removing a method or property on a class or protocol affects any 
  subclasses, adopters, or extensions of the type, as well as any users of the
  member, because it may change name binding. However, it does not affect files
  that simply have references with the given type and only call other methods.
  This does also applies to structs and enums unless the change is to a case or
  stored property.

.. note::

  If a function that may have been inlined changes or uses a declaration that
  may have changed, any file using that function will need to be recompiled as
  well, even if the function's interface didn't change.

.. admonition:: TODO

  Optimization beyond simply comparing the decl interfaces will require some
  kind of "build index" that tracks how decls are used in each file. This is
  more fine-grained than file-granular dependencies, and also accounts for
  "anti-dependencies", where introducing a new name in file A can cause file
  B to depend on A without any change in B.


AST Serialization
-----------------

When compiling each file, the compiler will load all other files in the target
into the current module. The output of compilation will be an object file or
LLVM bitcode file (for LTO, in `wrapper format`__) containing both the compiled
code and a serialized AST. This AST has two purposes: to speed up the build,
and for use in publishing a public API for the module.

__ http://goto.apple.com/?http://llvm.org/docs/BitCodeFormat.html#bitcode-wrapper-format

When the compiler is loading the other files in the module, it will first check
if any of the object files for the other sources are present and up-to-date. If
one is, the compiler will try to load a serialized AST from it first before
falling back to the source file.

.. admonition:: TODO

  There's still a fair amount of refactoring that needs to happen for this to 
  work.

.. sidebar:: Why not teach the linker to combine ASTs, or structure them so
             that this happens automatically?

  Each object file might have cross-references to the others, and at the very
  least these references should be resolved at compile time, rather than making
  every user of the module pay the price in the future. Making this a separate
  tool means that the system linker can still be used to link Swift object
  files into a binary.

Once all source files have been compiled, the final binary must be linked. In
addition to the normal linking of object files that happens in C or
Objective-C, Swift needs to merge the serialized ASTs from all the source
files, so that the binary itself can be used as an imported module.

.. admonition:: TODO

  The tool for this AST linking hasn't been written yet.


Objective-C Interoperation
==========================

It is possible to build a mixed Swift/Objective-C target, though there are some
stricter requirements on the Objective-C header files to make sure they are
Swift-compatible. 

0. Ensure that every Objective-C header imported by Swift is modular, i.e. it
can be compiled to a Clang pcm file.

1. Parse all Swift sources. Resolve all imports to external modules.

.. note::

  The syntax for importing a local header file has not been decided. It could
  be as simple as ``import NameOfHeader``, or more like ``@header import
  NameOfHeader``, or even ``import "NameOfHeader.h"``.

2. Attempt to build a module for each Clang header, skipping any function
bodies. If a header imports the Swift world, a new AST source is added to the
Clang subcomponent of Swift. When a Swift type or value is used from the Clang
header, Swift will validate that declaration on demand (just as it does for
type-checking in pure Swift code), and vend a stripped-down version of the
declaration to Clang.

.. note::

  "imports the Swift world" is deliberately vague; most likely this will
  semantically import the entire Swift half of the current target, rather than
  just specific source files. The likely interface for this is ``@import
  NameOfTarget`` in the header file.

The exact nature of the "stripped-down" declaration depends on the kind:

- **Structs:** Everything must be validated; Clang must know the layout of a
  struct in order to use it in most cases.
  
- **Enumerations:** Enumerations bridged to Objective-C will likely require a
  fixed representation type, which will probably have to be from a set of known
  types. Enumerator values can be used in constant expressions, but can be
  resolved individually on demand.

- **Protocols:** Since protocol adoption isn't checked until a class's
  ``@implementation`` block, it's actually possible to omit a protocol's
  contents for this mode. Inherited protocols do need to be type-checked,
  because they provide information needed for testing proper covariance.

- **Classes:** Like protocols, none of a class's members *need* to be exposed
  at this point. However, we need to be careful about Clang's notion of
  "overriding"; while it is not common for Objective-C programmers to redeclare
  overridden methods in a class's ``@interface`` block, it isn't forbidden
  either.

- **Functions:** If the Clang module builder skips function bodies, there is no
  reason to import Swift function declarations into the Clang context.

- **Globals:** Almost the same as functions, but perhaps we would want to allow
  certain globals to be used in constant expressions. C++11 ``decltype`` may
  also count as an interesting case in the future where it would be good to
  know the type.

For all types, if the declaration cannot be properly validated, a forward
declaration may still be good enough to parse the Clang header.

.. admonition:: TODO

  If not, however, there should be an error message that's better than simply
  "use of incomplete type" that references the cross-language import problem.

There is one additional wrinkle here; consider this dependency graph:

  A.swift |srarr| B.h |srarr| C.swift |srarr| D.h

In compiling the module for B.h, some types may be needed from C.swift.
However, these types may in turn depend on D.h. This implies that in the middle
of building a module for B.h, we may have to turn around and build a module for
D.h as well. We can detect this by seeing that the name comes from C.swift,
that the declaration cannot be resolved on its own, and that C.swift has an
unresolved import of a Clang header D.h.

3. Now that the Swift module is complete, each Objective-C source file can be
compiled properly. This time, importing the Swift module will provide the full
interface of the module, not just the "stripped-down" minimal declarations
described above. In addition, other *headers* in the same project must be
imported as modules, so that they do not provide conflicting declarations if included both directly and indirectly via the single Swift module.

.. admonition:: TODO

  An interesting idea: To avoid putting Swift into Clang proper, we'll need
  some kind of plugin interface for loading a Swift module into a Clang AST.
  The lowest-level interface here would simply request Clang ASTs for names,
  which is mostly how the "stripped-down external AST source" will work
  (described above). However, a more stable interface would simply emit header
  files for a Swift module, which Clang could then import itself. This could
  also remove the requirement that all headers in the target must be modular,
  only the ones imported into Swift.
  
  (credit to Argyrios and Dmitri)

4. Link the interfaces. For a framework, both the Swift interface and the
Objective-C interface should be available for both Swift and Objective-C
clients.

.. admonition:: FIXME

  What does this entail? The Swift module will probably have links to the
  Objective-C half of the framework, which will look like any other dependent
  import. The Objective-C side will need to expose the Swift interface, though.
  At worst, we can always auto-generate a header file.