Robert Widmann 859b87fd8c Move The Last Pieces for Cross-Module Incremental Builds
We're going to play a dirty, dirty trick - but it'll make our users'
lives better in the end so stick with me here.

In order to build up an incremental compilation, we need two sources of
dependency information:

1) "Priors" - Swiftdeps with dependency information from the past
   build(s)
2) "Posteriors" - Swiftdeps with dependencies from after we rebuild the
   file or module or whatever

With normal swift files built in incremental mode, the priors are given by the
swiftdeps files which are generated parallel to a swift file and usually
placed in the build directory alongside the object files. Because we
have entries in the output file map, we can always know where these
swiftdeps files are. The priors are integrated by the driver and then
the build is scheduled. As the build runs and jobs complete, their
swiftdeps are reloaded and re-integrated. The resulting changes are then
traversed and more jobs are scheduled if necessary. These give us the
posteriors we desire.

A module flips this on its head. The swiftdeps information serialized
in a module functions as the *posterior* since the driver consuming the
module has no way of knowing how to rebuild the module, and because its
dependencies are, for all intents and purposes, fixed in time. The
missing piece of the puzzle is the priors. That is, we need some way of
knowing what the "past" interface of the module looked like so we can
compare it to the "present" interface. Moreover, we need to always know
where to look for these priors.

We solve this problem by serializing a file alongside the build record:
the "external" build record. This is given by a... creative encoding
of multiple source file dependency graphs into a single source file
dependency graph. The rough structure of this is:

   SourceFile => interface <BUILD_RECORD>.external
   | - Incremental External Dependency => interface <MODULE_1>.swiftmodule
   | | - <dependency> ...
   | | - <dependency> ...
   | | - <dependency> ...
   | - Incremental External Dependency => interface <MODULE_2>.swiftmodule
   | | - <dependency> ...
   | | - <dependency> ...
   | - Incremental External Dependency => interface <MODULE_3>.swiftmodule
   | - ...

Sorta, `cat`'ing a bunch of source file dependency graphs together but
with incremental external dependency nodes acting as glue.

Now for the trick:

We have to unpack this structure and integrate it to get our priors.
This is easy. The tricky bit comes in integrate itself. Because the
top-level source file node points directly at the external build record,
not the original swift modules that defined these dependency nodes, we
swap the key it wants to use (the external build record) for the
incremental external dependency acting as the "parent" of the dependency
node. We do this by following the arc we carefully laid down in the
structure above.

For rdar://69595010
Goes a long way towards rdar://48955139, rdar://64238133
2020-12-10 18:45:21 -08:00
2020-07-17 07:01:31 +02:00

Swift logo

Swift Programming Language

Architecture main Package
macOS x86_64 Build Status Build Status
Ubuntu 16.04 x86_64 Build Status Build Status
Ubuntu 18.04 x86_64 Build Status Build Status
Ubuntu 20.04 x86_64 Build Status Build Status
CentOS 8 x86_64 Build Status Build Status
Amazon Linux 2 x86_64 Build Status Build Status

Swift Community-Hosted CI Platforms

OS Architecture Build
Ubuntu 16.04 PPC64LE Build Status
Ubuntu 18.04 AArch64 Build Status
Ubuntu 20.04 AArch64 Build Status
CentOS 8 AArch64 Build Status
Amazon Linux 2 AArch64 Build Status
Android ARMv7 Build Status
Android AArch64 Build Status
Windows 2019 (VS 2017) x86_64 Build Status
Windows 2019 (VS 2019) x86_64 Build Status

Swift TensorFlow Community-Hosted CI Platforms

OS Architecture Build
Ubuntu 18.04 x86_64 Build Status
macOS 10.13 x86_64 Build Status

Welcome to Swift

Swift is a high-performance system programming language. It has a clean and modern syntax, offers seamless access to existing C and Objective-C code and frameworks, and is memory safe by default.

Although inspired by Objective-C and many other languages, Swift is not itself a C-derived language. As a complete and independent language, Swift packages core features like flow control, data structures, and functions, with high-level constructs like objects, protocols, closures, and generics. Swift embraces modules, eliminating the need for headers and the code duplication they entail.

To learn more about the programming language, visit swift.org.

Contributing to Swift

Contributions to Swift are welcomed and encouraged! Please see the Contributing to Swift guide.

To be a truly great community, Swift.org needs to welcome developers from all walks of life, with different backgrounds, and with a wide range of experience. A diverse and friendly community will have more great ideas, more unique perspectives, and produce more great code. We will work diligently to make the Swift community welcoming to everyone.

To give clarity of what is expected of our members, Swift has adopted the code of conduct defined by the Contributor Covenant. This document is used across many open source communities, and we think it articulates our values well. For more, see the Code of Conduct.

Getting Started

If you are interested in:

We also have an FAQ that answers common questions.

Swift Toolchains

Building

Swift toolchains are created using the script build-toolchain. This script is used by swift.org's CI to produce snapshots and can allow for one to locally reproduce such builds for development or distribution purposes. A typical invocation looks like the following:

  $ ./swift/utils/build-toolchain $BUNDLE_PREFIX

where $BUNDLE_PREFIX is a string that will be prepended to the build date to give the bundle identifier of the toolchain's Info.plist. For instance, if $BUNDLE_PREFIX was com.example, the toolchain produced will have the bundle identifier com.example.YYYYMMDD. It will be created in the directory you run the script with a filename of the form: swift-LOCAL-YYYY-MM-DD-a-osx.tar.gz.

Beyond building the toolchain, build-toolchain also supports the following (non-exhaustive) set of useful options::

  • --dry-run: Perform a dry run build. This is off by default.
  • --test: Test the toolchain after it has been compiled. This is off by default.
  • --distcc: Use distcc to speed up the build by distributing the c++ part of the swift build. This is off by default.
  • --sccache: Use sccache to speed up subsequent builds of the compiler by caching more c++ build artifacts. This is off by default.

More options may be added over time. Please pass --help to build-toolchain to see the full set of options.

Installing into Xcode

On macOS if one wants to install such a toolchain into Xcode:

  1. Untar and copy the toolchain to one of /Library/Developer/Toolchains/ or ~/Library/Developer/Toolchains/. E.x.:
  $ sudo tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx.tar.gz -C /
  $ tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx.tar.gz -C ~/

The script also generates an archive containing debug symbols which can be installed over the main archive allowing symbolication of any compiler crashes.

  $ sudo tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx-symbols.tar.gz -C /
  $ tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx-symbols.tar.gz -C ~/
  1. Specify the local toolchain for Xcode's use via Xcode->Toolchains.

Build Failures

Try the suggestions in Troubleshooting build issues.

Make sure you are using the correct release of Xcode.

If you have changed Xcode versions but still encounter errors that appear to be related to the Xcode version, try passing --clean to build-script.

When a new version of Xcode is released, you can update your build without recompiling the entire project by passing --reconfigure to build-script.

Learning More

Be sure to look at the documentation index for a bird's eye view of the available documentation. In particular, the documents titled Debugging the Swift Compiler and Continuous Integration for Swift are very helpful to understand before submitting your first PR.

Description
The Swift Programming Language
Readme 1.5 GiB
Languages
C++ 48.2%
Swift 47.3%
Python 1.7%
CMake 1.1%
C 0.7%
Other 0.9%