diff --git a/snapcraft/explanation/lifecycle-processing-diagram.drawio b/snapcraft/explanation/lifecycle-processing-diagram.drawio new file mode 100644 index 0000000..2fb7283 --- /dev/null +++ b/snapcraft/explanation/lifecycle-processing-diagram.drawio @@ -0,0 +1,227 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/snapcraft/explanation/lifecycle-processing-diagram.png b/snapcraft/explanation/lifecycle-processing-diagram.png new file mode 100644 index 0000000..e235ddc Binary files /dev/null and b/snapcraft/explanation/lifecycle-processing-diagram.png differ diff --git a/snapcraft/explanation/parts-lifecycle.md b/snapcraft/explanation/parts-lifecycle.md new file mode 100644 index 0000000..58c0d78 --- /dev/null +++ b/snapcraft/explanation/parts-lifecycle.md @@ -0,0 +1,155 @@ +# About the parts lifecycle + +Snapcraft is a powerful tool used for building snaps. Building a snap involves going through one or more parts. Each part follows an ordered series of five steps, which forms the *lifecycle*, ensuring a coherent and reproducible build process. + +This document aims to help anyone who seeks to build a snap and understand the inner workings of parts—including their steps, commands, directories, and dependencies. + +- [Lifecycle](#heading--lifecycle) +- [Commands](#heading--commands) +- [Directories](#heading--directories) +- [Overriding a step](#heading--overriding) +- [Processing order and dependencies](#heading--processing-order) +- [Lifecycle processing diagram](#heading--processing-diagram) + +--- +

Lifecycle

+ +A part goes through the following five steps: + +1. **Pull**: downloads or otherwise retrieves the components needed to build the part. + * You can use the [`source-*`](/t/snapcraft-parts-metadata/8336#heading--source) keywords of a part to specify which components to retrieve. For instance, if `source` points to a Git repository, the pull step will clone that repository. +1. **Build**: constructs the part from the previously pulled components. + * The [`plugin`](/t/snapcraft-plugins/4284) of a part specifies how it is constructed. The [`meson`](/t/the-meson-plugin/8623) plugin, for example, executes `meson` and `ninja` to compile source code. + * Each part is built in a separate directory, but it can use the contents of the staging area if it specifies a dependency on other parts using the `after` keyword. See [Processing order and dependencies](#heading--processing-order) for more information. + * To rename or move files, you can use the [`organize`](/t/snapcraft-yaml-schema/4276#p-21225-organize-79) keyword. +1. **Stage**: copies the built components into the staging area. + * This is the first time all the different parts that make up the snap are actually placed in the same directory. If multiple parts provide the same file with differing contents, you will get a conflict. You can avoid these conflicts by using the [`stage`](/t/snapcraft-parts-metadata/8336#heading--stage) keyword to enable or block files coming from the part. You can also use this keyword to filter out files that are not required in the snap itself, for example build files specific to a single part. +1. **Prime**: copies the staged components into the priming area. + * This is very similar to the stage step, but files go into the priming area instead of the staging area. The `prime` step exists because the staging area might still contain files that are required for the build but not for the snap. For example, if you have a part that downloads and installs a compiler, then you stage this part so other parts can use the compiler during building. You can then use the `prime` filter keyword to make sure that it doesn't get copied to the priming area, so it's not taking up space in the snap. + * To apply specific permissions to files, you can use the `permissions` keyword. + * Some extra checks are also run during this step to ensure that all dependencies are satisfied for a proper run time. If confinement was set to `classic`, then files will be scanned and, if needed, patched to work with this confinement mode. +1. **Pack**: takes the entire contents of the `prime` directory and packs it into [a snap](/t/the-snap-format/698). + +

Commands

+ +Each of these lifecycle steps can be run from the command line, and the command can be part specific or be applied to all parts in a project. + +1. `snapcraft pull []` +1. `snapcraft build []` +1. `snapcraft stage []` +1. `snapcraft prime []` +1. `snapcraft pack` + +Note that each command also executes the previous lifecycle steps, so `snapcraft pack` executes all the lifecycle steps chained together. + +To access the part environment at any stage, add the `--shell` argument. For example, `snapcraft prime --shell` will run up to the *prime* step and open a shell. See [Iterating over a build](/t/iterating-over-a-build/12143) for more details. + +

Directories

+ +When running through its lifecycle steps, a part will use different working directories. The directories' names closely follow the steps' names. + +| Environment variable | Directory | Purpose | +|--|--|--| +| `CRAFT_PART_SRC` | **`parts//src`** | the location of the source after the *pull* step | +| `CRAFT_PART_BUILD` | **`parts//build`** | the working directory during the *build* step | +| `CRAFT_PART_INSTALL`| **`parts//install`** | contains the results of the *build* step and the stage packages. It is also the directory where the `organize` event renames the built files | +| `CRAFT_STAGE` | **`stage`** | shared by all parts, this directory contains the contents of each part's `CRAFT_PART_INSTALL` after the *stage* step. It can contain development libraries, headers, and other components (e.g. pkgconfig files) that need to be accessible from other parts | +| `CRAFT_PRIME` | **`prime`** | shared by all parts, this directory holds the final components after the *prime* step | +| `CRAFT_PROJECT_DIR` | path to the current project's subtree in the filesystem | used to access resources from the project's subtree, such as an icon or version file | + + + +

Overriding a step

+ +Each plugin defines the default actions that happen during a step. This behavior can be changed in two ways: +- By using `override-` in `snapcraft.yaml`. See [Overriding steps](/t/scriptlets/4892) for more details. +- By using a local plugin. This can inherit the parent plugin or scaffolding from the original. See [Local plugins](/t/writing-local-plugins/5125) for more details. Please note that using a local plugin has been deprecated in `base: core20` and fully removed in `base: core22`. + +

Processing order and dependencies

+ +By default, `snapcraft` runs the same lifecycle step of all parts before moving to the next step. However, you can change this behavior using the `after` keyword in the definition of a part in `snapcraft.yaml`. This creates a dependency chain from one part to another. + +The ordering rules are as follows: +- Parts are ordered alphabetically by name. +- When a part requires another part (using the ``after`` keyword), the required part will go through the build and stage steps before the initial part can run its build step. + +Note that each lifecycle step depends on the completion of the previous step for that part, so to reach a desired step, all prior steps need to have successfully run. Renaming or adding parts and modifying or removing ``after`` keywords can change that order. + +

Example 1 - Parts run alphabetically

+ +```yaml + parts: + A: + plugin: go + C: + plugin: go + B: + plugin: go +``` + +```text +Pulling A +Pulling B +Pulling C +Building A +Building B +Building C +Staging A +Staging B +Staging C +... +``` + +

Example 2 - A part has a dependency

+ +```yaml + parts: + A: + plugin: go + after: + - C + C: + plugin: go + B: + plugin: go +``` + +```text +Pulling C +Pulling A +Pulling B +Building C +Skipping pull for C (already ran) +Skipping build for C (already ran) +Staging C (required to build 'A') +Building A +Building B +Skipping stage for C (already ran) +Staging A +Staging B +... +``` + +In the above example, the part named `A` is built after the part named `C` has been successfully built _and_ staged. + +

Lifecycle processing diagram

+ +![Diagram explaining the lifecycle processing order for building a snap](./lifecycle-processing-diagram.png) + +# Further reading + +If you want to learn more about the Snapcraft ecosystem, see the following resources: +* [Snapcraft](/t/snapcraft-overview/8940) for an overview of Snapcraft. +* [Adding parts](/t/adding-parts/11473) to understand what parts are and how to use them. +* [Parts environment variables](/t/parts-environment-variables/12271) for a list of part-specific environment variables that can be accessed to help build a part. +* [Snapcraft plugins](/t/snapcraft-plugins/4284) to learn more about this essential component of any part. +* [Scriptlets](https://forum.snapcraft.io/t/scriptlets/4892) for more details on how to override steps. +* [Parts lifecycle reference]() for the summarised information of this page. diff --git a/snapcraft/how-to/quickstart/how-snapcraft-builds-a-snap.md b/snapcraft/how-to/quickstart/how-snapcraft-builds-a-snap.md new file mode 100644 index 0000000..0008564 --- /dev/null +++ b/snapcraft/how-to/quickstart/how-snapcraft-builds-a-snap.md @@ -0,0 +1,91 @@ +# How Snapcraft builds snaps +The Snapcraft tool can perform a variety of operations, including: + +* Snap creation operations like build, stage, prime, pack, etc. +* Store management operations like login, register, sign-build, etc. +* Extensions-specific operations like expand-extensions and list-extensions. + +These operations translate into practical tasks like: + +* Builds snaps locally or sends remote tasks to Launchpad. +* Allows the developer to register and log into the Snap Store. +* Uploads snaps to the Snap Store. +* Promotes snaps to different channels. + +The list of available global options and commands can be checked with one of: + +```bash +snapcraft help +snapcraft help --all +``` + +

snapcraft.yaml

+ +Snaps are created using a build recipe defined in a file called `snapcraft.yaml`. + +When the snapcraft tool is executed on the command line, it will look for the file in the current project work directory, either in the top-level folder or a `snap` subdirectory. If the file is found, snapcraft will then parse its contents and progress the build toward completion. + +`snapcraft.yaml` is a configuration file written in the YAML language, with stanzas defining the application structure and behavior. When snapcraft runs and parses this file, it will use the declared information to build the snap. For developers more familiar with the traditional Linux build systems, the process is somewhat similar to the use of a Makefile or an RPM spec file. + +You can create the `snapcraft.yaml` file manually, or you can run the `snapcraft init` command to create a template file in the `snap` subdirectory. + +```bash +snapcraft init +``` + +A generated template file contains just enough data to build, split across three stanzas: + +* Metadata. +* Confinement level. +* Build definition. + +

Main definitions inside snapcraft.yaml

+ +There is no one way for how a snap ought to be assembled. However, most `snapcraft.yaml` files have the same common elements, including a number of mandatory declarations. To learn more about the supported keys, including which ones one are mandatory or optional, please refer to the [Snapcraft.yaml schema](/t/snapcraft-yaml-schema/4276). + +Below is a short list of these keys, which will be further explained in the Examples sections later in the tutorial: +* **Metadata** - describes the snap functionality and provides identifiers by which the snap can be cataloged and searched in the Snap Store. +* **Security confinement** - describes the level of security of the snap. +* **Base** - describes which set of libraries the snap will use for its functionality. The base also defines the operating system version for the snap build instance in the virtual machine or container launched by Snapcraft. For instance, base: core18 means that Snapcraft will launch an Ubuntu 18.04 virtual machine or container, the set of tools and libraries used inside the snap will originate from the Ubuntu 18.04 repository archives, and the snap applications will “think” they are running on top of an Ubuntu 18.04 system, regardless of what the actual underlying Linux distribution is. +* **Parts** - describes the components that will be used to assemble the snap. +* **Apps** - describes the applications and their commands that can be run in an installed snap. + +It is important to note several additional details: + +* A snap may contain one or more parts. +* A snap may contain none, one or more applications +* Parts can be pre-assembled binaries or they may be compiled as part of the build process. +* The parts section of the `snapcraft.yaml` file uses Snapcraft build system or language-specific plugins to simplify the build process. +* The parts section may also include a list of [build packages](/t/build-and-staging-dependencies/11451) (build-packages) that will be used to create the snap applications but will not be included in the final snap. For instance, gcc or make. +* The parts section may also include a list of [stage packages](/t/build-and-staging-dependencies/11451) (stage-packages) that will be used by the snap’s applications at runtime, e.g.: python-bcrypt. These will be obtained from the repository archives in the build instance. +* If you need to pull packages from different *apt* sources, see [Snapcraft package repositories](/t/snapcraft-package-repositories/15475). + +

Snapcraft build output

+ +See [Parts lifecycle]() to consult the Parts lifecycle steps. + +The artifact of a successful Snapcraft build run is a snap file, which is itself a compressed Squashfs archive distinguished by the .snap suffix. + +A snap may contain one or more files that allow the applications to run without reliance on the underlying host system’s libraries. A snap will contain one or more applications, daemons, configuration files, assets like icons, and other objects. + +Typically, the content of a snap will resemble a Linux filesystem layout: + +```no-highlight +drwxr-xr-x 10 igor igor 4096 Jun 10 2020 ./ +drwxrwxrwx 14 igor igor 16384 Oct 17 16:40 ../ +drwxr-xr-x 2 igor igor 4096 Jun 10 2020 bin/ +drwxr-xr-x 10 igor igor 4096 Jun 10 2020 etc/ +-rw-r--r-- 1 igor igor 14 Jun 10 2020 flavor-select +drwxr-xr-x 3 igor igor 4096 Jun 10 2020 lib/ +drwxr-xr-x 2 igor igor 4096 Jun 10 2020 lib64/ +drwxr-xr-x 3 igor igor 4096 Jun 10 2020 meta/ +drwxr-xr-x 3 igor igor 4096 Jun 10 2020 snap/ +drwxr-xr-x 7 igor igor 4096 Jun 10 2020 usr/ +drwxr-xr-x 3 igor igor 4096 Feb 26 2018 var/ +``` + +The end user can examine the contents of a snap by either looking through the 'prime' directory of the build environment, or by extracting the snap archive: + +```bash +unsquashfs .snap +``` diff --git a/snapcraft/reference/parts-lifecycle.md b/snapcraft/reference/parts-lifecycle.md new file mode 100644 index 0000000..2303eac --- /dev/null +++ b/snapcraft/reference/parts-lifecycle.md @@ -0,0 +1,13 @@ +# Parts lifecycle + +Each part is composed of five steps, known as the *lifecycle*. + +| Step | Command | Purpose | Directory | Path +| ----------- | ----------- | ----------- | ----------- | ----------- | +| **Pull** | `snapcraft pull []` | Downloads or retrieve the components' sources and external dependencies needed to build the part. | Components' sources and external dependencies are put in CRAFT_PART_**SRC** or CRAFT_PART_SRC_**WORK** if the source subdirectory is overridden. | **`parts//src`** or **`parts//src/`** | +| **Build** | `snapcraft build []` | Builds the components from the previously pulled sources. | Build the sources in CRAFT_PART_**BUILD** and places the result in CRAFT_PART_**INSTALL**. | **`parts//build`** and **`parts//install`** | +| **Stage** | `snapcraft stage []` | Copies the built components into the staging area. | The built components are put in CRAFT_**STAGE**. | **`stage`** | +| **Prime** | `snapcraft prime []` | Copies the staged components into the priming area. | The staged components are put in CRAFT_**PRIME**. | **`prime`** | +| **Pack** | `snapcraft pack or snapcraft` | Takes the contents of the prime directory and packs it into a snap. | The snap is put in CRAFT_PROJECT_**DIR**. | The path to the current project’s subtree in the filesystem. | + +See [Part lifecycle](/t/parts-lifecycle/12231) and [Parts environment variables](/t/parts-environment-variables/12271) to learn more about the Parts lifecycle and its environment variables.