Feature request: option to mount host directories in container during sam build

[davidjb]

Describe your idea/feature/enhancement

When built natively on a host (e.g. without --use-container), SAM resources (Function or LayerVersion) can reference any arbitrary file paths, not just files within that given resource's directory. This allows a resource to reference relative paths outside of a given resource, such as in monorepo environment with shared libraries and code, reference private libraries, or other shared files. However, if building with sam build --use-container, only the given resource's source code is made available inside the build container (e.g. mounted as /tmp/samcli/source) meaning functions that try and use shared code outside this directory will fail.

Ideally, there would be a way to ensure that required paths can be exposed within the Docker container during sam build (aka docker run).

Proposal

The most flexible option to support arbitrary structured repos/filesystems in both host-based builds and container builds would be to way to configure exactly which host directory gets mounted and where inside the container. In other words, a way to configure the --volume argument to docker run for one or more directories as this is what it already does.

Akin to #5507, the mounts would be best configured on a per-resource basis for a best separation of concerns, which would imply either an additional Metadata resource attribute within the the SAM specification, or otherwise a CLI flag for sam build with support in samconfig.toml, similar to build_image, where individual resources can have their mounts configured individually.

A more general solution could also be the ability to pass a generic --container-flags argument from sam build, where a user could then fully customise any options passed to Docker, not just those related to volumes. As above, however, having the ability to set separate flags per function/layer is imperative.

Additional Details

A workaround for this could be to build separate LayerVersions for each bit of shared code, but this is finicky and requires explicitly building and re-building each layer whenever code changes, something a user would need to remember to do given how SAM's caching works. A second workaround could be to try and authorise the container to have access to the private code (such as Git/SSH keys or submodules inside the resource's directory) but this is infeasible as it would require in-development code to always be pushed prior to a build.

Using symlinks isn't feasible either as soft symlinks are relative to the host; hard symlinks cause an infinite loop to break SAM's dir_checksum calculations during build. Likewise, setting the --base-dir to be a different directory appears to have no effect on how a build container mounts directories.

[mildaniel]

I think these feature request makes a lot of sense. We have a build-in-source project that would address this issue for non-container builds, but I like this idea for container builds and per resource configurations. Let me bring it to the team for prioritization.