Use --layout=packed for all monolithic resolves. (cherrypick of #13400)

[stuhood]

As described in #13398, we should use --layout=packed for all monolithic resolves (lockfiles, all_constraints, etc), in addition to for internal_only resolves.

Note that this is not sufficient to allow for cache reuse across internal PEXes (i.e. those created for tests) and external PEXes (pex_binary), because the repository.pex is constructed with other differing arguments.

Fixes #13398.

[ci skip-rust]
[ci skip-build-wheels]

[joshua-cannon-techlabs]

@stuhood can you elaborate on your note a bit further for those of us that understand the details less? Specifically, what would it take to have the test PEXes re-use (or point to, not sure the terminology) the repository one?

Ideally there'd be as little duplication of deps as possible for obvious reasons. If I leak the tmpdir for my test run, I see:

• pytest.pex with all the deps for pytest, likely collected from the pytest_default_lockfile.txt
  • Could this be in the cache and pointed-to here?
• requirements.pex which is a superset of pytest.pex, and a subset of repository.pex, but the deps are duplicated.
• Some other PEXes I believe are likely just noise

[stuhood]

@stuhood can you elaborate on your note a bit further for those of us that understand the details less? Specifically, what would it take to have the test PEXes re-use (or point to, not sure the terminology) the repository one?

The note in the description refers to the resolve for package not being reusable for test, rather than to reuse of resolves within test: as far as we know, resolves for tests should be reused subsets (albeit copied: see below) of a single resolve.

• requirements.pex which is a superset of pytest.pex, and a subset of repository.pex, but the deps are duplicated.

This is a subset of the repository.pex, and the subset is built by (IIRC) hardlinking them out of PEX's cache in ~/.caches/pants/named_caches/pex. During the subset creation though, the sandbox captures them into its database... so when they are actually materialized into a test sandbox, they are unassociated files.

For a variety of reasons, we'll likely be picking up #12716 soon in order to improve sandbox creation for all languages (either by going the FUSE route, or by adding explicit support for immutable caches), and the effect of that would be avoiding actual copies in this case.