[Enhancement]: yarn on Windows with nodeLinker set to node-modules creates symlinks as absolute paths in node_modules folder

[jeff-wishnie]

Self-service

• I'd be willing to implement a fix

Describe the bug

Yarn 3.2.4
Windows 10
Node 18.x
Docker
node-linker

Running 'yarn install' on multiple workspaces that use 'portal:' or 'workspace:' to reference other workspaces results in symlinks in the top level node_modules that are absolute paths.

Running 'yarn install' on the same monorepo creates symlinks with relative paths, as expected.

The issue is with mounting the filesystem into Docker containers for running consistent builds.

The absolute paths are broken in Docker because we mount just the project directory as a Docker volume, not an entire hard drive.

Relative paths, as created in Yarn on Linux work correctly,

E.g.

Partial listing of node_modules when yarn install is run on Linux:

common-dev-tools -> ../common/dev-tools
common-front -> ../common/front
common-front-and-back -> ../common/front-and-back

vs. exact same repo, but yarn install run from WIndows:

common-dev-tools -> C:\Users\jeff.wishnie\source\repos\xxx\common\dev-tools\     
common-front -> C:\Users\jeff.wishnie\source\repos\xxx\common\front\ 
common-front-and-back -> C:\Users\jeff.wishnie\source\repos\xx\common\front-and-back\

To reproduce

// sorry, this issue is only windows, so sherlock can't be used. Instead here are high-level reproduction steps

• Create a new project: yarn init -2
• edit .yarnrc.yml to set nodeLiker to node-modules
• Create two workspaces: packages/a; packages/b
• Edit package json to add: workspaces: packages/*
• Add package a as a dependency in package b:
  • (from package/b): yarn add a@"workspace:*"

From WINDOWS run yarn install
NOTICE: the link created in node_modules is absolute not relative

Environment

System:
    OS: Windows 10 10.0.19044
    CPU: (16) x64 11th Gen Intel(R) Core(TM) i7-11850H @ 2.50GHz
  Binaries:
    Node: 18.11.0 - 
    Yarn: 3.2.4
    npm: 8.19.2

Additional context

No response

[larixer]

It is expected, nm linker uses junctions instead of symlinks on Windows. Junctions are available on all versions of Windows. Plus the main point of nm linker is compatibility with Yarn v1, which used junctions as well. You can check that Yarn v1 behaves the same:
https://github.com/yarnpkg/berry/blob/master/packages/plugin-nm/sources/NodeModulesLinker.ts#L667-L682

I think it is okay to have experimental support for using symlinks on Windows, but their usage should be behind configuration parameter, which will use junctions by default. Turning symlinks on Windows is an unknown territory, I'm afraid many users might be affected if we turn it on by default without extensive testing it first.

[jeff-wishnie]

Ok, that explains the situation, but for my multi-platform dev team (some folks on Linux, some on Windows) the difference in behavior is definitely causing problems.

I like the idea of a configurable SymLink option--I am new to Yarn and was in fact looking for a 'relative symlinks' option in the .yarnrc.yml docs.

I'd be happy to experiment with this a proposed a PR.

I started digging through the nm linker code a bit but haven't pulled all the threads on the fsutils, ppath and other helpers to find where the juntions are actually created.

If you can point me to the general section, it'd be a help!

(though I'm guessing I can search for 'junction' and find it...)

[jeff-wishnie]

Bingo!

berry/packages/plugin-nm/sources/NodeModulesLinker.ts

Line 677 in 77e525a

if (process.platform == `win32` && (!stats || stats.isDirectory())) {

[jeff-wishnie]

@larixer I have a fork off of Master (so 4.0.0-rcX) which adds the option (not 100% sure I got the versions correct)

Before I make a PR, question: should it make the PR to master or to a 3.x branch?

Thanks!

[jeff-wishnie]

Changing the title from [Bug] to [Enhancement]--I don't appear to have perms to change the label.

[jeff-wishnie]

Here's the PR--not sure I got the versions correct,.

Changes are exclusive to nm-linker plugin, so just bumped plugin-nm and cli . not sure if everything that depends on CLI (other plugins) needs bumping as well?

[merceyz]

Before I make a PR, question: should it make the PR to master or to a 3.x branch?

master, I'll backport it if needed.

Changes are exclusive to nm-linker plugin, so just bumped plugin-nm and cli . not sure if everything that depends on CLI (other plugins) needs bumping as well?

Nothing else needs to be bumped in this case 👍