feat(vike-server): Add smart dependency handling for standalone builds

[nitedani]

Introduces a new standaloneExternals plugin that intelligently manages external dependencies in standalone server builds:

• Adds support for external property in server config to specify dependencies that should not be bundled
• Traces dependencies using Vercel's NFT to find all required files
• Implements smart hoisting rules to minimize output size:
  Packages with a single version are directly copied to node_modules/
  Packages with multiple versions are stored in node_modules/.vike/@ with the newest version linked to the top level
  Workspace packages are mapped to node_modules/
• Configures Vite and esbuild externals automatically from server config

[nitedani]

It can go into a separate extension too :) But I'd like to know what you think about this.

[nitedani]

It even works when setting standalone:true and external: ['*'], dist/server will still be standalone, but instead of bundling it will copy every dependency. I imagine it's not a common use case, but worth testing.

[magne4000]

Tell me if I understand this correctly:

In a package A, if @vercel/nft detects an asset that rollup/esbuild does not (using fs.readFile for instance), the user will also need to put package A in external (current behaviour).
With this PR, it will then copy this dependency to dist/node_modules/....

What would then be the difference with npm install --prod? If a user executes this on its machine prior to sending it to another server, wouldn't it achieve the same thing?

[nitedani]

This plugin selectively copies only the exact files the app uses, creating a minimal production build. When @vercel/nft detects assets that bundlers miss (like files loaded via fs.readFile), it ensures only those specific required files get copied.

In contrast, npm install --prod installs complete packages with all their files, regardless of whether your code uses them or not.
The plugin's approach results in significantly smaller output size, handles multiple dependency versions intelligently (using symlinks only when necessary), requires no network connection during deployment, avoids potentially problematic post-install scripts, and creates predictable, minimal builds ideal for containerization.

For example, if your code only uses a single utility function from a large library, the plugin might copy just 2-3 required files, while npm install would bring in all entire packages with potentially hundreds/thousands of unnecessary files.

This plugin only runs if standalone: enabled, when the user wants a build that is directly ready for deployment without any additional steps.

[magne4000]

It seems like you were already on your way to a Vite plugin doing the same thing. What are the differences?
If possible, I'd like vike-server to just be able to use this plugin if possible, thus reducing the maintenance scope of this package (this is my main concern), on which many other packages will rely.

[nitedani]

Yes I have an older Vite plugin, the one you linked. The differences between that and this pr are:
This version:
Packages with a single version are directly copied to node_modules/
Packages with multiple versions are stored in node_modules/.vike/ with the newest version linked to the top level
Workspace packages are mapped to node_modules/[packageName]

Old version:
Everything directly flattened to node_modules/
Doesn't handle multiple package versions correctly, for example, even if our dependecies use 2 different versions of lodash, and lodash is externalized, there would be only one lodash at dist/server/node_modules/lodash, that's incorrect and might cause unexpected issues at runtime
Doesn't handle pnpm workspaces/packages outside node_modules (for example if our project depends on a local package in the workspace that is externalized, the old plugin copied it to the wrong location in dist/server/node_modules)

The new vesion took inspiration from https://github.com/nitrojs/nitro/blob/v3/src/build/plugins/externals.ts, but added support for handling packages outside node_modules(externalized local packages)

I can maintain the plugin separately, sure. vike-server doesn't need to do anything for that to work, and nothing in this pr is truly needed. I just thought it would be nice to have it built-in, but it doesn't have to be now, maybe later is good too 😄

[nitedani]

Tried to simplify the plugin and make it as clear as I could while preserving its functionality.

[nitedani]

Also I think when a standalone build is enabled and completed by esbuild, all of its input files could be deleted from dist(esbuild.metafile.inputs), and a single .js file left in dist/server. This can save space and make it clearer which file to launch for users. If somone wants to have the non-standalone output, they can always just build it without standalone mode enabled.
When users enable standalone mode, they're typically looking for a simpler deployment experience, so having just the single output file would align well with that goal, but I understand there might be other considerations I haven't thought of.

[brillout]

FYI: https://vike.dev/prerender#keepdistserver. Maybe a setting would also be a good approach here.

Thinking out loud: maybe a self-contained package vike-server/packages/standaloner/ that encapsulates all complexity could be a good compromise 👀 I didn't read the code so I may very well be missing something. I went ahead and grabbed https://www.npmjs.com/package/standaloner though 😁

[magne4000]

@nitedani I think you're right, keeping the files does not make much sense when generating a standalone build by default, but as @brillout said, I also think that a dedicated option to keep those files intact would help.
For instance, another plugin could use this standalone plugin to create another entry on top of a standalone one, and we do not want those to be deleted.

Regarding this feature as a whole, the remaining next steps would be:

• Make this an independent Vite plugin
• Add keepDistServer option
• Add more tests (at least for use case where a package uses fs.readFile, and for packages with multiple versions)
• Update vike-server once this is released