Skip to content

Commit 8530180

Browse files
esm: --experimental-default-type flag to flip module defaults
PR-URL: #49869 Reviewed-By: Guy Bedford <guybedford@gmail.com> Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
1 parent 5570c29 commit 8530180

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

42 files changed

+541
-41
lines changed

doc/api/cli.md

+31
Original file line numberDiff line numberDiff line change
@@ -587,6 +587,36 @@ On Windows, using `cmd.exe` a single quote will not work correctly because it
587587
only recognizes double `"` for quoting. In Powershell or Git bash, both `'`
588588
and `"` are usable.
589589

590+
### `--experimental-default-type=type`
591+
592+
<!-- YAML
593+
added:
594+
- REPLACEME
595+
-->
596+
597+
> Stability: 1.0 - Early development
598+
599+
Define which module system, `module` or `commonjs`, to use for the following:
600+
601+
* String input provided via `--eval` or STDIN, if `--input-type` is unspecified.
602+
603+
* Files ending in `.js` or with no extension, if there is no `package.json` file
604+
present in the same folder or any parent folder.
605+
606+
* Files ending in `.js` or with no extension, if the nearest parent
607+
`package.json` field lacks a `"type"` field; unless the `package.json` folder
608+
or any parent folder is inside a `node_modules` folder.
609+
610+
In other words, `--experimental-default-type=module` flips all the places where
611+
Node.js currently defaults to CommonJS to instead default to ECMAScript modules,
612+
with the exception of folders and subfolders below `node_modules`, for backward
613+
compatibility.
614+
615+
Under `--experimental-default-type=module` and `--experimental-wasm-modules`,
616+
files with no extension will be treated as WebAssembly if they begin with the
617+
WebAssembly magic number (`\0asm`); otherwise they will be treated as ES module
618+
JavaScript.
619+
590620
### `--experimental-import-meta-resolve`
591621

592622
<!-- YAML
@@ -2243,6 +2273,7 @@ Node.js options that are allowed are:
22432273
* `--enable-network-family-autoselection`
22442274
* `--enable-source-maps`
22452275
* `--experimental-abortcontroller`
2276+
* `--experimental-default-type`
22462277
* `--experimental-import-meta-resolve`
22472278
* `--experimental-json-modules`
22482279
* `--experimental-loader`

doc/api/esm.md

+6-4
Original file line numberDiff line numberDiff line change
@@ -106,10 +106,11 @@ provides interoperability between them and its original module format,
106106

107107
Node.js has two module systems: [CommonJS][] modules and ECMAScript modules.
108108

109-
Authors can tell Node.js to use the ECMAScript modules loader
110-
via the `.mjs` file extension, the `package.json` [`"type"`][] field, or the
111-
[`--input-type`][] flag. Outside of those cases, Node.js will use the CommonJS
112-
module loader. See [Determining module system][] for more details.
109+
Authors can tell Node.js to use the ECMAScript modules loader via the `.mjs`
110+
file extension, the `package.json` [`"type"`][] field, the [`--input-type`][]
111+
flag, or the [`--experimental-default-type`][] flag. Outside of those cases,
112+
Node.js will use the CommonJS module loader. See [Determining module system][]
113+
for more details.
113114

114115
<!-- Anchors to make sure old links find a target -->
115116

@@ -1059,6 +1060,7 @@ resolution for ESM specifiers is [commonjs-extension-resolution-loader][].
10591060
[URL]: https://url.spec.whatwg.org/
10601061
[`"exports"`]: packages.md#exports
10611062
[`"type"`]: packages.md#type
1063+
[`--experimental-default-type`]: cli.md#--experimental-default-typetype
10621064
[`--input-type`]: cli.md#--input-typetype
10631065
[`data:` URLs]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/Data_URIs
10641066
[`export`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/export

doc/api/packages.md

+30-13
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,8 @@ along with a reference for the [`package.json`][] fields defined by Node.js.
5555

5656
## Determining module system
5757

58+
### Introduction
59+
5860
Node.js will treat the following as [ES modules][] when passed to `node` as the
5961
initial input, or when referenced by `import` statements or `import()`
6062
expressions:
@@ -67,14 +69,9 @@ expressions:
6769
* Strings passed in as an argument to `--eval`, or piped to `node` via `STDIN`,
6870
with the flag `--input-type=module`.
6971

70-
Node.js will treat as [CommonJS][] all other forms of input, such as `.js` files
71-
where the nearest parent `package.json` file contains no top-level `"type"`
72-
field, or string input without the flag `--input-type`. This behavior is to
73-
preserve backward compatibility. However, now that Node.js supports both
74-
CommonJS and ES modules, it is best to be explicit whenever possible. Node.js
75-
will treat the following as CommonJS when passed to `node` as the initial input,
76-
or when referenced by `import` statements, `import()` expressions, or
77-
`require()` expressions:
72+
Node.js will treat the following as [CommonJS][] when passed to `node` as the
73+
initial input, or when referenced by `import` statements or `import()`
74+
expressions:
7875

7976
* Files with a `.cjs` extension.
8077

@@ -84,11 +81,30 @@ or when referenced by `import` statements, `import()` expressions, or
8481
* Strings passed in as an argument to `--eval` or `--print`, or piped to `node`
8582
via `STDIN`, with the flag `--input-type=commonjs`.
8683

87-
Package authors should include the [`"type"`][] field, even in packages where
88-
all sources are CommonJS. Being explicit about the `type` of the package will
89-
future-proof the package in case the default type of Node.js ever changes, and
90-
it will also make things easier for build tools and loaders to determine how the
91-
files in the package should be interpreted.
84+
Aside from these explicit cases, there are other cases where Node.js defaults to
85+
one module system or the other based on the value of the
86+
[`--experimental-default-type`][] flag:
87+
88+
* Files ending in `.js` or with no extension, if there is no `package.json` file
89+
present in the same folder or any parent folder.
90+
91+
* Files ending in `.js` or with no extension, if the nearest parent
92+
`package.json` field lacks a `"type"` field; unless the folder is inside a
93+
`node_modules` folder. (Package scopes under `node_modules` are always treated
94+
as CommonJS when the `package.json` file lacks a `"type"` field, regardless
95+
of `--experimental-default-type`, for backward compatibility.)
96+
97+
* Strings passed in as an argument to `--eval` or piped to `node` via `STDIN`,
98+
when `--input-type` is unspecified.
99+
100+
This flag currently defaults to `"commonjs"`, but it may change in the future to
101+
default to `"module"`. For this reason it is best to be explicit wherever
102+
possible; in particular, package authors should always include the [`"type"`][]
103+
field in their `package.json` files, even in packages where all sources are
104+
CommonJS. Being explicit about the `type` of the package will future-proof the
105+
package in case the default type of Node.js ever changes, and it will also make
106+
things easier for build tools and loaders to determine how the files in the
107+
package should be interpreted.
92108

93109
### Modules loaders
94110

@@ -1337,6 +1353,7 @@ This field defines [subpath imports][] for the current package.
13371353
[`"packageManager"`]: #packagemanager
13381354
[`"type"`]: #type
13391355
[`--conditions` / `-C` flag]: #resolving-user-conditions
1356+
[`--experimental-default-type`]: cli.md#--experimental-default-typetype
13401357
[`--no-addons` flag]: cli.md#--no-addons
13411358
[`ERR_PACKAGE_PATH_NOT_EXPORTED`]: errors.md#err_package_path_not_exported
13421359
[`esm`]: https://github.com/standard-things/esm#readme

doc/node.1

+5
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,11 @@ Requires Node.js to be built with
152152
.It Fl -enable-source-maps
153153
Enable Source Map V3 support for stack traces.
154154
.
155+
.It Fl -experimental-default-type Ns = Ns Ar type
156+
Interpret as either ES modules or CommonJS modules input via --eval or STDIN, when --input-type is unspecified;
157+
.js or extensionless files with no sibling or parent package.json;
158+
.js or extensionless files whose nearest parent package.json lacks a "type" field, unless under node_modules.
159+
.
155160
.It Fl -experimental-global-webcrypto
156161
Expose the Web Crypto API on the global scope.
157162
.

lib/internal/main/check_syntax.js

+2-1
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,8 @@ function loadESMIfNeeded(cb) {
6060
async function checkSyntax(source, filename) {
6161
let isModule = true;
6262
if (filename === '[stdin]' || filename === '[eval]') {
63-
isModule = getOptionValue('--input-type') === 'module';
63+
isModule = getOptionValue('--input-type') === 'module' ||
64+
(getOptionValue('--experimental-default-type') === 'module' && getOptionValue('--input-type') !== 'commonjs');
6465
} else {
6566
const { defaultResolve } = require('internal/modules/esm/resolve');
6667
const { defaultGetFormat } = require('internal/modules/esm/get_format');

lib/internal/main/eval_stdin.js

+4-2
Original file line numberDiff line numberDiff line change
@@ -25,12 +25,14 @@ readStdin((code) => {
2525

2626
const print = getOptionValue('--print');
2727
const loadESM = getOptionValue('--import').length > 0;
28-
if (getOptionValue('--input-type') === 'module')
28+
if (getOptionValue('--input-type') === 'module' ||
29+
(getOptionValue('--experimental-default-type') === 'module' && getOptionValue('--input-type') !== 'commonjs')) {
2930
evalModule(code, print);
30-
else
31+
} else {
3132
evalScript('[stdin]',
3233
code,
3334
getOptionValue('--inspect-brk'),
3435
print,
3536
loadESM);
37+
}
3638
});

lib/internal/main/eval_string.js

+3-2
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,10 @@ markBootstrapComplete();
2525
const source = getOptionValue('--eval');
2626
const print = getOptionValue('--print');
2727
const loadESM = getOptionValue('--import').length > 0 || getOptionValue('--experimental-loader').length > 0;
28-
if (getOptionValue('--input-type') === 'module')
28+
if (getOptionValue('--input-type') === 'module' ||
29+
(getOptionValue('--experimental-default-type') === 'module' && getOptionValue('--input-type') !== 'commonjs')) {
2930
evalModule(source, print);
30-
else {
31+
} else {
3132
// For backward compatibility, we want the identifier crypto to be the
3233
// `node:crypto` module rather than WebCrypto.
3334
const isUsingCryptoIdentifier =

lib/internal/modules/esm/formats.js

+29
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,12 @@
22

33
const {
44
RegExpPrototypeExec,
5+
Uint8Array,
56
} = primordials;
67
const { getOptionValue } = require('internal/options');
78

9+
const { closeSync, openSync, readSync } = require('fs');
10+
811
const experimentalWasmModules = getOptionValue('--experimental-wasm-modules');
912

1013
const extensionFormatMap = {
@@ -35,7 +38,33 @@ function mimeToFormat(mime) {
3538
return null;
3639
}
3740

41+
/**
42+
* For extensionless files in a `module` package scope, or a default `module` scope enabled by the
43+
* `--experimental-default-type` flag, we check the file contents to disambiguate between ES module JavaScript and Wasm.
44+
* We do this by taking advantage of the fact that all Wasm files start with the header `0x00 0x61 0x73 0x6d` (`_asm`).
45+
* @param {URL} url
46+
*/
47+
function getFormatOfExtensionlessFile(url) {
48+
if (!experimentalWasmModules) { return 'module'; }
49+
50+
const magic = new Uint8Array(4);
51+
let fd;
52+
try {
53+
// TODO(@anonrig): Optimize the following by having a single C++ call
54+
fd = openSync(url);
55+
readSync(fd, magic, 0, 4); // Only read the first four bytes
56+
if (magic[0] === 0x00 && magic[1] === 0x61 && magic[2] === 0x73 && magic[3] === 0x6d) {
57+
return 'wasm';
58+
}
59+
} finally {
60+
if (fd !== undefined) { closeSync(fd); }
61+
}
62+
63+
return 'module';
64+
}
65+
3866
module.exports = {
3967
extensionFormatMap,
68+
getFormatOfExtensionlessFile,
4069
mimeToFormat,
4170
};

lib/internal/modules/esm/get_format.js

+52-7
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,27 @@
11
'use strict';
2+
23
const {
34
RegExpPrototypeExec,
45
ObjectPrototypeHasOwnProperty,
56
PromisePrototypeThen,
67
PromiseResolve,
8+
StringPrototypeIncludes,
79
StringPrototypeCharCodeAt,
810
StringPrototypeSlice,
911
} = primordials;
1012
const { basename, relative } = require('path');
1113
const { getOptionValue } = require('internal/options');
1214
const {
1315
extensionFormatMap,
16+
getFormatOfExtensionlessFile,
1417
mimeToFormat,
1518
} = require('internal/modules/esm/formats');
1619

1720
const experimentalNetworkImports =
1821
getOptionValue('--experimental-network-imports');
22+
const defaultTypeFlag = getOptionValue('--experimental-default-type');
23+
// The next line is where we flip the default to ES modules someday.
24+
const defaultType = defaultTypeFlag === 'module' ? 'module' : 'commonjs';
1925
const { getPackageType, getPackageScopeConfig } = require('internal/modules/esm/resolve');
2026
const { fileURLToPath } = require('internal/url');
2127
const { ERR_UNKNOWN_FILE_EXTENSION } = require('internal/errors').codes;
@@ -66,6 +72,18 @@ function extname(url) {
6672
return '';
6773
}
6874

75+
/**
76+
* Determine whether the given file URL is under a `node_modules` folder.
77+
* This function assumes that the input has already been verified to be a `file:` URL,
78+
* and is a file rather than a folder.
79+
* @param {URL} url
80+
*/
81+
function underNodeModules(url) {
82+
if (url.protocol !== 'file:') { return false; } // We determine module types for other protocols based on MIME header
83+
84+
return StringPrototypeIncludes(url.pathname, '/node_modules/');
85+
}
86+
6987
/**
7088
* @param {URL} url
7189
* @param {{parentURL: string}} context
@@ -74,8 +92,37 @@ function extname(url) {
7492
*/
7593
function getFileProtocolModuleFormat(url, context, ignoreErrors) {
7694
const ext = extname(url);
95+
7796
if (ext === '.js') {
78-
return getPackageType(url) === 'module' ? 'module' : 'commonjs';
97+
const packageType = getPackageType(url);
98+
if (packageType !== 'none') {
99+
return packageType;
100+
}
101+
// The controlling `package.json` file has no `type` field.
102+
if (defaultType === 'module') {
103+
// An exception to the type flag making ESM the default everywhere is that package scopes under `node_modules`
104+
// should retain the assumption that a lack of a `type` field means CommonJS.
105+
return underNodeModules(url) ? 'commonjs' : 'module';
106+
}
107+
return 'commonjs';
108+
}
109+
110+
if (ext === '') {
111+
const packageType = getPackageType(url);
112+
if (defaultType === 'commonjs') { // Legacy behavior
113+
if (packageType === 'none' || packageType === 'commonjs') {
114+
return 'commonjs';
115+
}
116+
// If package type is `module`, fall through to the error case below
117+
} else { // Else defaultType === 'module'
118+
if (underNodeModules(url)) { // Exception for package scopes under `node_modules`
119+
return 'commonjs';
120+
}
121+
if (packageType === 'none' || packageType === 'module') {
122+
return getFormatOfExtensionlessFile(url);
123+
} // Else packageType === 'commonjs'
124+
return 'commonjs';
125+
}
79126
}
80127

81128
const format = extensionFormatMap[ext];
@@ -89,12 +136,10 @@ function getFileProtocolModuleFormat(url, context, ignoreErrors) {
89136
const config = getPackageScopeConfig(url);
90137
const fileBasename = basename(filepath);
91138
const relativePath = StringPrototypeSlice(relative(config.pjsonPath, filepath), 1);
92-
suggestion = 'Loading extensionless files is not supported inside of ' +
93-
'"type":"module" package.json contexts. The package.json file ' +
94-
`${config.pjsonPath} caused this "type":"module" context. Try ` +
95-
`changing ${filepath} to have a file extension. Note the "bin" ` +
96-
'field of package.json can point to a file with an extension, for example ' +
97-
`{"type":"module","bin":{"${fileBasename}":"${relativePath}.js"}}`;
139+
suggestion = 'Loading extensionless files is not supported inside of "type":"module" package.json contexts ' +
140+
`without --experimental-default-type=module. The package.json file ${config.pjsonPath} caused this "type":"module" ` +
141+
`context. Try changing ${filepath} to have a file extension. Note the "bin" field of package.json can point ` +
142+
`to a file with an extension, for example {"type":"module","bin":{"${fileBasename}":"${relativePath}.js"}}`;
98143
}
99144
throw new ERR_UNKNOWN_FILE_EXTENSION(ext, filepath, suggestion);
100145
}

lib/internal/modules/esm/resolve.js

+2-2
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ const preserveSymlinks = getOptionValue('--preserve-symlinks');
3535
const preserveSymlinksMain = getOptionValue('--preserve-symlinks-main');
3636
const experimentalNetworkImports =
3737
getOptionValue('--experimental-network-imports');
38-
const typeFlag = getOptionValue('--input-type');
38+
const inputTypeFlag = getOptionValue('--input-type');
3939
const { URL, pathToFileURL, fileURLToPath, isURL } = require('internal/url');
4040
const { getCWDURL } = require('internal/util');
4141
const { canParse: URLCanParse } = internalBinding('url');
@@ -1112,7 +1112,7 @@ function defaultResolve(specifier, context = {}) {
11121112
// input, to avoid user confusion over how expansive the effect of the
11131113
// flag should be (i.e. entry point only, package scope surrounding the
11141114
// entry point, etc.).
1115-
if (typeFlag) { throw new ERR_INPUT_TYPE_NOT_ALLOWED(); }
1115+
if (inputTypeFlag) { throw new ERR_INPUT_TYPE_NOT_ALLOWED(); }
11161116
}
11171117

11181118
conditions = getConditionsSet(conditions);

lib/internal/modules/run_main.js

+6-3
Original file line numberDiff line numberDiff line change
@@ -43,12 +43,15 @@ function shouldUseESMLoader(mainPath) {
4343
*/
4444
const userImports = getOptionValue('--import');
4545
if (userLoaders.length > 0 || userImports.length > 0) { return true; }
46-
const { readPackageScope } = require('internal/modules/cjs/loader');
47-
// Determine the module format of the main
46+
47+
// Determine the module format of the entry point.
4848
if (mainPath && StringPrototypeEndsWith(mainPath, '.mjs')) { return true; }
4949
if (!mainPath || StringPrototypeEndsWith(mainPath, '.cjs')) { return false; }
50+
51+
const { readPackageScope } = require('internal/modules/cjs/loader');
5052
const pkg = readPackageScope(mainPath);
51-
return pkg && pkg.data.type === 'module';
53+
// No need to guard `pkg` as it can only be an object or `false`.
54+
return pkg.data?.type === 'module' || getOptionValue('--experimental-default-type') === 'module';
5255
}
5356

5457
/**

0 commit comments

Comments
 (0)