Support ESM

[kumaranshu72]

The support with ES6 module is not provided.

Description

When we define type: "module" and define our new relic config as newrelic.cjs. The file is not found as the default file named to include the named as a constant const DEFAULT_FILENAME = 'newrelic.js' and hence the cjs file is not included.

Expected Behavior

The newrelic.cjs file should be considered for importing.

Steps to Reproduce

1. Create New Express project.
2. Add type: "module" in package.json
3. Create file newrelic.cjs (require will not work in modules. we have to use import).
4. Import 'newrelic' in the start file.

[carlo-808]

Thank for posting this issue @kumaranshu72 . I changed this issue to a feature request because the agent currently does not support instrumenting ES6 modules. We'll work to get this on the roadmap for future work.

[Dhruv-Garg79]

@carlo-808 any update on this? I am also unable to use newrelic because of this issue.

[kumaranshu72]

@Dhruv-Garg79 The alternate I used was inserting the keys and the information from environment.

[michaelgoin]

@Dhruv-Garg79 no timeframe on supporting ES modules.

We did ship an external contribution to allow loading the config as a .cjs file to allow the agent to load as a CommonJS module in an ES6 application. That being said, we wouldn't expect much besides the custom instrumentation to work in this scenario.

https://github.com/newrelic/node-newrelic/releases/tag/v7.5.0

[Dhruv-Garg79]

thanks for the update.

[bizob2828]

We will have to revisit the loader hooks to see if they provide enough functionality for us to properly support instrument ESM

[mikicho]

Hey, any news?
Also, new relic node agent does not work completely in ESM project? Or just some features? Which features?
Thanks!

[drmrbrewer]

@carlo-808 would you be able to give an update on this, and where this is on the roadmap? Thanks.

[michaelgoin]

Hi @drmrbrewer, Carlo does not work here anymore but I'm happy to chime in and also make sure the PM is aware of the continued interest here.

ESM support is still under consideration but not yet officially scheduled on the roadmap.

I see you've already upvoted this issue. We encourage others interested in this functionality to upvote as well.

The continued experimental nature of loader hooks support in Node.js itself, along with additional user burden, etc. makes this a bit more challenging of a scenario than usual which we continue to explore.

[rr0214]

Thank you @drmrbrewer for upvoting and expressing interest for ESM support. I am the team's PM. It is in consideration and we'll be monitoring this one for interest as well as evaluating it against our other initiatives.

[drmrbrewer]

@michaelgoin @rr0214 OK thanks... more and more libraries are moving over to ESM, and projects are accordingly having to migrate in order to keep up, so at some point in the near future I think that newrelic will need to follow suit if it is not be be left languishing behind.

[jedashford]

I didn't know people were still using common js. Most examples for libraries and services are all esm. Your competitors have been supporting this for a long time. https://docs.datadoghq.com/tracing/setup_overview/setup/nodejs/?tab=containers

[drmrbrewer]

@michaelgoin at the very least, it would be good to have some sort of "official" guidance from newrelic regarding how to make the best of what we've got for now, i.e. how to use newrelic in a project that is based on esm... I've seen responses here and there about how to work around certain issues but it's not clear to me how to pull this together into something coherent, and what features will be missing exactly. Is this possible, until esm is properly supported?

[drmrbrewer]

Is it expected that newrelic.recordMetric('my/metric', value) doesn't work when using newrelic in an ESM package (with "type": "module"), or am I doing something else wrong? It used to work fine, but since migrating my project to ESM I'm no longer seeing these metrics in the newrelic dashboard... there is no error when calling recordMetric() but nothing appears in the dashboard.

[solocommand]

@michaelgoin Any update on ESM plans for the node agent?

[michaelgoin]

@solocommand I'll let @coreyarnold answer that as they are the current acting PM.

I've transitioned into another role so don't want to misrepresent for the team.

[solocommand]

@coreyarnold can you give an update on ESM plans?

[coreyarnold]

@solocommand We're looking at the next group of features for planning and this is near the very top of the list. I don't have a more firm timeline yet, but I'm expecting in the next few weeks I'll have a good idea of when we should be getting this work completed and can update this issue with those details.

[bizob2828]

I'm putting this comment here to tackle when we build ESM support. Our current approach to registering instrumentation is to look at the path the application is specifying to load the module. We're finding that more recent libraries have relative paths and we have to instrument with a random relative path. This becomes very brittle and in some situations that relative path differs depending on the use case of the library. For example we had this issue in our Next.js plugin and we need a better way to register and detect we have to load instrumentation for a given module within a package and this was logged to address it: newrelic/newrelic-node-nextjs#69

[bizob2828]

This work is getting tracked in our new public jira instance: https://issues.newrelic.com/browse/NEWRELIC-3115

[solocommand]

@bizob2828 that Jira instance doesn’t seem to be publicly available

[bizob2828]

@solocommand hmm I'll poke around and let you know. Either way support for this is getting released very soon

[bizob2828]

I think our wires got crossed internally around flipping the switch on making it public. Either way be on the look out for 9.1.0 release of agent with support for ESM.

[bizob2828]

Added support for instrumentation CommonJS modules in an ESM application in 9.1.0

[mikicho]

@bizob2828 thanks!
What about instrumentation of ESM packages? Fastify will support ESM soon (if not already)

[bizob2828]

@mikicho it's coming next. We currently don't instrument any ESM packages. We wanted to incrementally deliver value.

According to the package.json of fastify, it's still commonjs. It would be good to find a ESM package because right now our plan is to build a test package to prove our hook will work.

[drmrbrewer]

@bizob2828 out of interest, how does Sentry's node.js package manage to work without apparently requiring any experimental features of node?

[bizob2828]

@drmrbrewer that's probably a question for the maintainers of the sentry package 🙂

[drmrbrewer]

I'm not particularly interested in the answer, nor would I probably understand the answer anyway, but I thought that you might :-)

[bizob2828]

@mikicho 9.2.0 supports ESM instrumentation. As I mentioned before we don't have any libraries that are ESM but we added the ability to register custom ESM instrumentation.

[jainritik]

Still facing same issue:
New Relic for Node.js is disabled due to an error:

Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/abc/Desktop/AA/repo-name/config/index.js from /Users/abc/Desktop/AA/repo-name/newrelic.cjs not supported.
Instead change the require of index.js in/Users/abc/Desktop/AA/repo-name/newrelic.cjs to a dynamic import() which is available in all CommonJS modules.

newrelic.cjs file:

const envConfig  = require('./config/index.js'); // for initializing environment variables

const config = {
  app_name: ["newRelic_test"],
  license_key: envConfig.NEWRELIC_LICENSE_KEY,
};
module.exports = {config}

console.log(JSON.stringify(config));

once client is created I have imported newrelic in the start of app as following:
import 'newrelic';

Can you please help me in this @bizob2828 I tried multiple workarounds as well like trying to use .js file
@coreyarnold @solocommand @jedashford @drmrbrewer @bizob2828

[jedashford]

Still facing same issue:
New Relic for Node.js is disabled due to an error:

Error [ERR_REQUIRE_ESM]: require() of ES Module /Users/abc/Desktop/AA/repo-name/config/index.js from /Users/abc/Desktop/AA/repo-name/newrelic.cjs not supported.
Instead change the require of index.js in/Users/abc/Desktop/AA/repo-name/newrelic.cjs to a dynamic import() which is available in all CommonJS modules.

newrelic.cjs file:

const envConfig  = require('./config/index.js'); // for initializing environment variables

const config = {
  app_name: ["newRelic_test"],
  license_key: envConfig.NEWRELIC_LICENSE_KEY,
};
module.exports = {config}

console.log(JSON.stringify(config));

once client is created I have imported newrelic in the start of app as following:
import 'newrelic';

Can you please help me in this @bizob2828 I tried multiple workarounds as well like trying to use .js file
@coreyarnold @solocommand @jedashford @drmrbrewer @bizob2828

I wish you luck, we moved to DataDog.

[bizob2828]

To be clear we support ESM and you can see here. This is just an issue with configuration which looks like it's tracked in #2275. @jedashford I'm not sure your comment is helping their situation. An interesting note we rely on the same package to support ESM as DataDog