Require perf optimization

[ghost]

Checklist

• make -j4 test (UNIX), or vcbuild test (Windows) passes
• tests and/or benchmarks are included
• documentation is changed or added
• commit message follows commit guidelines

Aim to solve:

Optimize the performance of require one file in a module multiple times

How

Cache a map of the requests and filenames in the parent module

Example

"use strict";

console.time(1);

for(var i = 0; i < 100000; i++){
	require('./require.cpu.empty.js');
}

console.timeEnd(1);

var Module = module.constructor;
var _resolveFilename = Module._resolveFilename

Module._resolveFilename = function(request, parent, isMain, options) {
  var res;
  
  // performance optimize here
  if(parent.resolveFilenameCache){
    if(parent.resolveFilenameCache[request]){
      return parent.resolveFilenameCache[request]
    }
  }else{
  	parent.resolveFilenameCache = {};
  }
  
  res = _resolveFilename(request, parent, isMain, options);
  
  parent.resolveFilenameCache[request] = res;
  
  return res;
}

console.time('2');

for(var i = 0; i < 100000; i++){
	require('./require.cpu.empty.js');
}

console.timeEnd('2');

// result in node V8
1: 1178.783ms
2: 10.344ms

Profile

before:

after:

Let me know and I'll append some tests or benchmarks if you think it's worth a try.

[apapirovski]

Hi @tswjs, could you explain the use case that this solves that caching it in a local variable doesn't? Thanks.

[ghost]

Hi @apapirovski , thanks for your reply.
The modules may be different with a change of requests, which means that the developer must maintain a cache map in the code.

const cache = {}

module.exports = (req, res) => {
  for(let i = 0; i < 100000; i++){
    switch (req.url) {
      case 'a':
        cache.a = cache.a ? cache.a : require('a');
        break;
      case 'b':
       cache.b = cache.b ? cache.b : require('b');
       break;
      ...
    }
  }
}

It can solve the problem, but it's really messy code. If the inside cache can solve the problem, people can use require under any circumstances without any concerns. It's a worthy change.

[bnoordhuis]

There is probably a better way to accomplish this than by adding a new property to the public module.Module object.

If you grep that file for stat.cache, you can find an optimization that we use for caching fs.stat results. You should be able to repurpose that.

[BridgeAR]

Ping @tswjs

[ghost]

Hi @bnoordhuis , do you mean use the new Map() to store the cache?

[bnoordhuis]

Something along those lines; be creative. The important point is that it stays internal.

[ghost]

Got it, I'll modify the code as you say

[ghost]

repurpose: #21404

---

By the way, do you know how to remove these irrelevant commit history. I tried
git reset --hard where_i_was
git push origin -f

But they are still there. :(