feat(ses): hostEvaluators lockdown option

[leotm]

Refs: #1891 (tracker), tested on #2334, Endo Sync: 2025-01-29

TODO

• add new lockdown option
  • legacyHermesTaming: safe (default), unsafe
  • hostEvaluators: '_legacy' (default), 'all' (regular), 'none' (strict csp), 'no-direct' (hermes)
  • fail on safeEval (default) and no-direct (hermes) early
    • i.e. require unsafeEval and no-direct on hermes (since with not supported)
  • add assertions (on non _legacy options) and types
  • update ses/error-codes/SES_DIRECT_EVAL.md
  • update lockdown.md
  • refactor assertDirectEvalAvailable to probeHostEvaluators
    • return results, now used for further assertions
    • test Function() constructor and eval() separately then together
  • ensure backwards compatibility (lockdown({}) - no option provided)
    • before: worked with a strict CSP (assertDirectEvalAvailable swallows strict CSP error then skips SES_DIRECT_EVAL error)
    • after: 'all' as default (breaking change) with a strict CSP will error, 'none' is now required
    • endo sync: 4th option undefined, warn (with new error reporter) to use the new lockdown option
    • since getEnvironmentOption default must be a string ('all'), warn instead of SES_DIRECT_EVAL error with a strict CSP
    • default to '_legacy' under-the-hood if lockdown({}), warn to use new option; SES_DIRECT_EVAL error with a normal CSP, also when set to 'all'
    • document TODOs to replace '_legacy' with 'all' as default in future (the breaking change)
  • disable compartment-shim when bundling ses for hermes
  • disable: globalThis Compartment and testCompartmentHooks
  • disable compartmentInstance.evaluate instead of deleting compartment
    • makeCompartmentConstructor setGlobalObjectEvaluators safeEvaluate is the default (no evalTaming yet)
    • CompartmentPrototype.evaluate returns compartmentEvaluate safeEvaluate is the default (no evalTaming)
    • endo sync: throw error on usage, if env lacks direct eval support
• test on hermes
  • evalTaming: noEval/safeEval/unsafeEval
  • hostEvaluators: all/none/no-direct
  • post-lockdown: eval(42), eval('42'), Function('return 42')()
• check prev test branch (below)
• fix typos
• update smoke test with Hermes eval assertions
• enable Hermes VM in CI

Follow-up: new Compartment() fails on removeUnpermittedIntrinsics at

• Tolerating undeletable intrinsics.%CompartmentPrototype%.importNow.prototype === undefined
• Uncaught TypeError: property is not configurable

test branch https://github.com/endojs/endo/tree/ses-hermes-p2 (from #2334)

• yarn build:hermes bundle ses for hermes
• yarn test:hermes run ses/test/_hermes-smoke.js

Hermes eval behaviour on bin/hermesc (standalone compiler) and bin/hermes (vm, eshost)

// Hermes direct eval cannot access or modify variables within its closure scope (sloppy and indirect)

eval = globalThis.eval; // remains direct eval, compiler: "error: invalid assignment left-hand side"

// calling globalThis.eval(...) uses indirect eval

// calling eval on Hermes VM emits: "warning: Direct call to eval(), but lexical scope is not supported."

print(eval(1 + 1));
print(eval("1 + 1"));

function a(eval, TEST) {
  // compiler: "error: cannot declare 'eval'"
  eval("TEST=true");
  return TEST;
}
print(a(eval, false)); // vm: false

function b(eval) {
  // compiler: "error: cannot declare 'eval'"
  var TEST = false;
  eval("TEST=true");
  return TEST;
}
print(b(eval)); // vm: false

function c() {
  var TEST = false;
  eval("TEST=true");
  return TEST;
}
print(c()); // vm: false

function d() {
  var TEST2 = 42;
  return eval("TEST2");
}
print(d()); // ReferenceError: Property 'TEST2' doesn't exist

// Note: eshost only runs bin/hermes (vm), not bin/hermesc (compiler)

[leotm]

flakey test https://github.com/endojs/endo/actions/runs/13784820806/job/38550261268?pr=2723

  ✘ [fail]: endo › indirect cancellation via worker Rejected promise returned by test

[leotm]

consider expanding current evalTaming test for Hermes case

endo/packages/ses/test/evalTaming-default.test.js

Lines 4 to 21 in 088b746

	lockdown({});
	test('safe eval when evalTaming is undefined.', t => {
	// eslint-disable-next-line no-unused-vars
	const a = 0;
	// eslint-disable-next-line no-eval
	t.throws(() => eval('a'));
	// eslint-disable-next-line no-eval
	t.is(eval('1 + 1'), 2);
	// should not throw
	const compartment = new Compartment();
	compartment.evaluate('(1, eval)("1 + 1")');
	// eslint-disable-next-line no-eval
	t.is(eval.toString(), 'function eval() { [native code] }');
	// eslint-disable-next-line no-eval
	t.is(eval.toString.toString(), 'function toString() { [native code] }');
	});

[leotm]

flakey test https://github.com/endojs/endo/actions/runs/13815628699/job/38647946438?pr=2723

yarn workspace @endo/benchmark run install-engines
  shell: /usr/bin/bash -e {0}
yarn version: 4.5.1
Installing engines...
Error installing XS or V8:
➤ YN0000: · Yarn 4.5.1
➤ YN0000: ┌ Resolution step
Resolution step
➤ YN0000: └ Completed in 0s 720ms
➤ YN0000: ┌ Fetch step
Fetch step
➤ YN0000: └ Completed in 0s 220ms
➤ YN0000: ┌ Link step
Link step
➤ YN0000: └ Completed in 1s 28ms
➤ YN0000: · Done in 2s 2ms

esvu ❯ version 1.2.16
XS ❯ Checking version...
esvu ✖ TypeError: body.find is not a function
    at XSInstaller.resolveVersion (/tmp/xfs-4191b526/dlx-1965/node_modules/esvu/src/engines/xs.js:[3](https://github.com/endojs/endo/actions/runs/13815628699/job/38647946438?pr=2723#step:6:3)7:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async XSInstaller.install (/tmp/xfs-[4](https://github.com/endojs/endo/actions/runs/13815628699/job/38647946438?pr=2723#step:6:5)191b526/dlx-1965/node_modules/esvu/src/installer.js:4[5](https://github.com/endojs/endo/actions/runs/13815628699/job/38647946438?pr=2723#step:6:6):21)
    at async installEngine (/tmp/xfs-4191b52[6](https://github.com/endojs/endo/actions/runs/13815628699/job/38647946438?pr=2723#step:6:7)/dlx-1965/node_modules/esvu/src/bin.js:121:3)
    at async main (/tmp/xfs-4191b526/dlx-1965/node_modules/esvu/src/bin.js:168:[7](https://github.com/endojs/endo/actions/runs/13815628699/job/38647946438?pr=2723#step:6:8))
esvu ✖ Some engines were not installed.
➤ YN0000: · Yarn 4.5.1
➤ YN0000: ┌ Resolution step
Resolution step
➤ YN0000: └ Completed in 0s 712ms
➤ YN0000: ┌ Fetch step
Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
Link step
➤ YN0000: └ Completed in 1s
➤ YN0000: · Done in 1s [8](https://github.com/endojs/endo/actions/runs/13815628699/job/38647946438?pr=2723#step:6:9)04ms

esvu ❯ version 1.2.16
V8 ❯ Checking version...
V8 ❯ Installing version 13.6.46
V8 ❯ Downloading https://storage.googleapis.com/chromium-v8/official/canary/v8-linux64-rel-13.6.46.zip
V8 ❯ Extracting /tmp/esvu-87ab78401[9](https://github.com/endojs/endo/actions/runs/13815628699/job/38647946438?pr=2723#step:6:10)4352020570b4679677fb0d.zip
V8 ❯ Installing /tmp/esvu-87ab7840194352020570b4679677fb0d.zip-extracted
V8 ❯ Testing...
V8 ✔ Installed with bin entries: v8
esvu ✖ Some engines were not installed.
Error: Process completed with exit code 1.

endo/packages/benchmark/install-engines.sh

Lines 12 to 18 in 0def310

	if are_engines_installed; then
	echo "Engines already installed. Skipping installation."
	else
	echo "Installing engines..."
	INSTALL_OUTPU_XS=$(yarn dlx esvu install xs 2>&1) || INSTALL_STATUS_XS=$?
	INSTALL_OUTPU_V8=$(yarn dlx esvu install v8 2>&1) || INSTALL_STATUS_V8=$?
	fi

perhaps it could be worth falling back on jsvu when esvu fails @muhammadahmadasifbhatti

[erights]

Unless there is a surprise, I request that #2739 get merged before this one, and then this one adjust to that. Thanks.