SwingSet tools for load testing contracts #11158

mhofman · 2025-03-25T01:46:29Z

Best reviewed commit-by-commit

Description

While reproducing the heap growth investigation for FastUSDC and smart wallet, I found myself wanting more comparable GC and snapshot points.

These change make it possible for the host, in this case a load test like for FastUSDC, to perform GC and snapshotting of vats at a known point. It also performs these only for vats that have seen any activity during the load test.

There is a non trivial change to maybeSaveSnapshot which I claim is equivalent to before, just avoids paging in vats unnecessarily when snapshotting is manually triggered.

Security Considerations

None. This introduces a new host capability that isn't used in production hosts.

Scaling Considerations

None

Documentation Considerations

Hopefully we can polish the FastUSDC load test as a reference example.

Testing Considerations

Manually tested in using mhofman/fu-impact

Upgrade Considerations

None, testing only

dckc

I'm inclined to defer to other reviewers.

dckc · 2025-03-25T16:25:18Z

packages/boot/tools/supports.ts

@@ -171,6 +173,8 @@ export const getNodeTestVaultsConfig = async ({
    );
  }

+  Object.assign(config, configOverrides);


This relies on loadSwingsetConfigFile not hardening its result, right? That seems odd.

I agree with @dckc. And since I still have a goal to either replace this or reïmplement it on top of makeCosmicSwingsetTestKit, I'd like any changes to be consistent with that.

const configFromFile: SwingSetConfig & { coreProposals?: any[] } = NonNullish( await loadSwingsetConfigFile(configPath), ); const config: SwingSetConfig & { coreProposals?: any[] } = { ...configFromFile, // exclude Pegasus from core proposals because it relies on IBC to Golang // that isn't running coreProposals: configFromFile.coreProposals?.filter( spec => spec !== '@agoric/pegasus/scripts/init-core.js', ), ...configOverrides, defaultManagerType, bundleCachePath: bundleDir, }; await fsAmbientPromises.mkdir(bundleDir, { recursive: true }); // make an almost-certainly-unique file name with a fixed-length prefix const configFilenameParts = …

dckc · 2025-03-25T16:28:47Z

packages/SwingSet/src/controller/controller.js

@@ -338,6 +338,10 @@ export async function makeSwingsetController(
      return kernel.reapAllVats(previousVatPos);
    },

+    async snapshotAllVats() {


is it important to skip a test for this new feature?

dckc · 2025-03-25T16:31:53Z

packages/SwingSet/src/kernel/vat-warehouse.js

+    kernelKeeper.vatIsAlive(vatID) || Fail`${q(vatID)}: not alive`;
+    const vatKeeper = kernelKeeper.provideVatKeeper(vatID);
+    const vatOptions = vatKeeper.getOptions();
+
+    if (!vatOptions.useTranscript) {
+      return false;


I don't understand this change well enough to review it without a test.

I second @dckc's request for test coverage of the new maybeSaveSnapshot parameter.

dckc · 2025-03-27T16:01:55Z

Deploying agoric-sdk with Cloudflare Pages

Latest commit:	`097ce77`
Status:	✅ Deploy successful!
Preview URL:	https://1e6cadde.agoric-sdk.pages.dev
Branch Preview URL:	https://mhofman-boot-load-test-tools.agoric-sdk.pages.dev

View logs

gibson042

Looks mostly good, but could use a few tweaks.

gibson042 · 2025-04-01T18:36:48Z

packages/boot/tools/supports.ts

@@ -171,6 +173,8 @@ export const getNodeTestVaultsConfig = async ({
    );
  }

+  Object.assign(config, configOverrides);


I agree with @dckc. And since I still have a goal to either replace this or reïmplement it on top of makeCosmicSwingsetTestKit, I'd like any changes to be consistent with that.

const configFromFile: SwingSetConfig & { coreProposals?: any[] } = NonNullish( await loadSwingsetConfigFile(configPath), ); const config: SwingSetConfig & { coreProposals?: any[] } = { ...configFromFile, // exclude Pegasus from core proposals because it relies on IBC to Golang // that isn't running coreProposals: configFromFile.coreProposals?.filter( spec => spec !== '@agoric/pegasus/scripts/init-core.js', ), ...configOverrides, defaultManagerType, bundleCachePath: bundleDir, }; await fsAmbientPromises.mkdir(bundleDir, { recursive: true }); // make an almost-certainly-unique file name with a fixed-length prefix const configFilenameParts = …

gibson042 · 2025-04-01T18:41:33Z

packages/boot/tools/supports.ts

@@ -141,13 +141,15 @@ export const keyArrayEqual = (
 * @param options.configPath - Path to the base config file
 * @param options.defaultManagerType - SwingSet manager type to use
 * @param options.discriminator - Optional string to include in the config filename
+ * @param options.configOverrides - Other SwingSet options to set in the config


Suggested change

* @param options.configOverrides - Other SwingSet options to set in the config

* @param options.configOverrides - Other SwingSet options to set in the config

(may be overridden by more specific options such as `bundleDir` and

`defaultManagerType`)

gibson042 · 2025-04-01T18:42:46Z

packages/boot/tools/supports.ts

@@ -417,6 +421,7 @@ type AckBehaviorType = (typeof AckBehavior)[keyof typeof AckBehavior];
 * @param options.defaultManagerType - SwingSet manager type to use
 * @param options.harness - Optional run harness
 * @param options.resolveBase - Base URL or path for resolving module paths
+ * @param options.configOverrides - Other SwingSet options to set in the config


Suggested change

* @param options.configOverrides - Other SwingSet options to set in the config

* @param options.configOverrides - Other SwingSet options to set in the config

(may be overridden by more specific options such as `bundleDir` and

`defaultManagerType`)

gibson042 · 2025-04-01T18:53:24Z

packages/SwingSet/src/kernel/kernel.js

+
+  function reapAllVats(previousVatPos = {}) {
+    /** @type {Record<string, number>} */
+    const currentVatPos = {};


I have a slight preference for these vat transcript positions to be represented as a Map instance rather than an object (because arbitrarily-keyed plain objects are unnecessary hazards, and Map better communicates intent anyway).

gibson042 · 2025-04-01T19:14:49Z

packages/SwingSet/src/kernel/vat-warehouse.js

   * is satisified
   *
   * @param {VatID} vatID
+   * @param {number} [forceThreshold]


Suggested change

* is satisified

*

* @param {VatID} vatID

* @param {number} [forceThreshold]

* is satisfied or a lower explicit delivery count has been reached.

*

* @param {VatID} vatID

* @param {number} [minDeliveryCount]

gibson042 · 2025-04-01T19:57:02Z

packages/SwingSet/test/reap-all/reap-all.test.js

+  );
+
+  for (let i = 0; i < 3; i += 1) {
+    const lastExported = exportedPromises.slice(-1)[0];


Suggested change

const lastExported = exportedPromises.slice(-1)[0];

const lastExported = exportedPromises.at(-1);

Typing of Array<T>.at(pos: number): T | undefined which is extremely annoying in these cases.

gibson042 · 2025-04-01T20:10:31Z

packages/SwingSet/test/reap-all/bootstrap-reap-all.js

+    async getExport() {
+      return Far('Bootstrap exported', {});
+    },


packages/vats/tools/bootstrap-chain-reflective.js and packages/SwingSet/tools/vat-puppet.js strive to eliminate the need for vat code dedicated to any particular tests. Consider replacing packages/SwingSet/test/reap-all/bootstrap-reap-all.js with the former and packages/SwingSet/test/reap-all/vat-dumbo.js with the latter.

vat-puppet.js doesn't support defining a kind or another virtual storage mechanism allowing to make one object hold onto another. I might be able to use makeRemotable for the base object in the chain, but that's about it.

This seems not worth the effort.

gibson042 · 2025-04-01T20:14:37Z

packages/SwingSet/test/reap-all/reap-all.test.js

+  }
+  await c.run();
+
+  // Drop our interest in the intermediary promises without gaining an interest in their resolution


Suggested change

// Drop our interest in the intermediary promises without gaining an interest in their resolution

// Drop our interest in the intermediary promises without gaining an interest

// in their settlement

gibson042 · 2025-04-01T20:15:52Z

packages/SwingSet/test/reap-all/reap-all.test.js

+  t.deepEqual(postReapReapPos, postMessagesReapPos);
+  checkQueues([]);
+
+  // Drop our interest in the last promise of the chain, dropping the refcount onto its resolution


Suggested change

// Drop our interest in the last promise of the chain, dropping the refcount onto its resolution

// Drop our interest in the last promise of the chain without gaining an

// interest in its settlement

gibson042 · 2025-04-01T20:21:43Z

packages/SwingSet/test/reap-all/reap-all.test.js

+  for (const vat of ['v11', 'v10', 'v9', 'v1']) {
+    const newReapPos = c.reapAllVats(prevReapPos);
+    checkQueues([vat]);
+    const { [vat]: prevVatPos, ...prevOtherPos } = prevReapPos;
+    const { [vat]: newVatPos, ...newOtherPos } = newReapPos;
+    t.log(`vat ${vat} prev reap pos ${prevVatPos}, new reap pos ${newVatPos}`);
+    t.deepEqual(newOtherPos, prevOtherPos);
+    prevReapPos = newReapPos;
+    await c.run();
+    checkQueues([]);
+  }


Suggested change

for (const vat of ['v11', 'v10', 'v9', 'v1']) {

const newReapPos = c.reapAllVats(prevReapPos);

checkQueues([vat]);

const { [vat]: prevVatPos, ...prevOtherPos } = prevReapPos;

const { [vat]: newVatPos, ...newOtherPos } = newReapPos;

t.log(`vat ${vat} prev reap pos ${prevVatPos}, new reap pos ${newVatPos}`);

t.deepEqual(newOtherPos, prevOtherPos);

prevReapPos = newReapPos;

await c.run();

checkQueues([]);

}

for (const vatID of ['v11', 'v10', 'v9', 'v1']) {

const { [vatID]: prevVatPos, ...prevOtherPos } = prevReapPos;

const newReapPos = c.reapAllVats(prevReapPos);

checkQueues([vatID]);

const { [vatID]: newVatPos, ...newOtherPos } = newReapPos;

t.log(`vat ${vatID} reap pos prev ${prevVatPos}, new ${newVatPos}`);

t.deepEqual(newOtherPos, prevOtherPos);

prevReapPos = newReapPos;

await c.run();

checkQueues([]);

}

gibson042 · 2025-04-02T16:32:13Z

packages/SwingSet/test/reap-all/reap-all.test.js

+  // prettier-ignore
+  const allVats =
+    harden(['v1', 'v6', 'v7', 'v8', 'v5', 'v2', 'v4', 'v9', 'v10', 'v11']);


Suggested change

// prettier-ignore

const allVats =

harden(['v1', 'v6', 'v7', 'v8', 'v5', 'v2', 'v4', 'v9', 'v10', 'v11']);

const { activeVats } = c.getStatus();

t.log(activeVats.map(({ id, options }) => `${id}: ${options.name}`));

// As noted in kernel.js, the comms vat is not reapable.

const reapable = activeVats.flatMap(({ id, options }) =>

options.name === 'comms' ? [] : [id],

);

// We care about the dynamic vats.

t.true(reapable.length >= 7); // bootstrap, staticDumbo{1,2,3}, dyn{1,2,3}

const [dyn1, dyn2, dyn3] = reapable.slice(-3);

gibson042 · 2025-04-02T17:19:36Z

packages/SwingSet/test/reap-all/reap-all.test.js

+  // Because of our chained references, our GC shakes lose an object for
+  // each reap round.
+  let prevReapPos = postMessagesReapPos;
+  for (const vat of ['v11', 'v10', 'v9', 'v1']) {


Accompanying the above suggestion:

Suggested change

for (const vat of ['v11', 'v10', 'v9', 'v1']) {

for (const vat of [dyn3, dyn2, dyn1, c.vatNameToID('bootstrap')]) {

gibson042 · 2025-04-02T17:20:26Z

packages/SwingSet/test/reap-all/reap-all.test.js

+  // Create a chain of references
+
+  /** @type {string[]} */
+  const exportedPromises = [];
+  exportedPromises.push(
+    /** @type {string} */ (c.queueToVatRoot('bootstrap', 'getExport')),
+  );
+
+  for (let i = 0; i < 3; i += 1) {
+    const lastExported = exportedPromises.slice(-1)[0];
+    exportedPromises.push(
+      /** @type {string} */ (
+        c.queueToVatObject(dynamicRoots[i], 'makeHolder', [kslot(lastExported)])
+      ),
+    );
+  }


Typing of Array<T>.at(pos: number): T | undefined which is extremely annoying in these cases.

That seems to be an issue anyway, so we might as well use type casting:

Suggested change

// Create a chain of references

/** @type {string[]} */

const exportedPromises = [];

exportedPromises.push(

/** @type {string} */ (c.queueToVatRoot('bootstrap', 'getExport')),

);

for (let i = 0; i < 3; i += 1) {

const lastExported = exportedPromises.slice(-1)[0];

exportedPromises.push(

/** @type {string} */ (

c.queueToVatObject(dynamicRoots[i], 'makeHolder', [kslot(lastExported)])

),

);

}

// Create a chain of references

const exportedPromises = [

/** @type {string} */ (c.queueToVatRoot('bootstrap', 'getExport')),

];

for (let i = 0; i < 3; i += 1) {

const lastExported = /** @type {string} */ (exportedPromises.at(-1));

exportedPromises.push(

/** @type {string} */ (

c.queueToVatObject(dynamicRoots[i], 'makeHolder', [kslot(lastExported)])

),

);

}

We don't use noUncheckedIndexedAccess so not a problem in practice. I just wish JSDoc had non-null type assertions!

fwiw I'd be happy to convert the SwingSet tests to .ts

mhofman added 5 commits March 25, 2025 01:26

feat(boot-tools): make slogSender available in context

30c2b7a

feat(boot-tools): add configOverrides

289720a

refactor(swingset): reapAllVats

19ba65b

feat(swingset): reapAllVats that have advanced

11bb1a6

feat(swingset): snapshotAllVats

cea852d

mhofman marked this pull request as ready for review March 25, 2025 02:05

mhofman requested a review from a team as a code owner March 25, 2025 02:05

mhofman requested review from AgoricTriage, dckc and gibson042 March 25, 2025 02:05

dckc requested a review from amessbee March 25, 2025 16:22

dckc reviewed Mar 25, 2025

View reviewed changes

mhofman added 2 commits April 1, 2025 07:14

chore(SwingSet): refactor reap-all test

486b350

fixup! feat(swingset): reapAllVats that have advanced

4270051

Merge branch 'master' into mhofman/boot-load-test-tools

097ce77

gibson042 requested changes Apr 1, 2025

View reviewed changes

gibson042 reviewed Apr 2, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SwingSet tools for load testing contracts #11158

SwingSet tools for load testing contracts #11158

mhofman commented Mar 25, 2025

dckc left a comment

dckc Mar 25, 2025

gibson042 Apr 1, 2025

dckc Mar 25, 2025

dckc Mar 25, 2025

gibson042 Apr 1, 2025

dckc commented Mar 27, 2025

cloudflare-workers-and-pages bot commented Apr 1, 2025 •

edited

Loading

gibson042 left a comment

gibson042 Apr 1, 2025

gibson042 Apr 1, 2025

gibson042 Apr 1, 2025

gibson042 Apr 1, 2025

gibson042 Apr 1, 2025

gibson042 Apr 1, 2025

mhofman Apr 1, 2025

gibson042 Apr 1, 2025

mhofman Apr 1, 2025

gibson042 Apr 1, 2025

gibson042 Apr 1, 2025

gibson042 Apr 1, 2025

gibson042 Apr 2, 2025

gibson042 Apr 2, 2025

gibson042 Apr 2, 2025

mhofman Apr 2, 2025

turadg Apr 2, 2025

	const lastExported = exportedPromises.slice(-1)[0];
	const lastExported = exportedPromises.at(-1);

	// Drop our interest in the intermediary promises without gaining an interest in their resolution
	// Drop our interest in the intermediary promises without gaining an interest
	// in their settlement

	// Drop our interest in the last promise of the chain, dropping the refcount onto its resolution
	// Drop our interest in the last promise of the chain without gaining an
	// interest in its settlement

-  // prettier-ignore
-  const allVats =
-    harden(['v1', 'v6', 'v7', 'v8', 'v5', 'v2', 'v4', 'v9', 'v10', 'v11']);
+  const { activeVats } = c.getStatus();
+  t.log(activeVats.map(({ id, options }) => `${id}: ${options.name}`));
+  // As noted in kernel.js, the comms vat is not reapable.
+  const reapable = activeVats.flatMap(({ id, options }) =>
+    options.name === 'comms' ? [] : [id],
+  );
+  // We care about the dynamic vats.
+  t.true(reapable.length >= 7); // bootstrap, staticDumbo{1,2,3}, dyn{1,2,3}
+  const [dyn1, dyn2, dyn3] = reapable.slice(-3);

	for (const vat of ['v11', 'v10', 'v9', 'v1']) {
	for (const vat of [dyn3, dyn2, dyn1, c.vatNameToID('bootstrap')]) {

SwingSet tools for load testing contracts #11158

Are you sure you want to change the base?

SwingSet tools for load testing contracts #11158

Conversation

mhofman commented Mar 25, 2025

Description

Security Considerations

Scaling Considerations

Documentation Considerations

Testing Considerations

Upgrade Considerations

dckc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dckc commented Mar 27, 2025

cloudflare-workers-and-pages bot commented Apr 1, 2025 • edited Loading

Deploying agoric-sdk with Cloudflare Pages

gibson042 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cloudflare-workers-and-pages bot commented Apr 1, 2025 •

edited

Loading