Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

recreate dynamic vats sequentially, not in parallel #2871

Closed
warner opened this issue Apr 14, 2021 · 0 comments
Closed

recreate dynamic vats sequentially, not in parallel #2871

warner opened this issue Apr 14, 2021 · 0 comments
Assignees
Labels
enhancement New feature or request SwingSet package: SwingSet

Comments

@warner
Copy link
Member

warner commented Apr 14, 2021

What is the Problem Being Solved?

We have three functions in loadVat.js to create a vat:

  • createVatDynamically() is used to create a dynamic vat for the first time. It does not wait for the vat to be created, but instead returns the newly-allocated vatID synchronously, so the vatAdminVat which invoked it (via a device) can learn the vatID to wait for.
  • recreateStaticVat() and recreateDynamicVat() are used during kernel reboot to reload vats created by the earlier instance. recreateStaticVat() returns a Promise that fires when the vat is fully loaded (which includes evaluating the source bundle). recreateDynamicVat() was not returning a Promise, rather it followed the pattern of createVatDynamically() and returned a synchronous vatID. The code in kernel.js which called it was using await, as if it returned a Promise.

As a result, rebooting a kernel with multiple dynamic vats would attempt to reload those vats in parallel. These vat reloads currently each take ~20s (for a dynamic ZCF vat with metering enabled), and are CPU bound, and the engine isn't multithreaded, so they do not benefit from parallelism (this will change when we have each vat in a separate subprocess). Also, the accidental parallelism interferes with attempts to measure the time each reload takes (to guide optimization efforts).

The task is to change recreateDynamicVat() to return the same sort of Promise that recreateStaticVat() does. This will reload dynamic vats sequentially, just like the reload of static vats.

We'll probably re-introduce parallelism again, once we have each vat in a separate subprocess. But when we do that, it should be deliberate, and justified, and limited (re-loading 200 vats at the same time on a 4-core machine isn't a good idea either).

@warner warner added enhancement New feature or request SwingSet package: SwingSet labels Apr 14, 2021
@warner warner self-assigned this Apr 14, 2021
warner added a commit that referenced this issue Apr 14, 2021
We have three functions in `loadVat.js` to create a vat.
`createVatDynamically()`is used to create a dynamic vat for the first time.
It does not wait for the vat to be created, but instead returns the
newly-allocated vatID synchronously, so the vatAdminVat which invoked it (via
a device) can learn the vatID to wait for.

recreateStaticVat() and recreateDynamicVat() are used during kernel reboot to
reload vats created by the earlier instance. recreateStaticVat() returns a
Promise that fires when the vat is fully loaded (which includes evaluating
the source bundle). recreateDynamicVat() was not returning a Promise, rather
it followed the pattern of `createVatDynamically()` and returned a
synchronous vatID. The code in `kernel.js` which called it was using `await`,
as if it returned a Promise.

As a result, rebooting a kernel with multiple dynamic vats would attempt to
reload those vats in parallel. These vat reloads currently each take
~20s (for a dynamic ZCF vat with metering enabled), and are CPU bound, and
the engine isn't multithreaded, so they do not benefit from parallelism (this
will change when we have each vat in a separate subprocess). Also, the
accidental parallelism interferes with attempts to measure the time each
reload takes (to guide optimization efforts).

This changes `recreateDynamicVat` to return the same sort of Promise that
`recreateStaticVat` does.

closes #2871
warner added a commit that referenced this issue Apr 14, 2021
We have three functions in `loadVat.js` to create a vat.
`createVatDynamically()`is used to create a dynamic vat for the first time.
It does not wait for the vat to be created, but instead returns the
newly-allocated vatID synchronously, so the vatAdminVat which invoked it (via
a device) can learn the vatID to wait for.

recreateStaticVat() and recreateDynamicVat() are used during kernel reboot to
reload vats created by the earlier instance. recreateStaticVat() returns a
Promise that fires when the vat is fully loaded (which includes evaluating
the source bundle). recreateDynamicVat() was not returning a Promise, rather
it followed the pattern of `createVatDynamically()` and returned a
synchronous vatID. The code in `kernel.js` which called it was using `await`,
as if it returned a Promise.

As a result, rebooting a kernel with multiple dynamic vats would attempt to
reload those vats in parallel. These vat reloads currently each take
~20s (for a dynamic ZCF vat with metering enabled), and are CPU bound, and
the engine isn't multithreaded, so they do not benefit from parallelism (this
will change when we have each vat in a separate subprocess). Also, the
accidental parallelism interferes with attempts to measure the time each
reload takes (to guide optimization efforts).

This changes `recreateDynamicVat` to return the same sort of Promise that
`recreateStaticVat` does.

closes #2871
warner added a commit that referenced this issue Apr 14, 2021
We have three functions in `loadVat.js` to create a vat.
`createVatDynamically()`is used to create a dynamic vat for the first time.
It does not wait for the vat to be created, but instead returns the
newly-allocated vatID synchronously, so the vatAdminVat which invoked it (via
a device) can learn the vatID to wait for.

recreateStaticVat() and recreateDynamicVat() are used during kernel reboot to
reload vats created by the earlier instance. recreateStaticVat() returns a
Promise that fires when the vat is fully loaded (which includes evaluating
the source bundle). recreateDynamicVat() was not returning a Promise, rather
it followed the pattern of `createVatDynamically()` and returned a
synchronous vatID. The code in `kernel.js` which called it was using `await`,
as if it returned a Promise.

As a result, rebooting a kernel with multiple dynamic vats would attempt to
reload those vats in parallel. These vat reloads currently each take
~20s (for a dynamic ZCF vat with metering enabled), and are CPU bound, and
the engine isn't multithreaded, so they do not benefit from parallelism (this
will change when we have each vat in a separate subprocess). Also, the
accidental parallelism interferes with attempts to measure the time each
reload takes (to guide optimization efforts).

This changes `recreateDynamicVat` to return the same sort of Promise that
`recreateStaticVat` does.

closes #2871
warner added a commit that referenced this issue Apr 14, 2021
We have three functions in `loadVat.js` to create a vat.
`createVatDynamically()`is used to create a dynamic vat for the first time.
It does not wait for the vat to be created, but instead returns the
newly-allocated vatID synchronously, so the vatAdminVat which invoked it (via
a device) can learn the vatID to wait for.

recreateStaticVat() and recreateDynamicVat() are used during kernel reboot to
reload vats created by the earlier instance. recreateStaticVat() returns a
Promise that fires when the vat is fully loaded (which includes evaluating
the source bundle). recreateDynamicVat() was not returning a Promise, rather
it followed the pattern of `createVatDynamically()` and returned a
synchronous vatID. The code in `kernel.js` which called it was using `await`,
as if it returned a Promise.

As a result, rebooting a kernel with multiple dynamic vats would attempt to
reload those vats in parallel. These vat reloads currently each take
~20s (for a dynamic ZCF vat with metering enabled), and are CPU bound, and
the engine isn't multithreaded, so they do not benefit from parallelism (this
will change when we have each vat in a separate subprocess). Also, the
accidental parallelism interferes with attempts to measure the time each
reload takes (to guide optimization efforts).

This changes `recreateDynamicVat` to return the same sort of Promise that
`recreateStaticVat` does.

closes #2871
@warner warner closed this as completed in fe6ab38 Apr 14, 2021
warner added a commit that referenced this issue Apr 16, 2021
We have three functions in `loadVat.js` to create a vat.
`createVatDynamically()`is used to create a dynamic vat for the first time.
It does not wait for the vat to be created, but instead returns the
newly-allocated vatID synchronously, so the vatAdminVat which invoked it (via
a device) can learn the vatID to wait for.

recreateStaticVat() and recreateDynamicVat() are used during kernel reboot to
reload vats created by the earlier instance. recreateStaticVat() returns a
Promise that fires when the vat is fully loaded (which includes evaluating
the source bundle). recreateDynamicVat() was not returning a Promise, rather
it followed the pattern of `createVatDynamically()` and returned a
synchronous vatID. The code in `kernel.js` which called it was using `await`,
as if it returned a Promise.

As a result, rebooting a kernel with multiple dynamic vats would attempt to
reload those vats in parallel. These vat reloads currently each take
~20s (for a dynamic ZCF vat with metering enabled), and are CPU bound, and
the engine isn't multithreaded, so they do not benefit from parallelism (this
will change when we have each vat in a separate subprocess). Also, the
accidental parallelism interferes with attempts to measure the time each
reload takes (to guide optimization efforts).

This changes `recreateDynamicVat` to return the same sort of Promise that
`recreateStaticVat` does.

closes #2871
warner added a commit that referenced this issue Apr 16, 2021
We have three functions in `loadVat.js` to create a vat.
`createVatDynamically()`is used to create a dynamic vat for the first time.
It does not wait for the vat to be created, but instead returns the
newly-allocated vatID synchronously, so the vatAdminVat which invoked it (via
a device) can learn the vatID to wait for.

recreateStaticVat() and recreateDynamicVat() are used during kernel reboot to
reload vats created by the earlier instance. recreateStaticVat() returns a
Promise that fires when the vat is fully loaded (which includes evaluating
the source bundle). recreateDynamicVat() was not returning a Promise, rather
it followed the pattern of `createVatDynamically()` and returned a
synchronous vatID. The code in `kernel.js` which called it was using `await`,
as if it returned a Promise.

As a result, rebooting a kernel with multiple dynamic vats would attempt to
reload those vats in parallel. These vat reloads currently each take
~20s (for a dynamic ZCF vat with metering enabled), and are CPU bound, and
the engine isn't multithreaded, so they do not benefit from parallelism (this
will change when we have each vat in a separate subprocess). Also, the
accidental parallelism interferes with attempts to measure the time each
reload takes (to guide optimization efforts).

This changes `recreateDynamicVat` to return the same sort of Promise that
`recreateStaticVat` does.

closes #2871
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request SwingSet package: SwingSet
Projects
None yet
Development

No branches or pull requests

1 participant