ManiSkill v3.0.0b19 Released!

[StoneT2000]

Overview

The latest version brings a number of optimizations around CPU parallelization as well as improved imitation learning baselines/documentation

Moreover we are nearing a more full/official release (v3.0.0 finally?). The main missing components that we are still working on is VR based teleoperation for many robots, low-cost reproducible sim2real manipulation, as well mac OS support. All 3 are still being worked on and cleaned up and coming soon!

Highlights

• CPU simulation and rendering is much faster than before, making most IL baselines run faster. Previously evaluation time due to slow rendering caused all vision based IL baselines to take forever to run.
• New flexible FrameStack wrapper available which handles flat/dict observations and supports both CPU and GPU sim backends. Example usage is in diffusion policy make_env.py code
• Diffusion policy with RGB inputs is now added with working baselines. Most results uploaded to wandb already
• Fix bug where for visual obs modes the final image is not cloned and is equal to the auto reset image
• Cool new DrawTriangle-v1 task and DrawSVG-v1 tasks. DrawTriangle is mostly solvable with diffusion policy.

Example of eval video of DP solving DrawTriangle

183.mp4

Breaking Changes

• By default we try to use the sapien_cuda rendering backend. If using the CPU sim this backend will now result in image data being returned as cuda tensors by default due to using a more optimized rendering backend. If a different backend is selected we will return data as cpu tensors. We recommend you to always run a .to(device) call on image data returned by ManiSkill to avoid device mismatches.

Full release notes here: https://github.com/haosulab/ManiSkill/releases/tag/v3.0.0b19