Update docs.

Author: DavidVonDerau

README.md

@@ -188,10 +188,9 @@ This tool currently is only useful for packing assets.
    * Nearly all assets are data-driven from serializable and hashable structures rather than hard-coded.
    * Buffers and images are asynchronously uploaded on dedicated transfer queue when available
    * Multi-pass material abstraction with bindable parameters
-   * Nodes/Render Jobs system - Inspired by the 2015 GDC talk "Destiny's Multithreaded Rendering Architecture."
+   * Render Features and Jobs system - Inspired by the 2015 GDC talk "Destiny's Multithreaded Rendering Architecture."
       * A job system with extract, prepare, and write phases
-      * Rendering is pipelined with simulation thread, and the job structure is intended to be highly parallel (not
-        actually executed parallel yet)
+      * Rendering is pipelined with simulation thread, and the job structure is parallelizable by the application
       * Handles multiple views and phases allowing advanced features like shadow mapping
       * Flexible sorting mechanism for interleaving and batching write commands from multiple rendering features
    * Visibility Region - Built on top of `rafx-visibility`

docs/framework/adding_features.md

@@ -1,16 +1,19 @@
 # Adding Features
 
-Features represent "things" that can be drawn. For example, the could be separate features for meshes, sprites, cloth,
+Features represent "things" that can be drawn. For example, the could be separate features for meshes, sprites, cloth, 
 debug draw, imgui, etc.
 
 ## Declare the Feature
 
 You may either implement `RenderFeature` or use this macro
 
 ```rust
+use rafx::render_feature_mod_prelude::*;
 rafx::declare_render_feature!(Debug3DRenderFeature, DEBUG_3D_FEATURE_INDEX);
 ```
 
+When using `rafx-renderer`, you should implement `RenderFeaturePlugin`.
+
 ## Register the Feature
 
 ```rust
@@ -20,43 +23,26 @@ let render_registry = rafx::nodes::RenderRegistryBuilder::default()
     .register_feature::<MeshRenderFeature>();
 ```
 
-Features that have been registered are assign a unique index. For example, to get the feature index of 
-`MeshRenderFeature` call `MeshRenderFeature::feature_index()`.
+Features that have been registered are assign a unique index. For example, to get the feature index of `MeshRenderFeature` 
+call `MeshRenderFeature::feature_index()`.
 
-## Implement `RenderNodeSet` and add it to the frame packet
+If using a `RenderFeaturePlugin`, the call to `register_feature` should go into `configure_render_registry`. This will be called
+after the `RenderFeaturePlugin` is registered with the `Renderer`. A `RenderFeaturePlugin` can be registered with the `Renderer`
+by calling `add_render_feature` on the `RendererBuilder`.
 
-See the demo for examples implementing `RenderNodeSet`. (Maybe there should be a macro to provide a default impl).
+## Define the Frame Packet and Submit Packet
 
-Render nodes will be updated during the extract phase and will "belong" to the render thread until the next extract
-phase. 
+Each `RenderFeature` defines two data structures of packed arrays -- the `RenderFeatureFramePacket` and the `RenderFeatureSubmitPacket`. 
+- `RenderFeatureFramePacket` contains the data extracted from the game world.
+- `RenderFeatureSubmitPacket` contains data prepared for the GPU and a list of sortable submit nodes for the `RenderFeatureWriteJob`.
 
-```rust
-// Create a frame packet builder. It needs to know about all the render nodes and views.
-// (Setting views not shown here)
-let frame_packet_builder = {
-    let mut sprite_render_nodes = resources.get_mut::<SpriteRenderNodeSet>().unwrap();
-    sprite_render_nodes.update();
-    let mut mesh_render_nodes = resources.get_mut::<MeshRenderNodeSet>().unwrap();
-    mesh_render_nodes.update();
-    let mut all_render_nodes = AllRenderNodes::default();
-    all_render_nodes.add_render_nodes(&*sprite_render_nodes);
-    all_render_nodes.add_render_nodes(&*mesh_render_nodes);
-
-    FramePacketBuilder::new(&all_render_nodes)
-};
-```
+See the demo for examples of features implementing the `FramePacket` and `SubmitPacket`.
 
-## Implement `ExtractJob` and add it to an extract jobs set
+## Implement the `RenderFeatureExtractJob`, `RenderFeaturePrepareJob`, and `RenderFeatureWriteJob`
 
-See the demo for examples implementing `ExtractJob`.
+- `RenderFeatureExtractJob` queries data from the game world and copies it into the `RenderFeatureFramePacket`. 
+- `RenderFeaturePrepareJob` processes extracted data into GPU-friendly data and creates the list of sortable submit nodes in the 
+`RenderFeatureSubmitPacket`.
+- `RenderFeatureWriteJob` defines the GPU commands for rendering each submit node.
 
-```rust
-// Create extract jobs for features we will render
-let mut extract_job_set = ExtractJobSet::new();
-extract_job_set.add_job(create_sprite_extract_job());
-extract_job_set.add_job(create_sprite_extract_job());
-
-// Kick off the extract
-let frame_packet = frame_packet_builder.build();
-extract_job_set.extract(&extract_context, &frame_packet, &extract_views)
-```
+See the demo for examples of features implementing the `RenderFeature` jobs.

docs/framework/adding_render_phases.md

@@ -5,7 +5,7 @@ could be render phases for shadow maps, standard 3d drawing, or drawing 2d UI on
 
 Phases combine the draw calls from multiple features into a single pass by sorting them according to a function.
 
-## Declare the Feature
+## Declare the Render Phase
 
 You may either implement `RenderPhase` or use this macro to reduce boilerplate code.
 
@@ -21,19 +21,17 @@ rafx::declare_render_phase!(
     shadow_map_render_phase_sort_submit_nodes
 );
 
-fn shadow_map_render_phase_sort_submit_nodes(mut submit_nodes: Vec<SubmitNode>) -> Vec<SubmitNode> {
+fn shadow_map_render_phase_sort_submit_nodes(submit_nodes: &mut Vec<SubmitNode>) {
     // Sort by feature
     log::trace!(
         "Sort phase {}",
         ShadowMapRenderPhase::render_phase_debug_name()
     );
     submit_nodes.sort_unstable_by(|a, b| a.feature_index().cmp(&b.feature_index()));
-
-    submit_nodes
 }
 ```
 
-## Register the Feature
+## Register the Render Phase
 
 ```rust
 let render_registry = rafx::nodes::RenderRegistryBuilder::default()

docs/framework/framework_architecture.md

@@ -1,4 +1,4 @@
-# Rendering Architecture
+# Framework Architecture
 
 `rafx-framework` provides an architecture similar to what is describe in the 2015 GDC talk 
 "[Destiny's Multithreaded Rendering Architecture](http://advances.realtimerendering.com/destiny/gdc_2015/Tatarchuk_GDC_2015__Destiny_Renderer_web.pdf)".
@@ -28,8 +28,8 @@ Features must be registered.
 Features need to submit draw calls at the correct time when rendering a scene. This is usually associated with a
 particular render pass.
 
-For example, a mesh may need to be drawn BOTH for creating shadows and for the main view. A feature may emit a "render 
-node" for a particular mesh in both phases.
+For example, a mesh may need to be drawn BOTH for creating shadows and for the main view. A feature may emit a "submit node" 
+for a particular mesh in both phases.
 
 Phases define a sorting order. This allows front-to-back, back-to-front, or batched-by-feature ordering of draw
 calls within the phase.
@@ -43,50 +43,56 @@ When rendering a scene, it's often necessary to draw the scene from different vi
 For example, when creating shadow maps, we must render a depth buffer from the viewpoint of all light sources that cast
 shadows. In this case, we would create a view for each shadow-casting light that will run just the shadow mapping phase.
 
-## Rendering a Frame
+## Render Resources
+
+The framework provides a "resource" table (similar to ECS resources, but simplified) that allows storing shared data
+between the draw scheduling logic and extract/prepare jobs. In general, it is better to pass/share data "normally"
+with plain rust code. However, there are cases where the flexibility is useful either for code modularity or when rust's
+borrow checking cannot verify that your code is safe.
+
+# Rendering a Frame
 
 Each frame will go through these steps:
 
-### Simulation
+## Simulation
 
 Process all game logic as normal. Game logic may be stored in whatever way you like. (ECS of your choice, or no ECS at 
 all).
 
-### Extract Job
+## Extract Jobs
 
-Copy all data from game state necessary to render the scene. Extract jobs implement the `ExtractJob` trait.
+Copy all data from game state necessary to render the scene. Extract jobs implement the `RenderFeatureExtractJob` trait.
 
 **Game state may not change during this time. This will likely block simulating the next frame until it completes.**
 However, extract jobs may run concurrently. (A similar pattern could be followed for other systems like audio.)
 
 Generally only **dynamic** data needs to be copied. Static data is safely shareable between threads 
 (via `Arc<T>` or other mechanisms).
 
-Extract jobs produce prepare jobs that can run on separate threads from the simulation.
+After the data has been extracted, the prepare and write jobs can be run on separate threads from the simulation.
 
-### Prepare Job
+## Prepare Jobs
 
-Process all the data collected during the extract job and produce **render nodes**. Prepare jobs implement the
-`PrepareJob` trait.
+Process all the data collected during the extract job and produce `SubmitNode`s. Prepare jobs implement the
+`RenderFeaturePrepareJob` trait.
 
-Render nodes are like a handle for something that can be rendered later. These handles/nodes are added to different 
-views/phases.
+`SubmitNode`s are like a handle for something that can be rendered later. The `SubmitNode`s are associated with a particular
+`RenderView` and `RenderPhase`.
 
 This job might make holistic decision for the feature, like choosing the 10 most important pieces of cloth to render at
 high LOD. Prepare jobs for different features may run concurrently.
 
-### Sorting
+## Submit Node Sorting
 
-The render nodes emitted by all prepare jobs are sorted within their respective phases. When sort order does not matter,
-they can be sorted by render feature to reduce pipeline changes. When sort order does matter (like when working with
-transparency), features can be sorted as needed, most likely by depth.
+The `SubmitNode`s emitted by all prepare jobs are sorted by the `RenderPhase` for each `RenderView`. When sort order does 
+not matter, they can be sorted by render feature to reduce pipeline changes. When sort order does matter (like when working 
+with transparency), features can be sorted as needed, most likely by depth.
 
-### Writing to Command Buffers  
+## Write Jobs
 
-Prepare jobs produce struct that implements `FeatureCommandWriter`. The framework will call functions on the writer
-to record draw calls for render nodes that feature emitted.
+Record the draw calls needed for each `SubmitNode` into command buffers. Write jobs implement the `RenderFeatureWriteJob` trait.
 
-## Draw Scheduling
+# Draw Scheduling
 
 When the frame is renderered, we must set up render passes and run the phases intended for that pass. In the destiny GDC
 talk, this was called a "script". It sounds like the manually handled this process. It's certainly not a bad way to go
@@ -104,11 +110,4 @@ graph_callbacks.set_renderpass_callback(node, move |args, user_context| {
         .prepared_render_data
         .write_view_phase::<OpaqueRenderPhase>(&main_view, &mut write_context)
 });
-```
-
-## Render Resources
-
-The framework provides a "resource" table (similar to ECS resources, but simplified) that allows storing shared data
-between the draw scheduling logic and extract/prepare jobs. In general, it is better to pass/share data "normally"
-with plain rust code. However, there are cases where the flexibility is useful either for code modularity or when rust's
-borrow checking cannot verify that your code is safe.
+```

docs/framework/visibility_region.md

@@ -2,34 +2,52 @@
 
 ![Overview](../images/visibility_region.png)
 
-A `VisibilityRegion` is a ref-counted wrapper around 2 `Zones`.  One of the zones is designated "static" and the other designated "dynamic". When registering objects with the region, the application may pick if it is "static" or "dynamic" -- this will assign it to the corresponding `Zone` and return a `VisibilityObjectArc`. Note that the "static" or "dynamic" assignment is not a meaningful distinction by the underlying `VisibilityWorld` -- it is ok to move an object that was registered as "static". The reason for separating "static" and "dynamic" assignments is to support a future capability for running the "static" visibility calculation earlier in the frame and combining it with the "dynamic" visibility later in the frame. [1] An application view is registered with the `VisibilityRegion` and returned as a `ViewFrustumArc`.
-
-The `VisibilityObjectArc` is a ref-counted wrapper around an `ObjectHandle`. This struct contains functions for setting the position, `id`, and other fields. The set functions are implemented under the hood using async commands over a channel to the `VisibilityWorld`. Each `VisibilityObjectArc` contains a list of features registered with that handle. Particular features may be shown or hidden on an entity in the world by adding or removing the feature from the `VisibilityObjectArc` associated with that entiy.
-
-The `ViewFrustumArc` is a ref-counted wrapper around 1 or 2 `ViewFrustumHandle` representing a view of the "static" `Zone` and a view of the "dynamic" `Zone` in the `VisibilityRegion`. This struct contains functions for setting the location, `id`, projection, and querying for visibility. The set functions are implemented under the hood using async commands over a channel to the `VisibilityWorld`. Each `RenderView` requires a `ViewFrustumArc` so that visibility can be calculated for that view.
-
-`EntityId` is a helper to transmute between a struct `T: Copy` with the same as a `u64` and the `u64` required for the `id` in the `VisibilityWorld`. 
+A `VisibilityRegion` is a ref-counted wrapper around 2 `Zones`.  One of the zones is designated "static" and the other 
+designated "dynamic". When registering objects with the region, the application may pick if it is "static" or "dynamic" 
+-- this will assign it to the corresponding `Zone` and return a `VisibilityObjectArc`. Note that the "static" or "dynamic" 
+assignment is not a meaningful distinction by the underlying `VisibilityWorld` -- it is ok to move an object that was 
+registered as "static". The reason for separating "static" and "dynamic" assignments is to support a future capability 
+for running the "static" visibility calculation earlier in the frame and combining it with the "dynamic" visibility later 
+in the frame. [1] An application view is registered with the `VisibilityRegion` and returned as a `ViewFrustumArc`.
+
+The `VisibilityObjectArc` is a ref-counted wrapper around an `ObjectHandle`. This struct contains functions for setting 
+the position, `id`, and other fields. The set functions are implemented under the hood using async commands over a channel 
+to the `VisibilityWorld`. Each `VisibilityObjectArc` contains a list of features registered with that handle. Particular 
+features may be shown or hidden on an entity in the world by adding or removing the feature from the `VisibilityObjectArc` 
+associated with that entiy.
+
+The `ViewFrustumArc` is a ref-counted wrapper around 1 or 2 `ViewFrustumHandle` representing a view of the "static" `Zone` 
+and a view of the "dynamic" `Zone` in the `VisibilityRegion`. This struct contains functions for setting the location, 
+`id`, projection, and querying for visibility. The set functions are implemented under the hood using async commands over 
+a channel to the `VisibilityWorld`. Each `RenderView` requires a `ViewFrustumArc` so that visibility can be calculated 
+for that view.
+
+`ObjectId` is a helper to transmute between a struct `T: 'static + Copy + Hash + Eq + PartialEq` with the same size as a 
+`u64` and the `u64` required for the `id` in the `VisibilityWorld`. 
 
 ```rust
-let entity = world.push((transform_component, mesh_component));
+let entity = world.push((transform_component.clone(), mesh_component));
 let mut entry = world.entry(entity).unwrap();
 entry.add_component(VisibilityComponent {
-    handle: {
+    visibility_object_handle: {
         let handle = visibility_region.register_static_object(
-            EntityId::from(entity),
-            load_visible_bounds(&floor_mesh_asset),
+            ObjectId::from(entity),
+            CullModel::VisibleBounds(load_visible_bounds(&floor_mesh_asset)),
         );
         handle.set_transform(
             transform_component.translation,
             transform_component.rotation,
             transform_component.scale,
         );
-        handle.add_feature(floor_mesh.as_raw_generic_handle());
+        handle.add_render_object(&floor_mesh_render_object);
         handle
     },
 });
 ```
 
-When a `RenderView` is aded to the frame packet using `add_view`, the associated `ViewFrustum` is queried for visibility. Only visible features relevant to the `RenderView` will be added to the frame packet.
+When a `RenderView` is needed in the current frame, the associated `ViewFrustum` is queried for visibility. The visible 
+`ObjectHandle`s are mapped to their `VisibilityObjectArc`s and the associated `RenderObject`s are added to the `FramePacket` 
+of the relevant `RenderFeature` -- if the `RenderView` is registered for the `RenderFeature` and any `RenderPhase` required 
+by it. 
 
 [1] http://advances.realtimerendering.com/destiny/gdc_2015/Tatarchuk_GDC_2015__Destiny_Renderer_web.pdf

docs/images/extract_prepare_write.png

@@ -31,6 +31,9 @@
 * rafx-assets
     * `distill` Architecture and Features
     * [Asset Triangle Example](../rafx/examples/asset_triangle/asset_triangle.rs)
+* rafx-renderer
+    * [Renderer Architecture](renderer/renderer_architecture.md)
+    * [Renderer Triangle Example](../rafx/examples/renderer_triangle/renderer_triangle.rs)
 * Shader Authoring with `rafx-shader-processor`
     * [Shader Processor](shaders/shader_processor.md)
     * [Custom Shader Markup](shaders/shader_annotation.md)
@@ -60,7 +63,8 @@
 * [Key crate dependencies](images/crate_dependencies.png)
 * [Pipelining](images/pipelining.png)
 * [Shader Processor](images/shader_processor.png)
-* [Visibility region](images/visibility_region.jpg)
+* [Visibility region](images/visibility_region.png)
+* [Extract, prepare, write](images/extract_prepare_write.png)
 
 ## Other Resources