Do we have any examples of rendering a .drc model with materials using THREE.js?

[FarhadG]

I've looked and haven't found a working example of rendering a .drc model with materials with THREE.js. Ive used the the mtl-loader and drc-loader to load my model, however, associating the material with the model has been a challenge. For example:

        // Bluebird promise API
        return new Promise.props({
          material: new Promise((res) => {
            _mtlLoader.setPath(path);
            _mtlLoader.load(mtl, (material) => {
              material.preload();
              res(material)
            });
          }),
          model: new Promise((res) => {
            _drcLoader.load(`${path}${drc}`, (geometry) => {
              res(geometry);
            });
          })
        });

When I receive the model (i.e. BufferGeometry) and material (i.e. THREE.MTLLoader.MaterialCreator) which comprises 4 materials, I am not able to render the model with the following:

      const mesh = new THREE.Mesh(model, material);
      this._scene.add(mesh);

[zwcloud]

Maybe: https://github.com/google/draco/blob/master/javascript/example/webgl_loader_draco_advanced.html

[FarhadG]

Thanks for the link, @zwcloud, however, none of these examples (basic or advanced) show how to apply the .mtl textures as I've outlined above. I have a large model that I load via Draco and when I get the materials after using the MTLLoader, I cannot seem to locate how to apply these materials to the model.

[FarhadG]

I've got the following to work where I reference a particular material from the materials, however, it's only a single texture and it's obviously not referenced properly.

          const mesh = new THREE.Mesh(
            geometry,
            material.materials['material0000'] // the name of the first material of the set of 4
          );

So, then, I applied the array of materials with the following:

          const mesh = new THREE.Mesh(
            geometry,
            _.map(material.materials, m => m)
          );

The result is the same. The texture is applied but the mapping is incorrect as it's duplicated and incorrectly referenced throughout the model position.

[ondys]

if the issue is that you have multiple materials, you would have to split the geometry first (as far as I know, three.js allow only one material per geometry). You can see for example this issue for a discussion how to do that. In future we may add a function to the javascript API that would do this automatically, but for now the splitting has to be done manually.

[FarhadG]

Thanks for the quick response, @ondys. THREE.js does provide a way for a Mesh to have multiple materials by providing an array of materials as the argument. That said, it's complicated to split up a model as it's generated by our users (via photogrammetry), so I'm not sure if there's a programatic way for doing this.

I'm really hoping we are able to leverage .drc here, but a large model without the textures would not be useful in our case. I found out about Draco by analyzing Pix4D's model viewer and realized they were leveraging your framework. I'm hoping to achieve something very similar.

Looking forward to your response.

[ondys]

@FarhadG I'm not sure if I understand the problem completely. If THREE.js supports more materials per single object, you should be able to avoid the splitting altogether. Draco encodes material ids that would have to be somehow translated into material ids of THREE.js. I'm not familiar how THREE.js handles mapping between materials and faces but if they support it you should be able to modify DRACOLoader.js to fill whatever array the three js geometry needs to perform this mapping.

[FarhadG]

Thanks for the response, @ondys. THREE does, indeed, support multiple textures.

For example, if you visit this photogrammetry service where you get a large OBJ with multiple textures, you can download the assets and give it a go within the Draco examples of rendering a DRC model.

I tried this in a variety of ways but could not get it to render the textures correctly. Hence, why I've started looking into partitioning the OBJ to the number of textures provided.

Let me know if anything's unclear, @ondys. Thank you!

[ondys]

Since three.js supports multiple materials per object, you just need to map them to the correct faces. As I said, I'm not familiar with the proper way to handle the setup in three.js so I would suggest asking at their forums instead. To get the material ids for faces you can use following code (in case the .obj was encoded with --metadata flag):

var materialAttrId = decoder.GetAttributeIdByName(dracoGeometry, "material");        
var materialAttributeData;
if (materialAttrId != -1) { 
           var materialAttribute = decoder.GetAttribute(dracoGeometry,
                                                       materialAttrId);
        materialAttributeData = new dracoDecoder.DracoInt32Array();
        decoder.GetAttributeInt32ForAllPoints(dracoGeometry,
                                                                     materialAttribute,
                                                                     materialAttributeData);
}

then in the DRACOLoader.js code where the faces, you can update it to store material ids as:

for (var i = 0; i < numFaces; ++i) {
   decoder.GetFaceFromMesh(dracoGeometry, i, ia);
   var index = i * 3;
   geometryBuffer.indices[index] = ia.GetValue(0);
   geometryBuffer.indices[index + 1] = ia.GetValue(1);
   geometryBuffer.indices[index + 2] = ia.GetValue(2);
   materialId[i] = materialAttributeData[ia.GetValue(0)];
}

[JohnnyPosi]

@ondys does --metadata works when I'm encoding .ply files as well?

By the way, when I'm encoding .obj I have x3 larger .drc than when use .ply file.
For example:
model.obj (18,667 KB) -> model.drc (608 KB)
model.obj (18,667 KB) -> model.ply (7,998 KB) -> model.drc (204 KB)

Am I missing some data using .ply? Thanks!

[FarhadG]

Since I've received a few emails on whether I've resolved this, here's some good news for individuals with a similar use case as I've mentioned above.

I opted to leverage the glTF file format. With some recent developments, glTF-pipeline is using Draco for compression. It works like a charm and it has served me better than even using the original .obj files.

Here's the discussion for reference: https://discourse.threejs.org/t/mapping-multiple-materials-to-an-obj-model-in-three-js/2577/14

[FrankGalligan]

@JohnnyPosi Can you post model.obj and model.ply? If the models are the same the compression should not be different.