Joint 2D and 3D Detection Inference Support

[PaulSudarshan]

Hi, thank you for the impressive work and remarkable results in the 3D open-set object detection domain!

I’ve successfully run 3D detection inference using the provided model (the documentation was very clear and easy to follow). Now, I’m interested in performing joint inference using both the 3D and 2D detection heads, since I noticed that the architecture includes a 2D detection component as well.

Could you please clarify whether this is currently supported in the repository? If not, would minimal modifications be sufficient to enable joint 2D–3D inference, and could you share some guidance or example changes?

Thanks again for your excellent work and for making the code so accessible!

[RoyYang0714]

Thank you for being interested in our work! As shown here, our model already runs joint 2D-3D inference, so you can directly parse the results from the output NamedTuple.

You can also refer to our local demo code here.

[PaulSudarshan]

Thanks for the quick feedback. I’m planning to run this on a custom dataset to generate both 3D and 2D annotations. Do the result tuples preserve the same instance order in 2D and 3D so that I can associate them one-to-one?

Or if not, could you suggest some work around for this?

[RoyYang0714]

The instance order is aligned.