Pulsar perspective camera bug? Rendering same perspective camera gets different results using PulsarRenderer and PointRenderer

[Rima121212]

Rendering the same Pointclouds instance with the same PerspectiveCameras parameter returns a different results when using PointsRenderer and PulsarRenderer. It seems that the pulsar rendering result is offseted along the x and y axis from the PointsRenderer output. I'm attaching an image below.

The rasterizer used is a bit different, as the code complains otherwise. I'm attaching a sample code below.
Also, browsing at the pulsar code it seems like it might only support orthographic projection at the moment, and doesn't use the given focal length parameter. Is that correct? This could explain the issue but seems to be undocumented.

Instructions To Reproduce the Issue:

#cam - PerspectiveCamera with R,T,focal_length, and principal point provided
cam = PerspectiveCameras(device=device,focal_length=focal_length,
principal_point=principal_point,R=R,T=T,)
#Points
raster_settingsPoints = PointsRasterizationSettings(
image_size=nxy,
radius=0.001,
bin_size=128,
)
rasterizerPoints = PointsRasterizer(cameras=cam,raster_settings=raster_settingsPoints)
rendererPoints = PointsRenderer(
rasterizer=rasterizerPoints,
compositor=NormWeightedCompositor()
)
imagePoints = rendererPoints(point_cloudPoints,
bg_col=torch.ones((3,),dtype=torch.float32,device=device),)[0].cpu()

#Pulsar
vert_rad_init = 0.001*torch.ones(pts_init.shape[0],dtype=torch.float32,device=device)
raster_settingsPulsar = PointsRasterizationSettings(
image_size=nxy,
radius=vert_rad_init,
)
rasterizerPulsar = PointsRasterizer(cameras=cam,raster_settings=raster_settingsPulsar)
renderer = PulsarPointsRenderer(rasterizer=rasterizerPulsar).to(device)
gamma = 0.1
imagePulsar = renderer(
point_cloudPulsar,
gamma=(gamma,),
zfar=(45.0,),
znear=(0.1,),
radius_world=True,
bg_col=torch.zeros((3,),dtype=torch.float32,device=device),
focal_length=cam.focal_length,
principal_point=cam.principal_point,
R=cam.R,
T=cam.T,
)[0]

Thank you in advance for your reply!

[Rima121212]

[classner]

Hi @Rima121212 !

Indeed, there is a confirmed bug in the transformation of camera parameters in the unified interface. We are aware of the problem and have a plan to address this by replacing the current transformation with a step-wise transformation PyTorch3D -> OpenCV -> Pulsar, whereas both parts PyTorch3D->OpenCV (#522, inverse) and OpenCV->Pulsar (#519) are well tested on their own.

[Rima121212]

Thank you for your reply! It will be nice to have this functionality :-)

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.