Skip to content

Commit d7fe311

Browse files
authored
Fix idefics3 vision embeddings indices dtype (#40360)
fix idefics3 vision embeddings Signed-off-by: Isotr0py <mozf@mail2.sysu.edu.cn>
1 parent cf487cd commit d7fe311

3 files changed

Lines changed: 6 additions & 6 deletions

File tree

‎src/transformers/models/idefics2/modeling_idefics2.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -152,8 +152,8 @@ def forward(self, pixel_values: torch.FloatTensor, patch_attention_mask: torch.B
152152
nb_patches_h = p_attn_mask[:, 0].sum()
153153
nb_patches_w = p_attn_mask[0].sum()
154154

155-
h_indices = torch.arange(nb_patches_h, device=position_ids.device, dtype=position_ids.dtype)
156-
w_indices = torch.arange(nb_patches_w, device=position_ids.device, dtype=position_ids.dtype)
155+
h_indices = torch.arange(nb_patches_h, device=position_ids.device, dtype=pixel_values.dtype)
156+
w_indices = torch.arange(nb_patches_w, device=position_ids.device, dtype=pixel_values.dtype)
157157

158158
fractional_coords_h = h_indices / nb_patches_h * (1 - 1e-6)
159159
fractional_coords_w = w_indices / nb_patches_w * (1 - 1e-6)

‎src/transformers/models/idefics3/modeling_idefics3.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -151,8 +151,8 @@ def forward(self, pixel_values: torch.FloatTensor, patch_attention_mask: torch.B
151151
nb_patches_h = p_attn_mask[:, 0].sum()
152152
nb_patches_w = p_attn_mask[0].sum()
153153

154-
h_indices = torch.arange(nb_patches_h, device=position_ids.device, dtype=position_ids.dtype)
155-
w_indices = torch.arange(nb_patches_w, device=position_ids.device, dtype=position_ids.dtype)
154+
h_indices = torch.arange(nb_patches_h, device=position_ids.device, dtype=pixel_values.dtype)
155+
w_indices = torch.arange(nb_patches_w, device=position_ids.device, dtype=pixel_values.dtype)
156156

157157
fractional_coords_h = h_indices / nb_patches_h * (1 - 1e-6)
158158
fractional_coords_w = w_indices / nb_patches_w * (1 - 1e-6)

‎src/transformers/models/smolvlm/modeling_smolvlm.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -146,8 +146,8 @@ def forward(self, pixel_values: torch.FloatTensor, patch_attention_mask: torch.B
146146
nb_patches_h = p_attn_mask[:, 0].sum()
147147
nb_patches_w = p_attn_mask[0].sum()
148148

149-
h_indices = torch.arange(nb_patches_h, device=position_ids.device, dtype=position_ids.dtype)
150-
w_indices = torch.arange(nb_patches_w, device=position_ids.device, dtype=position_ids.dtype)
149+
h_indices = torch.arange(nb_patches_h, device=position_ids.device, dtype=pixel_values.dtype)
150+
w_indices = torch.arange(nb_patches_w, device=position_ids.device, dtype=pixel_values.dtype)
151151

152152
fractional_coords_h = h_indices / nb_patches_h * (1 - 1e-6)
153153
fractional_coords_w = w_indices / nb_patches_w * (1 - 1e-6)

0 commit comments

Comments
 (0)