Skip to content

[AQT-bug] AffineQuantizedTensor raises error with .cuda() #698

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gau-nernst opened this issue Aug 17, 2024 · 1 comment · Fixed by #726
Closed

[AQT-bug] AffineQuantizedTensor raises error with .cuda() #698

gau-nernst opened this issue Aug 17, 2024 · 1 comment · Fixed by #726
Assignees

Comments

@gau-nernst
Copy link
Collaborator

gau-nernst commented Aug 17, 2024

import torch
from torchao.quantization.quant_api import quantize_, int8_weight_only

model = torch.nn.Linear(2048, 2048)
quantize_(model, int8_weight_only())
# model.to("cuda")  # this works
model.cuda()  # this doesn't
  File "/home/xx/code/ao/torchao/dtypes/affine_quantized_tensor.py", line 975, in _
    args[0].to(*args[1:], **kwargs)._apply_fn_to_data(torch.clone),
  File "/home/xx/code/ao/torchao/dtypes/affine_quantized_tensor.py", line 291, in to
    kwargs = self._get_to_kwargs(*args, **kwargs)
  File "/home/xx/code/ao/torchao/dtypes/affine_quantized_tensor.py", line 277, in _get_to_kwargs
    device, dtype, _, memory_format = torch._C._nn._parse_to(*args, **kwargs)
TypeError: to() received an invalid combination of arguments - got (device=torch.device, layout=torch.layout, dtype=torch.dtype, ), but expected one of:
 * (torch.device device = None, torch.dtype dtype = None, bool non_blocking = False, bool copy = False, *, torch.memory_format memory_format = None)
 * (torch.dtype dtype, bool non_blocking = False, bool copy = False, *, torch.memory_format memory_format = None)
 * (Tensor tensor, bool non_blocking = False, bool copy = False, *, torch.memory_format memory_format = None)

I faced similar problems with other subclasses that I implemented before. Basically need to remove the layout kwarg.

cc: @jerryzh168

@jerryzh168
Copy link
Contributor

thanks, yeah I think we want to add some basic tests to https://github.com/pytorch/ao/blob/main/test/dtypes/test_affine_quantized.py

@jerryzh168 jerryzh168 self-assigned this Aug 17, 2024
jerryzh168 added a commit to jerryzh168/ao that referenced this issue Aug 22, 2024
Summary:
Fixes: pytorch#698

Also added `TorchAOBaseTensor` addressing part of pytorch#710

Test Plan:
python test/dtypes/test_affine_quantized.py

Reviewers:

Subscribers:

Tasks:

Tags:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants