Load awq models in CPU ? #2326

Closed

Closed

Load awq models in CPU ?#2326

is it possible to load the awq qunatized models into CPU memory instead of GPU memory ?
similar issue : #999

Metadata

Assignees

No one assigned

Labels

No labels

No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests