Remove support for quant_llm_linear

[howardzhang-cv]

Stack from ghstack (oldest at bottom):

• -> Remove support for quant_llm_linear #3520

Summary: Deleted fp6_linear.cu and rest of fp6_llm folder
Modified ops.py (torchao/ops.py) and test_ops.py (test/test_ops.py) to remove quant_llm_linear calls

Tasks: Related to issue #3516

Differential Revision: D89908990

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3520

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• B200 runners are down due to network issues

✅ No Failures

As of commit 80d8cd1 with merge base 27c5eb9 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[jerryzh168]

probably have to delete this and related tests etc. as well:

ao/torchao/prototype/quantization/quant_api.py

Line 620 in 7035fb7

class FPXWeightOnlyConfig(AOBaseConfig):

you can search for quant_llm_linear in the code base (https://github.com/search?q=repo%3Apytorch%2Fao%20quant_llm_linear&type=code) and delete all the related code

[howardzhang-cv]

First time working with the torchao repo so not really sure if this is the right way to do it:
I deleted the entire fp6_llm folder, and modified ops.py and test_ops.py to remove calls to quant_llm_linear. Is this what we wanted? Or did we want to just delete fp6_llm, keep the calls to quant_llm_linear, and just raise an error or something?
Also, if we are deleting quant_llm_linear, should I keep the floatx_tensor_core? I might be misunderstanding, but it seems like the point of those functions were just to create the fp6 that could use quant_llm_linear? In any case, there is still a reference to quant_llm_linear in floatx_tensor_core_layout.py and the README in that same folder that I have not removed. Just wanted some confirmation that this is what I'm supposed to be doing before continuing.

[jerryzh168]

@howardzhang-cv I think it might be cleaner if you delete the floatx_tensor_core_layout and the FPXWeightOnlyConfig in a separate PR first, before doing this

[howardzhang-cv]

Since they seemed pretty intertwined (since floatx seems to only use quant_llm_linear), it made a bit more sense to me to add them to this PR as well. I removed all the references/tests to floatx_tensor_core_layout and FPXWeightOnlyConfig. That included quite a bit more code changes and deletions, so please check over to make sure I didn't delete anything important.

There was also the float8 layouts and cutlass_semi_sparse layouts that were in the floatx folder that I kept and did not delete. There are a couple references to floatx in the repo that are actually referring to these I believe? (from_hp_to_floatx for example). Please let me know if this is correct.

[jerryzh168]

looks good, thanks! we can land as long as the CI pass I think

[howardzhang-cv]

@howardzhang-cv has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.