Skip to content

Commit 67e9e42

Browse files
cccclaifacebook-github-bot
authored andcommitted
fix qnn export (#9808)
Summary: Pull Request resolved: #9808 One missing item for the new tokenizer lib Differential Revision: D72263224
1 parent b66c319 commit 67e9e42

File tree

1 file changed

+1
-1
lines changed
  • examples/qualcomm/oss_scripts/llama

1 file changed

+1
-1
lines changed

examples/qualcomm/oss_scripts/llama/llama.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,6 @@
5757
from executorch.examples.models.llama.source_transformation.quantize import (
5858
get_quant_embedding_transform,
5959
)
60-
from executorch.examples.models.llama.tokenizer.tiktoken import Tokenizer as Tiktoken
6160
from executorch.examples.qualcomm.oss_scripts.llama.model.static_llama import (
6261
LlamaModel,
6362
ModelArgs,
@@ -77,6 +76,7 @@
7776
from executorch.extension.llm.export.builder import DType
7877
from pytorch_tokenizers import get_tokenizer
7978
from pytorch_tokenizers.llama2c import Llama2cTokenizer as SentencePieceTokenizer
79+
from pytorch_tokenizers import TiktokenTokenizer as Tiktoken
8080

8181
from torch.ao.quantization.observer import MinMaxObserver
8282
from torch.ao.quantization.quantize_pt2e import convert_pt2e, prepare_pt2e

0 commit comments

Comments
 (0)