AttributeError: 'tuple' object has no attribute 'update'

[YananLi18]

I have uninstalled the official peft package and used the version you provided.
Unfortunately, I encountered the following problem:

Traceback (most recent call last):
File "finetune.py", line 350, in
fire.Fire(train)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/fire/core.py", line 143, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/fire/core.py", line 477, in _Fire
component, remaining_args = _CallAndUpdateTrace(
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/fire/core.py", line 693, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
File "finetune.py", line 317, in train
trainer.train(resume_from_checkpoint=resume_from_checkpoint)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/trainer.py", line 1624, in train
return inner_training_loop(
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/trainer.py", line 1961, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/trainer.py", line 2902, in training_step
loss = self.compute_loss(model, inputs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/trainer.py", line 2925, in compute_loss
outputs = model(**inputs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/accelerate/utils/operations.py", line 822, in forward
return model_forward(*args, **kwargs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/accelerate/utils/operations.py", line 810, in call
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
return func(*args, **kwargs)
File "/data/yananli/lora_transfer/LLM-Adapters/peft/src/peft/peft_model.py", line 568, in forward
return self.base_model(input_ids=input_ids, past_key_values=past_key_values, **kwargs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1176, in forward
outputs = self.model(
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1019, in forward
layer_outputs = decoder_layer(
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 740, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/yananli/.conda/envs/llm/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 367, in forward
key_states, value_states = past_key_value.update(key_states, value_states, self.layer_idx, cache_kwargs)
AttributeError: 'tuple' object has no attribute 'update'

It seems that the past_key_value returned by self.get_prompt(batch_size) of PeftModelForCausalLM is a tuple but a dictionary is needed. Can you give me any advice on how to fix this on your specific peft? or The torch and transformers versions you recommended?

My current environment versions are:
pytorch 1.13.1
pytorch-cuda 11.7
pytorch-mutex 1.0
torch 2.2.1
torchaudio 0.13.1
torchvision 0.14.1
transformers 4.38.2

Thank you for your attention and looking forward to your reply

[HZQ950419]

Hi,
Can I ask what kind of adapter are you using and the command used to run finetune.py?

[YananLi18]

Hi, Can I ask what kind of adapter are you using and the command used to run finetune.py?

I want to use prefix tuning and the command used to run finetune.py is:
CUDA_VISIBLE_DEVICES=4 python finetune.py --base_model '../llama-7b-hf/' --data_path './ft-training_set/math_10k.json' --output_dir './trained_models/llama-7B-math-prefix' --batch_size 16 --micro_batch_size 4 --num_epochs 3 --learning_rate 3e-4 --cutoff_len 256 --val_set_size 120 --adapter_name prefix-tuning --eval_step 10 --save_step 10 --num_virtual_tokens 10

[HZQ950419]

For prefix-tuning, I can run the command with transformers==4.35. The later versions will cause different errors, for example huggingface/peft#1252. Please try with transformers==4.35.

[YananLi18]

For prefix-tuning, I can run the command with transformers==4.35. The later versions will cause different errors, for example huggingface/peft#1252. Please try with transformers==4.35.

Thanks for your quick reply. It works well with transformers==4.35 installed.

By the way, the specific peft module you provided is built on top with which official version?

[HZQ950419]

The LLM-Adapters framework is built on top of peft==0.3.0.