Weights for loading loras

[pdoane]

Is your feature request related to a problem? Please describe.
LoRAs when loaded/applied to a pipeline typically include a user provided weight that scales the weights in the LoRA. E.g. see Compel's LoraWeight class.

Describe the solution you'd like

Add a parameter to load_lora_weights that scales the weights in the file.

Describe alternatives you've considered
I initially assumed network_alpha added in #3437 was for this purpose, but it looks like it must match what is in the file or there is an error. I'm not following how this is supposed to be used.

Maybe repurpose network_alpha for this?

Additional context
Add any other context or screenshots about the feature request here.

[freespirit]

It sounds like you need the scale parameter of the LoRAAttnProcessor.__call__ method.
You can provide it during inference and according to the docs

A scale value of 0 is the same as not using your LoRA weights and you’re only using the base model weights, and a scale value of 1 means you’re only using the fully finetuned LoRA weights. Values between 0 and 1 interpolates between the two weights.

Example (from the same place in the documentation):

image = pipe(
    "A pokemon with blue eyes.", num_inference_steps=25, guidance_scale=7.5, cross_attention_kwargs={"scale": 0.5}
).images[0]

[pdoane]

Sounds good - works for one LoRA model which I think is all that is supported at this point.