Release 2.2 breaks enviroment prefix

[dleifeld]

This .envs file:

TESTING=TRUE

CONNECTION_ADDRESS="example.com"
CONNECTION_PORT=443

worked in Pydantic Settings 2.1 with this python code without problems:

from pydantic_settings import BaseSettings, SettingsConfigDict

class ConnectionSettings(BaseSettings):
    model_config = SettingsConfigDict(env_prefix='connection_', env_file='.envs')

    address: str
    port: int


def get_conn_params() -> ConnectionSettings:
    settings = ConnectionSettings()   # pyright: ignore
    print(settings)

    return settings

# if executed as a script
if __name__ == "__main__":
    get_conn_params()

But in Pydantic Settings 2.2 this error is thrown:

pydantic_core._pydantic_core.ValidationError: 1 validation error for ConnectionSettings
testing
  Extra inputs are not permitted [type=extra_forbidden, input_value='TRUE', input_type=str]

In my opinion the setting env_prefix should imply, that inputs without this prefix are ignored.
I am not sure why else a prefix should be used.

[hramezani]

Thanks @dleifeld for reporting this.

Actually, it is not a breaking change. it was a bug that got fixed in 2.2

You can fix your problem by ignoring the extra by adding extra='ignore' to config

[dleifeld]

I didn't know that env_prefix is only an alias, which means CONNECTION_PORT and PORT treated as the same.

In the documentation this behavior is mentioned only at the very end. I find this behavior surprising and would suggest to mention it earlier in the docs.

[hramezani]

Would you like to make a PR and improve the docs?

[benjsec]

This was a breaking change for us too. We have a project with internal components that take different settings. In the code there are different pydantic-settings classes, but we keep all the env vars in a single .env file.
I appreciate the workaround is easy to implement but disagree about it not being a breaking change.

[hramezani]

Thanks @benjsec for your opinion here.

I am sorry that the new release breaks your code.

The change that is introduced in 2.2 was a fix for these two issues: #219 and #220. which I think was a bug.

The bug was there from the beginning. we had two options:

1. Continue with the bug in V2 and release the fix in V3
2. Or, Fix it now and release it in V2.2.

As, there was a way(adding extra='ignore') to fix the codes, I decided to release it in V2.2

BTW, I really appreciate your feedbacks @benjsec and @dleifeld. This will help us in improving the library

[dleifeld]

Thank you for your helpful comments and the work on pydantic!

[djpugh]

@hramezani - I just wanted to say that I do understand the comments above but I'd like to flag two things:

• This was a breaking change for me too (and even if the previous behaviours were a bug given the documentation) - this has led to me debugging something for a while to get to this issue - I'd suggest if we are using semver for pydantic-settings (as we seem to be) then this would be a major release update, as it is a breaking change (even as a bugfix) or if the kind of changes shown here is going to be the release policy for pydantic-settings it would be good to know so as library users we can pin to a minor release).
• I'm not sure the current (and documented) behaviour is actually desired - yes it can be fixed by extra='allow', but this now means that e.g. passing in extras in python init will not throw a ValidationError unlike before. The previous behaviour was more logical to common development patterns - using .env files for all environment variables for a given environment level (dev/test etc.) - could be fixed by an env_file_extra variable or similar

[hramezani]

Sorry @djpugh to hear that the change breaks your code and I am agree that this could be release in V3. but there are some major change that we think about it for V3 and those are not ready for V3. That's why I decided to release the fix soon.

I'm not sure the current (and documented) behaviour is actually desired

As I mentioned before it was a fix for reported issues and I think this is right. I would suggest using one .env file per model.
regarding having env_file_extra config I would like to keep the number of configs less because of maitainace effort. for this special config, I think it makes it more confusing because we already have the extra config.

Thanks @djpugh for your valuable feedback it helps us a lot.

[djpugh]

@hramezani - thanks for the reply, it seems that we should expect changes that may require downstream code changes for minor releases of pydantic-settings. I guess I'll be making sure to pin the minor versions then.

Using one dotenv file per model feels like a big imposition on how people might expect to use dotenv files more widely (at least in my experience with them), and it feels counter-intuitive to how the EnvSettingsSource logic works, I guess I'll implement a workaround for my library's usage of pydantic-settings.