Skip to content

Feature allow unset pointer #80

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 16, 2025

Conversation

rostislaved
Copy link

Description

This pull request adds a config flag AllowUnsetPointer to allow pointer fields in structs to be treated as optional during decoding. When enabled, missing pointer fields will not be reported as unset even if ErrorUnset is true.

This is useful when decoding into structs with optional pointer fields, while still maintaining strict handling (ErrorUnset) for required non-pointer fields.


Current behavior example

type Config struct {
	URL  *string
	Port int
}

input := map[string]any{
	"Port": 8080,
}

var cfg Config

dec, err := mapstructure.NewDecoder(&mapstructure.DecoderConfig{
	ErrorUnset: true,
	Result:     &cfg,
})
if err != nil {
	panic(err)
}

err = dec.Decode(input)
// err is not nil: "has unset fields: URL"

New behavior with AllowUnsetPointer: true

type Config struct {
	URL  *string
	Port int
}

input := map[string]any{
	"Port": 8080,
}

var cfg Config

dec, err := mapstructure.NewDecoder(&mapstructure.DecoderConfig{
	ErrorUnset:        true,
	AllowUnsetPointer: true,
	Result:            &cfg,
})
if err != nil {
	panic(err)
}

err = dec.Decode(input)
// err is nil
// cfg.URL == nil
// cfg.Port == 8080

Comment on lines +1470 to +1472
if !(d.config.AllowUnsetPointer && fieldValue.Kind() == reflect.Ptr) {
targetValKeysUnused[fieldName] = struct{}{}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be more elegant to continue here and leave setting the map after the condition.

WDYT?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi! Did you mean like this?

		if !rawMapVal.IsValid() {
				// There was no matching key in the map for the value in
				// the struct. Remember it for potential errors and metadata.
				if !(d.config.AllowUnsetPointer && fieldValue.Kind() == reflect.Ptr) {
					targetValKeysUnused[fieldName] = struct{}{}

					continue
				}

				continue
			}

If yes, I am fine with this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the correct implementation would be:

		if !rawMapVal.IsValid() {
				if d.config.AllowUnsetPointer && fieldValue.Kind() == reflect.Ptr {
					continue
				}

				// There was no matching key in the map for the value in
				// the struct. Remember it for potential errors and metadata.
				targetValKeysUnused[fieldName] = struct{}{}


				continue
		}

Or did I misread what it does?

Copy link
Author

@rostislaved rostislaved May 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your implementation did the thing. However, if we set AllowUnsetPointer == true and use "continue" we won't add the fieldName to the set and it seems to be formally incorrect, because there is still no "matching key in the map for the value in the struct".

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sagikazarmark What do you think?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sagikazarmark could you please have a look?

@sagikazarmark sagikazarmark merged commit 8c61ec1 into go-viper:main Jun 16, 2025
10 checks passed
project-mirrors-bot-tu bot pushed a commit to project-mirrors/forgejo-runner-as-gitea-act-runner-fork that referenced this pull request Jul 3, 2025
…RITY] (#622)

This PR contains the following updates:

| Package | Change | Age | Confidence |
|---|---|---|---|
| [github.com/go-viper/mapstructure/v2](https://github.com/go-viper/mapstructure) | `v2.2.1` -> `v2.3.0` | [![age](https://developer.mend.io/api/mc/badges/age/go/github.com%2fgo-viper%2fmapstructure%2fv2/v2.3.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/go/github.com%2fgo-viper%2fmapstructure%2fv2/v2.2.1/v2.3.0?slim=true)](https://docs.renovatebot.com/merge-confidence/) |

---

### mapstructure May Leak Sensitive Information in Logs When Processing Malformed Data
[GHSA-fv92-fjc5-jj9h](GHSA-fv92-fjc5-jj9h)

<details>
<summary>More information</summary>

#### Details
##### Summary

Use of this library in a security-critical context may result in leaking sensitive information, if used to process sensitive fields.

##### Details

OpenBao (and presumably HashiCorp Vault) have surfaced error messages from `mapstructure` as follows:

https://github.com/openbao/openbao/blob/98c3a59c040efca724353ca46ca79bd5cdbab920/sdk/framework/field_data.go#L43-L50

```go
			_, _, err := d.getPrimitive(field, schema)
			if err != nil {
				return fmt.Errorf("error converting input for field %q: %w", field, err)
			}
```

where this calls `mapstructure.WeakDecode(...)`: https://github.com/openbao/openbao/blob/98c3a59c040efca724353ca46ca79bd5cdbab920/sdk/framework/field_data.go#L181-L193

```go

func (d *FieldData) getPrimitive(k string, schema *FieldSchema) (interface{}, bool, error) {
	raw, ok := d.Raw[k]
	if !ok {
		return nil, false, nil
	}

	switch t := schema.Type; t {
	case TypeBool:
		var result bool
		if err := mapstructure.WeakDecode(raw, &result); err != nil {
			return nil, false, err
		}
		return result, true, nil
```

Notably, `WeakDecode(...)` eventually calls one of the decode helpers, which surfaces the original value:

https://github.com/go-viper/mapstructure/blob/1a66224d5e54d8757f63bd66339cf764c3292c21/mapstructure.go#L679-L686

https://github.com/go-viper/mapstructure/blob/1a66224d5e54d8757f63bd66339cf764c3292c21/mapstructure.go#L726-L730

https://github.com/go-viper/mapstructure/blob/1a66224d5e54d8757f63bd66339cf764c3292c21/mapstructure.go#L783-L787

& more.

##### PoC

To reproduce with OpenBao:

```
$ podman run -p 8300:8300 openbao/openbao:latest server -dev -dev-root-token-id=root -dev-listen-address=0.0.0.0:8300
```

and in a new tab:

```
$ BAO_TOKEN=root BAO_ADDR=http://localhost:8300 bao auth enable userpass
Success! Enabled userpass auth method at: userpass/
$ curl -X PUT -H "X-Vault-Request: true" -H "X-Vault-Token: root" -d '{"password":{"asdf":"my-sensitive-value"}}' "http://localhost:8300/v1/auth/userpass/users/adsf"
{"errors":["error converting input for field \"password\": '' expected type 'string', got unconvertible type 'map[string]interface {}', value: 'map[asdf:my-sensitive-value]'"]}
```

##### Impact

This is an information disclosure bug with little mitigation. See https://discuss.hashicorp.com/t/hcsec-2025-09-vault-may-expose-sensitive-information-in-error-logs-when-processing-malformed-data-with-the-kv-v2-plugin/74717 for a previous version. That version was fixed, but this is in the second part of that error message (starting at `'' expected a map, got 'string'` -- when the field type is `string` and a `map` is provided, we see the above information leak -- the previous example had a `map` type field with a `string` value provided).

This was rated 4.5 Medium by HashiCorp in the past iteration.

#### Severity
- CVSS Score: 5.3 / 10 (Medium)
- Vector String: `CVSS:3.1/AV:N/AC:H/PR:N/UI:R/S:U/C:H/I:N/A:N`

#### References
- [https://github.com/go-viper/mapstructure/security/advisories/GHSA-fv92-fjc5-jj9h](https://github.com/go-viper/mapstructure/security/advisories/GHSA-fv92-fjc5-jj9h)
- [https://github.com/go-viper/mapstructure](https://github.com/go-viper/mapstructure)

This data is provided by [OSV](https://osv.dev/vulnerability/GHSA-fv92-fjc5-jj9h) and the [GitHub Advisory Database](https://github.com/github/advisory-database) ([CC-BY 4.0](https://github.com/github/advisory-database/blob/main/LICENSE.md)).
</details>

---

### Release Notes

<details>
<summary>go-viper/mapstructure (github.com/go-viper/mapstructure/v2)</summary>

### [`v2.3.0`](https://github.com/go-viper/mapstructure/releases/tag/v2.3.0)

[Compare Source](go-viper/mapstructure@v2.2.1...v2.3.0)

#### What's Changed

- build(deps): bump actions/checkout from 4.1.7 to 4.2.0 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#46
- build(deps): bump golangci/golangci-lint-action from 6.1.0 to 6.1.1 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#47
- \[enhancement] Add check for `reflect.Value` in `ComposeDecodeHookFunc` by [@&#8203;mahadzaryab1](https://github.com/mahadzaryab1) in go-viper/mapstructure#52
- build(deps): bump actions/setup-go from 5.0.2 to 5.1.0 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#51
- build(deps): bump actions/checkout from 4.2.0 to 4.2.2 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#50
- build(deps): bump actions/setup-go from 5.1.0 to 5.2.0 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#55
- build(deps): bump actions/setup-go from 5.2.0 to 5.3.0 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#58
- ci: add Go 1.24 to the test matrix by [@&#8203;sagikazarmark](https://github.com/sagikazarmark) in go-viper/mapstructure#74
- build(deps): bump golangci/golangci-lint-action from 6.1.1 to 6.5.0 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#72
- build(deps): bump golangci/golangci-lint-action from 6.5.0 to 6.5.1 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#76
- build(deps): bump actions/setup-go from 5.3.0 to 5.4.0 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#78
- feat: add decode hook for netip.Prefix by [@&#8203;tklauser](https://github.com/tklauser) in go-viper/mapstructure#85
- Updates by [@&#8203;sagikazarmark](https://github.com/sagikazarmark) in go-viper/mapstructure#86
- build(deps): bump github/codeql-action from 2.13.4 to 3.28.15 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#87
- build(deps): bump actions/setup-go from 5.4.0 to 5.5.0 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#93
- build(deps): bump github/codeql-action from 3.28.15 to 3.28.17 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#92
- build(deps): bump github/codeql-action from 3.28.17 to 3.28.19 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#97
- build(deps): bump ossf/scorecard-action from 2.4.1 to 2.4.2 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#96
- Update README.md by [@&#8203;peczenyj](https://github.com/peczenyj) in go-viper/mapstructure#90
- Add omitzero tag. by [@&#8203;Crystalix007](https://github.com/Crystalix007) in go-viper/mapstructure#98
- Use error structs instead of duplicated strings by [@&#8203;m1k1o](https://github.com/m1k1o) in go-viper/mapstructure#102
- build(deps): bump github/codeql-action from 3.28.19 to 3.29.0 by [@&#8203;dependabot](https://github.com/dependabot) in go-viper/mapstructure#101
- feat: add common error interface by [@&#8203;sagikazarmark](https://github.com/sagikazarmark) in go-viper/mapstructure#105
- update linter by [@&#8203;sagikazarmark](https://github.com/sagikazarmark) in go-viper/mapstructure#106
- Feature allow unset pointer by [@&#8203;rostislaved](https://github.com/rostislaved) in go-viper/mapstructure#80

#### New Contributors

- [@&#8203;tklauser](https://github.com/tklauser) made their first contribution in go-viper/mapstructure#85
- [@&#8203;peczenyj](https://github.com/peczenyj) made their first contribution in go-viper/mapstructure#90
- [@&#8203;Crystalix007](https://github.com/Crystalix007) made their first contribution in go-viper/mapstructure#98
- [@&#8203;rostislaved](https://github.com/rostislaved) made their first contribution in go-viper/mapstructure#80

**Full Changelog**: go-viper/mapstructure@v2.2.1...v2.3.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "" (UTC), Automerge - Between 12:00 AM and 03:59 AM ( * 0-3 * * * ) (UTC).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xLjQiLCJ1cGRhdGVkSW5WZXIiOiI0MS4xLjQiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbXX0=-->

Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/622
Reviewed-by: earl-warren <[email protected]>
Co-authored-by: Renovate Bot <[email protected]>
Co-committed-by: Renovate Bot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants