[New Feature] Detect any repeated word

[Abirdcfly]

I expected an automatic detection of duplicate words and not a list of words.

source

1. The detected units should be words, and duplicate symbols should not be displayed, e.g. %s %s is not considered a duplicate.
2. Should Test in real code.

[Abirdcfly]

The result of running on the istio project shows that if all duplicate words are detected, you need to manually determine whether the duplicate here is a bug or not.

$ cd istio.io/istio
$ git fetch upstream
$ git reset --hard upstream/master
HEAD 现在位于 1949d4c269 Quick fix to get builds working on a Mac (#40912)
$ dupword ./...
/Users/xxx/go/src/istio.io/istio/cni/pkg/repair/repair_test_helpers.go:86:14: Duplicate words (blah) found
/Users/xxx/go/src/istio.io/istio/pkg/config/validation/validation.go:3773:27: Duplicate words (set) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/model/addressmap.go:116:3: Duplicate words (if) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/model/context.go:500:2: Duplicate words (of) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/model/telemetry.go:172:1: Duplicate words (namespace,) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/networking/core/v1alpha3/gateway.go:1028:2: Duplicate words (we) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/xds/debug.go:431:1: Duplicate words (is) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/xds/nds.go:26:1: Duplicate words (istiod) found
/Users/xxx/go/src/istio.io/istio/pkg/istio-agent/xds_proxy.go:299:1: Duplicate words (API) found
/Users/xxx/go/src/istio.io/istio/security/pkg/server/ca/authenticate/kubeauth/kube_jwt.go:124:2: Duplicate words (in) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/model/test/mockopenidserver.go:90:2: Duplicate words (error,,success,) found
/Users/xxx/go/src/istio.io/istio/pkg/test/framework/components/environment/kube/flags.go:202:27: Duplicate words (mapping) found
/Users/xxx/go/src/istio.io/istio/cni/pkg/log/uds_test.go:70:3: Duplicate words (debug) found
/Users/xxx/go/src/istio.io/istio/cni/pkg/repair/repair_test.go:50:16: Duplicate words (blah) found
/Users/xxx/go/src/istio.io/istio/istioctl/cmd/istiodconfig_test.go:30:37: Duplicate words (ads) found
/Users/xxx/go/src/istio.io/istio/operator/cmd/mesh/profile-diff_test.go:54:32: Duplicate words (default) found
/Users/xxx/go/src/istio.io/istio/operator/cmd/mesh/profile-diff_test.go:58:32: Duplicate words (demo) found
/Users/xxx/go/src/istio.io/istio/operator/cmd/mesh/profile-diff_test.go:62:32: Duplicate words (openshift) found
/Users/xxx/go/src/istio.io/istio/operator/pkg/util/util_test.go:116:10: Duplicate words (err3) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/networking/core/v1alpha3/cluster_test.go:931:70: Duplicate words (outbound) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/networking/core/v1alpha3/cluster_test.go:1194:2: Duplicate words (failover) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/networking/core/v1alpha3/cluster_test.go:2795:2: Duplicate words (if) found
/Users/xxx/go/src/istio.io/istio/pilot/pkg/networking/core/v1alpha3/listener_test.go:1107:17: Duplicate words (found) found
/Users/xxx/go/src/istio.io/istio/pkg/config/validation/validation_test.go:7778:5: Duplicate words (this) found
/Users/xxx/go/src/istio.io/istio/pkg/config/validation/validation_test.go:7657:4: Duplicate words (set) found
/Users/xxx/go/src/istio.io/istio/security/pkg/nodeagent/sds/sdsservice_test.go:213:3: Duplicate words (get) found

IMO, 14/26 duplicate words is correct, 12/26 duplicate words should be fixed

1. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/cni/pkg/repair/repair_test_helpers.go#L86 duplicate words is correct
2. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pkg/config/validation/validation.go#L3773
3. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/model/addressmap.go#L116
4. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/model/context.go#L500
5. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/model/telemetry.go#L172 duplicate words is correct
6. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/networking/core/v1alpha3/gateway.go#L1028
7. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/xds/debug.go#L431
8. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/xds/nds.go#L26 duplicate words is correct
9. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pkg/istio-agent/xds_proxy.go#L299
10. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/security/pkg/server/ca/authenticate/kubeauth/kube_jwt.go#L124
11. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/model/test/mockopenidserver.go#L90 duplicate words is correct
12. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pkg/test/framework/components/environment/kube/flags.go#L202
13. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/cni/pkg/log/uds_test.go#L70 duplicate words is correct
14. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/cni/pkg/repair/repair_test.go#L50 duplicate words is correct
15. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/istioctl/cmd/istiodconfig_test.go#L30 duplicate words is correct
16. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/operator/cmd/mesh/profile-diff_test.go#L54 duplicate words is correct
17. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/operator/cmd/mesh/profile-diff_test.go#L58 duplicate words is correct
18. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/operator/cmd/mesh/profile-diff_test.go#L62 duplicate words is correct
19. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/operator/pkg/util/util_test.go#L116 duplicate words is correct
20. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/networking/core/v1alpha3/cluster_test.go#L931 duplicate words is correct
21. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/networking/core/v1alpha3/cluster_test.go#L1194 duplicate words is correct （but add a . is better)
22. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/networking/core/v1alpha3/cluster_test.go#L2795
23. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pilot/pkg/networking/core/v1alpha3/listener_test.go#L1107
24. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pkg/config/validation/validation_test.go#L7778 duplicate words is correct （but add a . is better)
25. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/pkg/config/validation/validation_test.go#L7657
26. https://github.com/istio/istio/blob/1949d4c2697d3bb634b3587502fcd60091fb0248/security/pkg/nodeagent/sds/sdsservice_test.go#L213