`CommandLineToArgvW` has different behavior than `SegmentCommandLine`

[carlreinke]

#70608 replaced use of SegmentCommandLine with a call to CommandLineToArgvW, but the algorithms are slightly different.

Given the input test.exe a"b"" c d (example taken from here):

Method	[1]	[2]	[3]
SegmentCommandLine	ab" c d
CommandLineToArgvW	ab"	c	d

The comment on SegmentCommandLine claimed that it "mimics [...] CommandLineToArgvW", but it actually mimicked what MSVCRT does before main, which is slightly different.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

---

#70608 replaced use of SegmentCommandLine with a call to CommandLineToArgvW, but the algorithms are slightly different.

Given the input test.exe a"b"" c d (example taken from here):

Method	[1]	[2]	[3]
SegmentCommandLine	ab" c d
CommandLineToArgvW	ab"	c	d

The comment on SegmentCommandLine claimed that it "mimics [...] CommandLineToArgvW", but it actually mimicked what MSVCRT does before main, which is slightly different.

Author:	carlreinke
Assignees:	-
Labels:	area-System.Runtime, untriaged
Milestone:	-

[dakersnar]

cc @huoyaoyuan @jkotas, is this an intended breaking change from the referenced PR?

[huoyaoyuan]

SegmentCommandLine was only used in rare cases changed by that PR. (A delegate created by unmanaged host acquires the command line).

[jkotas]

@carlreinke How important is fixing this corner difference for you?

If it is important, we can switch to a local copy of the segmentation algorithm from C runtime. It is ~100 lines.

[carlreinke]

I happened to notice the difference when I was trying to figure out what segmentation algorithm was being used in the normal case (i.e. running an exe). I assumed both cases would use the same algorithm, so I started using CommandLineToArgvW too... and eventually found that it produced different results in some cases.

So, on the one hand, it's not very important, in that I have never used the delegate-created-by-an-unmanaged-host feature. On the other hand, the use of CommandLineToArgvW led me down the wrong path, and I wasted an afternoon trying to figure out what I did wrong, so restoring the original algorithm might save someone else that pain.

[jkotas]

@huoyaoyuan Are you interested in submitting a fix for this? (The implementation should be in C#.)

[huoyaoyuan]

I'm interested, but I don't have corresponding dev environment in my current machine. If it's not emergent, I can implement later this month.

I'm also curious about what's the really "expected" result. How about languages other than C, like Rust?

[ChrisDenton]

Rust attempts to be consistent with C. See sys/windows/args.rs.