Figure out the exact set of compiler flags used to produce Diablo.exe (1.09b)

[mewmew]

This issue tracks the effort to figure out the complier flags used to produce Diablo.exe (version 1.09b). It may be considered a subtask of #11, as given information about compiler flags, we can ensure that the same input source code produce the same output object code.

From what I can tell, it seems like /O1 is used rather than /O2. This is based on the padding between functions.

/O2 produces (given that #110 has been merged):

  401000:	e9 0b 00 00 00       	jmp    0x401010
  401005:	90                   	nop
  401006:	90                   	nop
  401007:	90                   	nop
  401008:	90                   	nop
  401009:	90                   	nop
  40100a:	90                   	nop
  40100b:	90                   	nop
  40100c:	90                   	nop
  40100d:	90                   	nop
  40100e:	90                   	nop
  40100f:	90                   	nop
  401010:	a1 28 b1 40 00       	mov    eax,ds:0x40b128
  401015:	a3 84 1a 41 00       	mov    ds:0x411a84,eax
  40101a:	c3                   	ret    
  40101b:	90                   	nop
  40101c:	90                   	nop
  40101d:	90                   	nop
  40101e:	90                   	nop
  40101f:	90                   	nop
  401020:	8b 44 24 04          	mov    eax,DWORD PTR [esp+0x4]
  401024:	85 c0                	test   eax,eax
  401026:	74 0f                	je     0x401037
  401028:	6a 00                	push   0x0
  40102a:	6a ff                	push   0xffffffff
  40102c:	68 40 d0 40 00       	push   0x40d040
  401031:	50                   	push   eax
  401032:	e8 69 0c 00 00       	call   0x401ca0
  401037:	c3                   	ret    

While /O1 produces:

  401000:	e9 00 00 00 00       	jmp    0x401005
  401005:	a1 28 c1 40 00       	mov    eax,ds:0x40c128
  40100a:	a3 84 2a 41 00       	mov    ds:0x412a84,eax
  40100f:	c3                   	ret    
  401010:	83 7c 24 04 00       	cmp    DWORD PTR [esp+0x4],0x0
  401015:	74 12                	je     0x401029
  401017:	6a 00                	push   0x0
  401019:	6a ff                	push   0xffffffff
  40101b:	68 40 e0 40 00       	push   0x40e040
  401020:	ff 74 24 10          	push   DWORD PTR [esp+0x10]
  401024:	e8 8f 09 00 00       	call   0x4019b8
  401029:	c3                   	ret    

This is to be compared to the original version of Diablo.exe (1.09b):

  401000:	e9 00 00 00 00       	jmp    0x401005
  401005:	a1 00 94 47 00       	mov    eax,ds:0x479400
  40100a:	a3 30 79 4b 00       	mov    ds:0x4b7930,eax
  40100f:	c3                   	ret    
  401010:	83 7c 24 04 00       	cmp    DWORD PTR [esp+0x4],0x0
  401015:	74 12                	je     0x401029
  401017:	6a 00                	push   0x0
  401019:	6a ff                	push   0xffffffff
  40101b:	68 b0 30 48 00       	push   0x4830b0
  401020:	ff 74 24 10          	push   DWORD PTR [esp+0x10]
  401024:	e8 8d 87 06 00       	call   0x4697b6
  401029:	c3                   	ret    

[heinermann]

It's worth noting that individual compilation units can have different optimization settings (just to keep in mind).

Another comparison with Diablo.exe 1.09b:

• O1: http://www.mergely.com/TCffp23K/
• O2: http://www.mergely.com/VXbxcrZo/

I actually want to highlight in particular the line and byte ptr [ecx], 0. This is actually an assignment, but it was optimized into an and, which is only produced under O1.

Under O1:

• *ptr = 0 becomes and byte ptr [ecx], 0, even though assignment was used
• *ptr &= 0 becomes and byte ptr [ecx], 0

Under O2:

• *ptr = 0 becomes mov byte ptr [eax], 0
• *ptr &= 0 becomes mov byte ptr [eax], 0, even though AND was used

The project now uses /O1 for both debug and retail builds. I figured that's what was used, since the output size is much closer (760 KB). The main reason I never changed it was in the early days optimizations broke the code entirely. It took a few months of fixing up before it was stable enough to change it.

[heinermann]

You may also be able to find out other compiler flags by looking at the binary header (MZ/PE) and seeing what differs. Imports too probably.

Good call, I noticed the original binary has about 2x as many import descriptors. Which is weird, since it there's only a few differences in the KERNEL32 imports. We'll have to look at that sometime. Also, the release build seems to scramble the code a little more, where as the debug build is closer to the original. Setting aside debug sections of course.

It could also be a good idea to "trace" the developers steps. We know the game was originally written in VC 4.20, but then upgraded to 5.10. So perhaps generating the project using 4.20's default settings could get us closer.

[mewmew]

So perhaps generating the project using 4.20's default settings could get us closer

Sounds reasonable. Report back the compiler flags used :) I still don't have a Windows box to play with.

[seritools]

Warning: Big writeup ahead! :)

As @mewmew wrote up over here, more often than not, the compiler we currently use (5.0 SP3) generated slightly different instructions than the original 1.09b binary.

The first thing I checked is the linker versions. For that I grabbed every version ever, basically, and confirmed that the linker used must be from 5.0 SP3:

SP0: 5.00.7022
SP1: 5.02.7132
SP2: 5.02.7132
SP3: 5.10.7303

Diablo 1.09b PE Header: 5.10

While browsing for compiler option info, I stumbled over the #pragma optimize directive, allowing to toggle optimization settings on a per-function basis. Playing around with disabling size-based code and other optimizations, the generated instructions were even further away from the original, so I knew that was probably not what was happening.

For some reason, the compiler used by Blizzard was just more "clever" than the one we use...

After going off-track by checking the #pragma optimize directive, I went back to linker stuff. Just to make sure, I checked VS6's link.exe (starting from SP6, for some reason).

The output was:

C:\Program Files\Microsoft Visual Studio\VC98\Bin>link /?
Microsoft (R) Incremental Linker Version 6.00.8447
Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

usage: LINK [options] [files] [@commandfile]

   options:

      /ALIGN:#
      /BASE:{address|@filename,key}
[...]
      /LINK50COMPAT

Looking at that options list it instantly clicked. See that /LINK50COMPAT? Googling it revealed that link.exe as well as lib.exe both have that flag in version 6. It seems that with VC++ 6, they changed the structure of lib files, so the lib files generated by lib.exe 5.x and lower are not compatible with link.exe 6.x and vice versa. Microsoft implemented that switch so that older library files can be used with VC++ 6, and code compiled by VC++ 6 can generate lib files compatible with the linker of VC++ 5.

So I just fired up VS6SP6, upgraded the project file, compiled, checked in IDA, and voilà:

(left is Diablo.exe 1.09b, right is devilution compiled/linked with VC++ 6 SP6)

The instructions are identical (at least for the function from PR #135)!

So, what I guess happened:

• With the time, Blizzard upgraded to VS/VC++ 6
• Hovever, something prevented them to use the VC++ 6 linker.
• Thus, they compiled newer versions of Diablo with VC++ 6, generated the .lib files from the .obj files via VC++ 6's lib.exe with the /LINK50COMPAT, generating library files compatible with VC++5's linker
• Then, they linked those with the linker of 5.0SP3, leading to the PE header containing 5.10 as linker version, but optimizations only possible with a V6 compiler!
• It seems like they literally only used /O1 :D

The 1.09b .exe has a creation date of 2001-05-12. This means that the newest VC++6 version they could have used it Service Pack 5 (released April 2001). Service Pack 4 (released January 2001) would be a good contender as well.

Anyways, that was my journey today. :)

What's left to do:

• Find out which service pack was used (only needed if we still find significant changes where there shouldn't be any)
• Build a VC++ 6 workspace that does a custom build generating lib files compatible with link.exe v5.10, and call that linker to generate the output file.

In retrospective, all of this makes sense seeing that Storm.dll and the .SNP files are even linked with v6 of the linker! 🙂

[StephenCWills]

That's way cool! Excellent work.

[fearedbliss]

@seritools Wow that's a great read and find, the asm is exactly the same (excluding different memory locations)!

Beautiful, now we just have to figure out how they automated everything 😉 It seems tedious to have used two different toolchains, so maybe there's another option we're missing or they modded their VC6 installation.

Awhile back I documented the versions they used for D2:

Version          Released   Patch
MSVC++ 6.0 SP 3  5/21/1999  1.00-1.03
MSVC++ 6.0 SP 4  6/15/2000  1.04-1.05
MSVC++ 6.0 SP 5  2/26/2001  1.06-1.10
MSVC++ 6.0 SP 6  3/29/2004  N/A
VS .NET 2003     ?/??/2003  1.11+

The linker for both SP 4 and 5 generates code exactly the same, I didn't test SP 6 since it was made long after those patches. We can assume that the D2 devs used their current setup to compile D1 at the time, so 1.09b would've been compiled with SP 5 and 1.08 SP 3.

[mewmew]

@seritools Wow Dennis, that's great! Hats off. Very happy that you managed to pin this down.

Do we get similar sizes with the optimization? I am curious what a bindiff would result in. It might be useless to know, but I think that it would tell us how close we are.