STM32L-Discovery (STM32L152R8T6) can't write firmware to chip with st-flash on Windows 10

[hydroconstructor]

• Programmer/board type: STM32L-Discovery, STLINK /V2-onboard
• Operating system and version: Windows 10 x64
• stlink tools version: 1.7.0, current develop branch
• stlink commandline tool name: st-flash
• Target chip and board: STM32L152R8T6, board STM32L-Discovery

Commandline output:

mmap() size_t overflow for file mb.bin

Expected/description:

Debugging revealed that problem is in file common.c, function map_file, in code below:

  if (sizeof(st.st_size) != sizeof(size_t)) {
    // on 32 bit systems, check if there is an overflow
    if (st.st_size > (off_t)SSIZE_MAX) {
      fprintf(stderr, "mmap() size_t overflow for file %s\n", path);
      goto on_error;
    }
  }

Type of st.st_size is off_t, which means long int. Value is size of firmware file and for me is 602 bytes.
SSIZE_MAX defines somewhere in libc headers. It means maximum size of signed integer. But in my system it compiled in long long int. Not 4 bytes long int as off_t, but 8 bytes long long int! So there is overflow in casting types without warnings during compile. As a result, instead of:
if (602 > 2^32-1) {
I got:
if (602 > -1) {
And then function returned with error message.

Proposed solution, which works on my computer:
In header section of common.c define macro OFF_T_MAX such a way:

#ifndef OFF_T_MAX
#define OFF_T_MAX 1073741824 // long int max value
#endif

In map_file function replace SSIZE_MAX by newly defined OFF_T_MAX.

Also please note that SSIZE_MAX defined in unistd.h in such a way:

#define ssize_t int
#ifndef SSIZE_MAX
#define SSIZE_MAX ((sizeof(ssize_t) == 4) ? 1073741824 : ‭4611686018427387904‬)
#endif

There is an error in numbers. SSIZE_MAX is max. value of signed int and must be equal 0x400... . But here it equals 0x7FFF ... in both cases, which is true for UNsigned int. For signed int both values are -1. But in my system SSIZE_MAX defined in libc, so this macro is not used.

[Nightwalker-87]

To me it makes absolutely no sense to define ssize_t as type int because as per design it should never turn to negative values. Thus please correct the type to uint first before further testing. Afterwards we should have another look the specific problem. It appears better to proceed in this way to rule out unintended side effects. Please also take note of #909 which is related to this topic. As it appears several variables appear not to make use of correct types.

[hydroconstructor]

Never turn to negative? Sorry, but can't agree. Many functions return -1 in case of errors. For example, send_recv from #909 mentioned by you. Refactor them all is too difficult - at least for me, these are my first steps in programming.
So if you do not mind I prepare next commit with changes in common.c described above. File unistd.h with #define ssize_t int should be refactored with all other files.

[Ant-ON]

@hydroconstructor I think this check

  if (sizeof(st.st_size) != sizeof(size_t)) {
    // on 32 bit systems, check if there is an overflow
    if (st.st_size > (off_t)SSIZE_MAX) {
      fprintf(stderr, "mmap() size_t overflow for file %s\n", path);
      goto on_error;
    }
  }

may be replaced by

  if (st.st_size > (1<<20) /*1GB*/) {
    // limit file size to 1GB
    fprintf(stderr, "mmap() file %s too big\n", path);
    goto on_error;
  }

The sizes of the MCU firmware will not go beyond this size for a long time, while we will eliminate the wrong choice of files.

[hydroconstructor]

Good idea. Doing this just now...

[Nightwalker-87]

@hydroconstructor One needs to distinguish between functions and variables. I was talking about types for variables.
A function may return an error, but not a variable itself which is figuratively only a "number".

[hydroconstructor]

@Nightwalker-87, functions have types. Type of send_recv from #909 is ssize_t, and it returns -1 in case of error. There are other functions with type ssize_t, returning -1 in case of error. If I make global change from #define ssize_t int to #define ssize_t uint for all project code, I should check each such function and change it's return value if necessary, right?

[Ant-ON]

@Nightwalker-87 Here the problem is in the specific library libc, which has a non-standard type st_size. An error occurs due to the different dimension and the С types conversion.