Understand try-finally code generation and reduce duplication of finally blocks

[iritkatriel]

We stumbled upon code generation behaviour that we need to understand better. Finally block can be duplicated 4 times for one try.

see here.

[iritkatriel]

This is interesting:

def jumpy():
    try:
        1 / 0
    except ZeroDivisionError:
        print("Here we go, here we go, here we go...")
    else:
        print("There was no exception")
        if jumpy.__name__ == 'Fred':
            print("Never reach this")
    finally:
        print("OK, now we're done")

import dis
dis.dis(jumpy)

This generates 4 copies of the finally block. But if we comment out two lines in the else block, we get only 3:

def jumpy():
    try:
        1 / 0
    except ZeroDivisionError:
        print("Here we go, here we go, here we go...")
    else:
        print("There was no exception")
#        if jumpy.__name__ == 'Fred':
#            print("Never reach this")
    finally:
        print("OK, now we're done")

import dis
dis.dis(jumpy)

[iritkatriel]

The relevant part of the output (for the 4-finally case) is this:

  8          50 LOAD_GLOBAL              2 (jumpy)
             52 LOAD_ATTR                3 (__name__)
             54 LOAD_CONST               5 ('Fred')
             56 COMPARE_OP               2 (==)
             58 POP_JUMP_IF_FALSE       41 (to 82)

  9          60 LOAD_GLOBAL              1 (print)
             62 LOAD_CONST               6 ('Never reach this')
             64 CALL_NO_KW               1
             66 POP_TOP
             68 NOP

 11          70 LOAD_GLOBAL              1 (print)
             72 LOAD_CONST               7 ("OK, now we're done")
             74 CALL_NO_KW               1
             76 POP_TOP
             78 LOAD_CONST               0 (None)
             80 RETURN_VALUE

  8     >>   82 NOP

 11          84 LOAD_GLOBAL              1 (print)
             86 LOAD_CONST               7 ("OK, now we're done")
             88 CALL_NO_KW               1
             90 POP_TOP
             92 LOAD_CONST               0 (None)
             94 RETURN_VALUE

  5     >>   96 NOP

You can see that there is a copy of finally for each of the exits from the else block.

[iritkatriel]

And if I change the order of the prints in the else block, which makes the else-block have just one exit, we get 3 finally blocks again:

def jumpy():
    try:
        1 / 0
    except ZeroDivisionError:
        print("Here we go, here we go, here we go...")
    else:
        if jumpy.__name__ == 'Fred':
            print("Never reach this")
        print("There was no exception")
    finally:
        print("OK, now we're done")

import dis
dis.dis(jumpy)

[iritkatriel]

We can get this from try-finally alone:

3 copies of finally:

def jumpy():
    try:
        print("There was no exception")
        if jumpy.__name__ == 'Fred':
            print("Never reach this")
    finally:
        print("OK, now we're done")

import dis
dis.dis(jumpy)

2 copies of finally:

def jumpy():
    try:
        if jumpy.__name__ == 'Fred':
            print("Never reach this")
        print("There was no exception")
    finally:
        print("OK, now we're done")

import dis
dis.dis(jumpy)

[iritkatriel]

I’m closing this because we understand it now.

[iritkatriel]

I'm reopening this because I see now that one copy of the finally block is due to a bug that we can fix.

Input:

def f():
   try:
     x = "try"
   except:
     x = "except"
   finally:
     x = "finally"
     return x

import dis
dis.dis(f)

Output:

  1           0 RESUME                   0

  2           2 NOP

  3           4 LOAD_CONST               1 ('try')
              6 STORE_FAST               0 (x)
              8 JUMP_FORWARD             9 (to 28)
        >>   10 PUSH_EXC_INFO

  4          12 POP_TOP

  5          14 LOAD_CONST               2 ('except')
             16 STORE_FAST               0 (x)
             18 POP_EXCEPT
             20 JUMP_FORWARD             8 (to 38)
        >>   22 COPY                     3
             24 POP_EXCEPT
             26 RERAISE                  1

  3     >>   28 NOP

  7          30 LOAD_CONST               3 ('finally')
             32 STORE_FAST               0 (x)

  8          34 LOAD_FAST                0 (x)
             36 RETURN_VALUE

  5     >>   38 NOP

  7          40 LOAD_CONST               3 ('finally')
             42 STORE_FAST               0 (x)

  8          44 LOAD_FAST                0 (x)
             46 RETURN_VALUE
        >>   48 PUSH_EXC_INFO

  7          50 LOAD_CONST               3 ('finally')
             52 STORE_FAST               0 (x)

  8          54 LOAD_FAST                0 (x)
             56 SWAP                     2
             58 POP_TOP
             60 SWAP                     2
             62 POP_EXCEPT
             64 RETURN_VALUE
        >>   66 COPY                     3
             68 POP_EXCEPT
             70 RERAISE                  1
ExceptionTable:
  4 to 6 -> 10 [0]
  8 to 8 -> 48 [0]
  10 to 16 -> 22 [1] lasti
  18 to 26 -> 48 [0]
  48 to 60 -> 66 [1] lasti

Lines 28-36 are exactly the same as lines 38-46. The reason they got duplicated is because of duplicate_exits_without_lineno, which makes copies of exit blocks that don't have a line number, and can be reached from more than one place (because line numbers are static).

However, the check is "is it an exit block and does the first instruction have a lineno". In our case, the first instruction is a virtual POP_BLOCK (which eventually becomes a NOP). The virtual instruction doesn't have a line number, but the following instructions (the finally body) do. Note that what we really care about is whether the last instruction has a line number. So if I make this change, the block is no longer copied:

 static inline int
 is_exit_without_lineno(basicblock *b) {
-    return b->b_exit && b->b_instr[0].i_lineno < 0;
+    if (!b->b_exit) {
+        return 0;
+    }
+    for (int i = 0; i < b->b_iused; i++) {
+        if (b->b_instr[i].i_lineno >= 0) {
+            return 0;
+        }
+    }
+    return 1;
 }

(I'm not sure it's enough to only check the last instruction - there is some propagation of line numbers within basicblocks).

Now the output is:

  1           0 RESUME                   0

  2           2 NOP

  3           4 LOAD_CONST               1 ('try')
              6 STORE_FAST               0 (x)
              8 JUMP_FORWARD             9 (to 28)
        >>   10 PUSH_EXC_INFO

  4          12 POP_TOP

  5          14 LOAD_CONST               2 ('except')
             16 STORE_FAST               0 (x)
             18 POP_EXCEPT
             20 JUMP_FORWARD             3 (to 28)
        >>   22 COPY                     3
             24 POP_EXCEPT
             26 RERAISE                  1

  7     >>   28 LOAD_CONST               3 ('finally')
             30 STORE_FAST               0 (x)

  8          32 LOAD_FAST                0 (x)
             34 RETURN_VALUE
        >>   36 PUSH_EXC_INFO

  7          38 LOAD_CONST               3 ('finally')
             40 STORE_FAST               0 (x)

  8          42 LOAD_FAST                0 (x)
             44 SWAP                     2
             46 POP_TOP
             48 SWAP                     2
             50 POP_EXCEPT
             52 RETURN_VALUE
        >>   54 COPY                     3
             56 POP_EXCEPT
             58 RERAISE                  1
ExceptionTable:
  4 to 6 -> 10 [0]
  8 to 8 -> 36 [0]
  10 to 16 -> 22 [1] lasti
  18 to 26 -> 36 [0]
  36 to 48 -> 54 [1] lasti

[iritkatriel]

PR for that is here.