IronPython shows weak memory behaviors not visible in CPython

[luisggpina]

Hi,

We're a research group focused on testing concurrent runtimes. Our work-in-progress prototype found a behavior that happens in IronPython but not in CPython 3.14 (with or without the GIL) when using concurrent reads and writes on shared variables by different threads. The main idea is to have two threads T1 and T2 and two shared variables x and y such that:

#T1
write(x,1)
read(y)

#T2
write(y,1)
read(x)

There is no scheduling that explains both threads reading 0. However, we found this behavior is possible on IronPython on x86 and ARM64 architectures with the following program (please allow 20 minutes at least to observe):

from threading import Thread, Barrier, Lock
​
iters = 10000000
​
lock = Lock()
done = False
​
x = 0
y = 0
rx = -1
ry = -1
​
# write x, read y
def t0(b1, b2):
    global x
    global y
    global rx
    global ry
​
    while True:
        b1.wait()
​
        # TEST BEGIN
        x = 1
        tmp = y
        # TEST END
​
        with lock:
            ry = tmp
​
            if done:
                return
​
        b2.wait()
​
# write y, read x
def t1(b1, b2):
    global x
    global y
    global rx
    global ry
​
    while True:
        b1.wait()
​
        # TEST BEGIN
        y = 1
        tmp = x
        # TEST END
​
        with lock:
            rx = tmp
​
            if done:
                return
​
        b2.wait()
​
# 3, to synchronize t0, t1, and main thread
b1 = Barrier(3)
b2 = Barrier(3)
​
tt1 = Thread(target=t0, args=(b1, b2))
tt2 = Thread(target=t1, args=(b1, b2))
​
tt1.start()
tt2.start()
​
ret = 0
​
# repeat test "iters" amount of times
for i in range(iters):
    b1.wait()
    b2.wait()
​
    # by program order, at least one of rx or ry should hold the value "1"
    if ry == 0 and rx == 0:
        # weak behavior found, end program and clean up
        print(f"WEAK BEHAVIOR DETECTED ON ITERATION {i}")
        with lock:
            done = True
        b1.wait()
        tt1.join()
        tt2.join()
        ret = 1
​
        exit(ret)
​
​
    # reset values
    x = 0
    y = 0
    rx = -1
    ry = -1
​
print("Done")
print(f"Did not observe any weak behaviors")
​
with lock:
    done = True
​
b1.wait()
​
tt1.join()
tt2.join()
​
exit(ret)

We also observed the following test failing in x86 and ARM64 architectures, where IronPython allows the result y==2 && tmp==0, which cannot be explained by any interleaving of the operations:

#T1
write(x,1)
write(y, 1)

#T2
write(y,2)
tmp = read(x)

We also observed the following test failing in ARM64 architectures, where IronPython allows Thread T2 to observe the result y==1 && x==0, which cannot be explained by any interleaving of the operations:

#T1
write(x,1)
write(y,1)

#T2
read(y)
read(x)

All the behaviors described above are impossible in CPython 2.X, and CPython 3.X (with and without the GIL) in all architectures.

@mqbal is part of the team, adding them so they get notified about further discussion.