Fix markdown math brackets render problem (#31420)

[GiteaBot]

Backport #31420 by @charles7668

close #31371

compare

original text:

$i = \sqrt{-1}$ ($i = \sqrt{-1}$)

$i = \sqrt{-1}$ [$i = \sqrt{-1}$]

$i = \sqrt{-1}$ {$i = \sqrt{-1}$}

$abcd

(this is a test)

[$test]

$test$

$a$ ($b$) [$c$] {$d$}

before fix:

after fix: