Allow indentation to work inside parens

[odersky]

Actually indentation was always enabled inside parens, but the rule which indentation
width to use was too simplistic and often required an extra line.

This is now fixed, so both of these work:

def test1 = g(1, x =>
    val y = x * x
    y * y
  )

def test2 = g(1,
  x =>
    val y = x * x
    y * y
  )

[odersky]

This does not work currently because we find idioms like these

f[[X, Y] =>
      X
      |
      Y
 ]

Since we are in an indentation region after =>, a newline is inserted. Previously that was not the case, because the
old algorithm decided that the indentation width of the outer [ region was the indentation width of the X after =>, so
no indentation level was opened up. But now there is one.

I believe the best way out would be to go further, and also allow leading infix operators that appear on their own as only token on a line. We want that anyway since it seems a pity that

condition1
|| condition2

should work but

condition1
||
condition2

doesn't. Let's see how much breaks when we enable this.

[odersky]

Summary so far:

• two generalizations of syntax rules:

  1. Allow indentation within parentheses and brackets without requiring an extra line
  2. Allow infix operators on their own line

• Nothing broke, or rather, a single test broke, which tested specifically the pathological behavior we are now abolishing, i.e.

   1
   +
   2

  in the test meant a statement 1, followed by an expression +2.

[smarter]

Otherwise LGTM, this matches what I had in mind!

[smarter]

If this is trying to match all whitespace characters, then it would be better to use an isSpace method for clarity (we can reuse the existing https://github.com/lampepfl/dotty/blob/ea65338e06142f41f0e68b23250c8e22b0ba8837/compiler/src/dotty/tools/dotc/parsing/xml/Utility.scala#L113-L119), this would also avoid matching non-space characters that precede ' ' in ASCII like \b.

The documentation of this method also needs to be updated: it currently states "it is followed on the same line by at least one ' ' and a token that can start an expression."