Switch to less aggressive whitespace removal for tokens

[pokey]

Instead of removing whitespace if tokens are not merged, instead the token that is before or after the removed token can indicate whether it wants to remove whitespace.

The following tokens will remove whitespace if the token before them is deleted:

• closing delimiters
• comma
• semicolon?

The following tokens will remove whitespace if the token after them is deleted:

• closing delimiters

Note that these preferences are checked on the token that remains, not the token that was deleted. We will first check if we have trailing whitespace, then ask following token if it wants to remove whitespace. If it does not, or if there is no trailing whitespace, then try to remove leading whitespace, asking leading token if it wants that

Note that this impl requires us to factor out the code in surrounding pairs that determines whether something is an opening or closing delimiter

Examples

Input:

foo = bar.baz
"foo-bar bongo-bazman"
hello there,
"foo bar" baz-bongo
foo = bar + baz;

Desired output:

foo = baz
"foo-bar"
hello,
"foo bar" bongo
foo = bar;