Speed up token filtering

[DmitryOlshansky]

I told that we are not going to see more easy speed-ups but turns out I was wrong :)

The token filtering loop in popFront was a performance sucking pump as it did the work that is already done inside lexXYZ. Thus I short-circuited the relevant paths insde of advance based on flags and gained another 4-5% of speed over the pull #32.

I suggest to rename IterationStyle to TokenFilter, and all simillar names likewise to mention filter. Iteration style impies that you are iterating the same items in e.g. different order. What happens instead is that some tokens are pre-filtered and never came out of TokenRange.

P.S. I constantly mess your indention. I'm using 4 spaces as does the phobos and druntime. I've seen some tabs in the source but dunno if that's a fault on my or your side of things.

dmitry@dmitry-VirtualBox ~/Dscanner $ avgtime -q -r 400 ./dscanner-ldc --tokenCount phobos/std/datetime.d

---

Total time (ms): 9262.23
Repetitions : 400
Sample mode : 23 (181 ocurrences)
Median time : 23.073
Avg time : 23.1556
Std dev. : 0.96712
Minimum : 21.571
Maximum : 32.811
95% conf.int. : [21.2601, 25.0511] e = 1.89552
99% conf.int. : [20.6644, 25.6467] e = 2.49114
EstimatedAvg95%: [23.0608, 23.2503] e = 0.0947761
EstimatedAvg99%: [23.031, 23.2801] e = 0.124557

dmitry@dmitry-VirtualBox ~/Dscanner $ avgtime -q -r 400 ./pull32-dscanner --tokenCount phobos/std/datetime.d

---

Total time (ms): 9670.87
Repetitions : 400
Sample mode : 23 (181 ocurrences)
Median time : 24.0055
Avg time : 24.1772
Std dev. : 1.24027
Minimum : 22.407
Maximum : 36.968
95% conf.int. : [21.7463, 26.6081] e = 2.43088
99% conf.int. : [20.9825, 27.3719] e = 3.19472
EstimatedAvg95%: [24.0556, 24.2987] e = 0.121544
EstimatedAvg99%: [24.0174, 24.3369] e = 0.159736

[Hackerpilot]

The tabs are likely from my editor because I use "1 tab = 1 level of indent" in my code and sometimes forget to change the setting when editing the lexer.