Allowing constants in the expressions of derived variables

[anagainaru]

This PR does the following:

• Sneaking in a bunch of code cleanups and removing one level of abstraction when dealing with derived variables (needed to be able to have constants)
• Bison changes to capture constants (extracted from Liz's PR)
• Allowing constants to be used in expressions (for ADD and MULTIPLY for now)
• Adding testing for constants
• Allowing only functions in the expression (i.e. add) and not operators (i.e. +) to deal with negative numbers

These are supported now:

    EXPR1 = "d = data \n add(d, 5, 1)"
    EXPR2 = "d = data \n multiply(d, 5, 2)"
    EXPR3 = "a = data \n b = moredata \n multiply(a, b, 2)"

Not supported (until we have a better grammar):

    EXPR1 = "a = data \n b = moredata \n a + b"
    EXPR2 = "d = data \n d * 2"

[anagainaru]

This is ready for review, I cleaned the history so it's easier to follow all the changes. The testing passed when the derived variables were on but I turned them off for merging.

[eisenhauer]

Worried that you've got ambiguity here.

How does "X-2" tokenize and parse? Same as "x -2" and "x - 2"? Or do the first two of those come through as VAR NUM rather than VAR OP NUM? Because both of those would be valid with these definitions where the - might be a separate operator or it might be part of the constant.

The C and C++ specs don't associate the '-' with the constant. Instead if you have a negative constant it goes through lexical analysis separately and parses as a unary operator on the positive constant. That approach produces an unambiguous result for all of those inputs. Now, we don't currently have a unary operator in the language, which is probably also an issue, because negation isn't exactly unheard of.

[anagainaru]

For this PR I picked parts of what Liz did to capture numbers. The parser line is:
num -?[0-9]*.?[0-9]+

so I do agree that d-2 could potentially be VAR NUM or VAR OP NUM

When I try "d-2" this is tokenized as VAR OP NUM on my laptop (don't ask me why) which is fine if we supported constants for minus (we don't but should be easy to add if we want to). But the problem you are showing is bigger than that.

Surprisingly X * -2 or -2 * X all give errors during parsing and will not detect NUM.

[anagainaru]

The testing passes when the derived variables are on. Frontier fails because of an unrelated test:

99% tests passed, 1 tests failed out of 762
Total Test time (real) = 144.92 sec
The following tests FAILED:
	109 - Unit.Remote.OpenRead.Serial (Failed)

As far as I can tell this test doesn't have any derived variables.

[anagainaru]

I had to turn derived variables off, @eisenhauer please approve again

[eisenhauer]

At some point are we going to leave these on?

[anagainaru]

It should be done in one of the releases (as soon as we consider that the API will not change anymore), we can discuss if we should turn them on by default in the next release or not.

Frontier is still failing (unrelated to this PR) and I still don't have permission to overwrite this. If you force merge @eisenhauer keep the commits (the history is clean). This way I can check the one that removed the operators when Scott will want them back.

[eisenhauer]

Hmm. Frontier is not in the required set, so if you have write permissions, you should be able to merge yourself (I see the option without the need to override). But I can do it if there is some issue. But you don't want to Squash? Just a simple merge commit?

[anagainaru]

Ah you are right, Frontier is not :) Yes, I didn't want to squash