Draft for discussion: reuse of the configuration storage space in RAM

[GioCC]

I'm considering whether that the entire config buffer in RAM might be eventually disposed of, once config transfer operations are over, and reused to accommodate working variables (with a substantial gain in available RAM).

An overview of what happens currently in configuration management may be useful (please correct me if I got anything wrong).

At boot:

1. Working config is reset/cleaned up
2. Config data are read from the EEPROM and copied directly to configBuffer (all in "verbose" form)
3. Data structures are filled from the content of configBuffer.
   Notice that configBuffer is then - with the exception of some user-defined names, where used - no longer required for operation.

If the PC requires the config to be read back:

4. Config is read from the EEPROM and directly transmitted
   (and even though I think this is the preferrable way, I wondered why not read it from configBuffer, since it always remains there)

If a config upload is requested from the PC:

5. Working config is reset/cleaned up (but this must be explicitly requested by the PC)
6. Config data are read from the PC in "verbose" form and stored in configBuffer
7. Config data are copied unaltered to the EEPROM (but remain in configBuffer)
8. New config is activated (see step 3)

According to the above procedures (and with the noted exceptions), the buffer is basically only necessary to contain the received config data before they are saved to EEPROM;
during boot (or after a configuration upload), data are read from the buffer, but they might just as well be read from the EEPROM directly, just as it happens when they must be sent back to the PC.

There seem to be only two obstacles in the way:

• currently, some I/O blocks still need to maintain a link to the user-assigned name because it is required to form the serial messages. This could be easily solved by using the convention proposed in Draft for discussion: Implicit I/O naming #577 for transmitted data.
• reusing the config buffer would preclude the possibility of uploading a "temporary" configuration and activate it without saving it (therefore preserving the previous one).
  I have a question about this point: is this just a possibility, or is it desired feature? Is it ever really exploited?

[elral]

Sorry, I just saw this discussion here, wasn't aware ofit before.

I am not sure if I understood point 4) correct. In the actual FW the config buffer is still required, as it stores all names of the devices. To get the names from the configbuffer all delimiters ('.' ';' ...) are replaced by NULL. So you can't transfer this configbuffer back to the UI. That was the reason to read from the EEPROM directly.

I also think that the configbuffer is not necessary anymore. Not only to save the required RAM but also to use the complete EEPROM for the configuration. To achieve this the configuration must directly written to the EEPROM, the function readConfig() must be modified just to read the pin numbers from the EEPROM and the handlers send only the pin numbers w/o the name.
Within the UI are changes required, that the pin numbers are read and "translated" to the names (read in on startup like existing).

I have a already a fork where I am on the "half" way, writing direct to the EEPROM, reading the config from the EEPROM and just assigning the names to the configBuffer (which could be just deleted in the future).

Using the complete EEPROM size would espacially for the Mega dramatically increase the available size for the config and would enable to name much more inputs/outputs (shifters ;) ). But it can be also seen that there is a limitation for the ProMicro/Uno in the numbers of devices or if a device should be supported.

[GioCC]

HI Elral, thanks for chiming in.
Yes, you probably got it right about point 4. The relevant point is that reading back from the EEPROM is a requirement that's luckily already implemented that way; this just didn't look consistent with the rest of the handling, but I missed the detail about the terminator substitution, which answers my question.
Everything else you say matches my observations perfectly, except you're clearly some steps ahead of me also because you've already started drafting an implementation. I'm glad to find you on the same page in this matter.

[GioCC]

I would particularly stress the point where you say "the handlers send only the pin numbers w/o the name", which is related to discussion #577.

[elral]

Storing the config directly into the EEPROM and processing the config out of the EEPROM is running fine in my branch. Also only the names are stored in the config buffer, the other informations are directly assigned to there addButton()/.... functions and not anymore in the configBuffer.
When the UI is capable of reading "just" the pin's and not the names the configBuffer is not required anymore. This saves a couple of RAM and the configuration can be increased to nearly the complete EEPROM.

Yes, you are completly right. It is also related to the discussion in #577, didn't metioned it explictily.

[elral]

To be more precise, only the names of the three inputs are saved in the configBuffer as only they are needed for now.