Refactor logs operation

[pav-kv]

LogOperation and its co-types are poorly designed:

• They are meant to be stateless, but in reality there are caches in various places like SequencerManager (see the list below). We should rather make an explicit per-tree cache that can be extended (e.g. with compact ranges cached between sequencing runs as in sequencer: Cache compact.Tree between sequencing runs #1598).
• Interfaces and method signatures are redundant. For example, log.NewSequencer takes a Signer created from a Tree, but then the same Tree is passed into sequencer.IntegrateBatch when in fact there is only one Tree that can be accepted.
• log_operation_manager.go is meant to be agnostic of sequencing, but it contains a bunch of sequencing-related metrics. It is likely that LogOperation abstraction is YAGNI.
• TODO: Keep listing changes.

The list of things we cache:

• Log names (in OperationManager).
• Signers (in SequencerManager).
• Masterships (in OperationManager).
• Compact ranges (not yet).

[pav-kv]

The current design has 3 types (OperationManager, SequencerManager and Sequencer), which all have their own caches for individual tree IDs.

I propose to shift "caching" one level up the stack, so that OperationManager merely does couple of things: tracks log IDs and the corresponding masterships, and for each active log ID has an Operation object. Operation object encapsulates all "cached" items that were previously scattered across different types (log names, signers, compact ranges, etc), and contains the code for handling only one log (effectively some blend of former SequencerManager and Sequencer, but with fixed Tree/treeID).

[pav-kv]

@Martin2112 @AlCutter WDYT?

[Martin2112]

Yes the structure could be improved. If you're going to cache tree related things then it must be done in a way that's completely safe. You might want to make that a separate work item.

[pav-kv]

Yep. I will probably start from just restructuring the code with equivalent safety guarantees. For example, consider SequencerManager and its Signers cache: it is never cleared, and its items are never updated. Same with OperationManager and log names.

Once it's restructured, we could improve guarantees. E.g. if an Operation fails, we can delete it altogether, so on the next run it will be re-created with fresh signer, log name, etc.