Presently, the compute sled sequencer tasks (`cosmo_seq` and `gimlet_seq`) duplicate a large number of type definitions for ereport messages. This is due to our present practice of defining a task's ereports as one big enum of all the ereports it may send, which is almost always unique to that task. This is unfortunate, as it means that we must duplicate these messages when we would like them to be consistent between multiple tasks. Furthermore, the sequencer tasks currently only emit ereports for a small subset of events that we care about --- mostly V<sub>core</sub> regulator PMBus alerts. There are several other events that should definitely be reported. I had previously attempted to add some new ereports to the sequencer tasks in #2242, but at the time, neither @rmustacc nor I were particularly satisfied with the content and class hierarchy of those ereports, and I was unhappy with the fact that these ereports added additional duplicated code between the two sequencer tasks. This PR is a second attempt to add some of the sequencer ereports I would like, using the new API added in #2397 to define those ereport messages in a shared crate, rather than duplicating them between the two sequencer tasks. In particular, I've done the following: 1. Introduce a new `lib/ereports` crate that contains shared ereport message type definitions, using the `#[ereport(...)]` attribute from #2397 to define the ereports as separate types. 2. Refactor the existing `gimlet_seq` and `cosmo_seq` ereports to move the message types for PMBus alerts and BMR491 mitigation failures to the `ereports` crate, and use them in both sequencer tasks. 3. Also refactor the `psc_seq` task to use the shared definition of `PmbusStatus` registers. I didn't change the actual ereport messages there, as they're pretty different from the CPU sequencer ones. 4. Added tracking of when the sequencer transitioned to the current power state, so that this can be included in some of the ereports, as @rmustacc suggested in [this comment][1]. 5. Add new ereports for the following events: + The CPU asserting its `THERMTRIP_L` pin + The `CPU_PRESENT` pin is not asserted + The CPU asserting its `SMERR_L` pin (on Cosmo, as I'm not sure if the Gimlet sequencer task can notice this) + Failure to sequence due to an unsupported CPU type (`coretype` and `sp3r{x}`/`sp5r{x}` pins) 6. Assorted refactoring of the ereport code in both sequencers I did _not_ add ereports for MAPO or NIC MAPO events, as per a conversation on Matrix with @rmustacc, I think we will want to report MAPOs by adding better monitoring of `POWER_GOOD` status from various power rails. Some of that is described in issues #2372, #2394, and perhaps also #2398. This does _not_ actually close #2142, as that describes sequencing failures due to timeouts waiting for `POWER_GOOD` on various rails, which I will want to implement subsequently. Closes #2242. [1]: #2242 (comment)

ereport: sequencing failure #2142

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions