Hart enumeration algorithm (Sec 3.3) is impractical and conflicts with hart state definitions (Sec 3.4)

[dokunira]

I would like to propose a discussion on the hart enumeration algorithm in Section 3.3 "Selecting Harts". The current mandatory (must) algorithm appears to be impractical and based on a premise that conflicts with other parts of the specification.

The specification currently requires debuggers to stop scanning for harts as soon as they find one that is nonexistent:

"...a debugger must ... selects each hart starting from 0 until either anynonexistent in dmstatus is 1, or the highest index... is reached."

This requirement creates several significant problems:

1. The Algorithm Conflicts with Real-World Practice

This algorithm assumes the debugger has zero knowledge of the target. In reality, debuggers (or their users) almost always know the target's configuration (i.e., NUM_HARTS) beforehand. This information is typically provided via a debugger configuration script (e.g., an OpenOCD .cfg file). A mandatory scan is unnecessary when the hart count is already known.

2. Redundancy with confstrptr

The confstrptr mechanism (Section 3.14.9) is provided for configuration discovery. If this is available, the "scan-until-nonexistent" algorithm is completely redundant.

3. Contradictory Definitions for "Holes" (Section 3.4)

The definitions in Section 3.4 "Hart DM States" are contradictory regarding permanently disabled harts ("holes"):

• nonexistent is defined as "if they will never be part of this hardware platform". This perfectly describes a "hole".
• However, the spec also states that "holes... must show up as unavailable".
• This forces implementations to use the semantically incorrect unavailable status (which implies the hart "might... become available") for something that is truly nonexistent.

4. The Algorithm Prevents Flexible Implementations

Because of the mandatory "Stop-at-Nonexistent" rule, implementations cannot use the more logical nonexistent status for "holes". If an implementation did report a "hole" (e.g., hart 2) as nonexistent, a spec-compliant debugger would incorrectly stop its scan and fail to find existing harts at higher indices (e.g., hart 3).

Proposal:

The specification should be updated to align with real-world use and logical definitions.

1. Allow nonexistent Harts as "Holes": Implementations should be allowed to have existing harts at indices above a nonexistent hart.
2. Change the Enumeration Algorithm (Section 3.3):
   • Remove the "Stop-at-Nonexistent" Rule. The algorithm should not require debuggers to stop at the first nonexistent hart.
   • The primary method for enumeration should be reading confstrptr (Sec 3.14.9) or using user-provided configuration (which is the common case).
   • The full scan (up to the maximum HARTSELLEN) should be only a fallback, and nonexistent should not halt it.
3. Clarify Definitions (Section 3.4):
   • nonexistent should be defined as any hart that is permanently disabled (including "holes").
   • unavailable should be defined only as any hart that might genuinely become available later (e.g., powered down, in reset).

This approach provides a more logical definition of hart states, aligns with realistic debugging practices, and gives implementers the flexibility needed for modern hardware (like binned devices).

[dokunira]

Even if we strictly follow the rule in Section 3.4 (that "holes" must be reported as unavailable, and a nonexistent hart only ever appears at the very end of a contiguous range of valid hart IDs):

The algorithm mandated (as a must) in Section 3.3—"selects each hart starting from 0 until either anynonexistent in dmstatus is 1"—still seems inefficient and unnecessarily restrictive.

If nonexistent truly only marks the end of the hart range, there is a much simpler way to determine the hart count (or the maximum valid hartsel value).

For example, the specification could simply define the behavior of the hartsel register to clamp to the maximum valid hart index (N-1) if a value greater than or equal to the number of harts (N) is written (instead of just clamping to HARTSELLEN).

With this behavior, a debugger could write all ones to hartsel and immediately determine the maximum hart index (NUM_HARTS - 1) in a single access. (And this is ignoring the fact that NUM_HARTS is often known anyway via confstrptr or config files.)

Given that NUM_HARTS can be determined efficiently, even if an iterative scan from 0 is required (e.g., to check the status of each hart), the algorithm does not need to use anynonexistent as a stopping condition. It could simply scan from 0 to the determined NUM_HARTS - 1.

This "scan-until-nonexistent" requirement seems to be the root cause that forces impractical implementations, as it (and Section 3.4) does not allow for nonexistent "holes", which was the main point of this issue.

[en-sc]

@dokunira, thank you for raising your concerns!

Unfortunately, I strongly disagree with your suggestions.

The first point you make is that the hart enumeration algorithm is redundant because debuggers (OpenOCD) still require to specify the number of cores in the config.
The main issue IMHO is not the number of harts but the number of DMs and the hart - DM assignment.
From my experience this is a pain-point and a defect in OpenOCD -- the user should be able to write the configuration file with minimal knowledge of Debug Subsystem internals (e.g. whether the harts share a debug module or not). Please see riscv-collab/riscv-openocd#1147 and riscv-collab/riscv-openocd#278 for more info.

The second suggestion, AFAIU, is to make "holes" appear as nonexistent instead of unavailable.
I don't think this will work particularly well. My understanding of a "hole" is some hart that is disabled in the design due to a manufacturing defect (perhaps this is not the only use-case, but it is a valid one).
So consider a perfect CPU and two CPUs of the same design with different defunct harts (e.g. the desing has 4 harts, on CPU0 all the harts are functional, on CPU1 hart with mhartid == 1 is defunct, on CPU2 hart with mhartid == 2 is defunct). I'd like to reuse the same config for any of these CPUs (after all they are of the same design). Therefore my config will have 4 harts. Moreover, I'd like to be able to transfer a testing scenario between these CPUs, meaning I should be able to take into account the defunct hart. Therefore I need for that defunct hart to have some representation in the debugger. While we could add a defunct state to the debugger that differs from unavailable in the fact that a defunct hart may never become available I don't see that much difference from the debuggers point of view to justify the complications.

The algorithm mandated (as a must) in Section 3.3—"selects each hart starting from 0 until either anynonexistent in dmstatus is 1"—still seems inefficient and unnecessarily restrictive.

The algorithm is not inefficient -- it's actually zero-overhead. A real-world debugger needs to read dmstatus to discover the current state of the hart (halted/running).
Current implementation in OpenOCD does hart enumeration separately, but this is an OpenOCD defect, not a defect of the spec.