Memory requirement for new test framework

[stnolting]

I'm trying to port the new test framework to a custom RISC-V core.

While testing each of the new tests I came across the jump/branch tests - for example the I/jal-01.S test:

inst_0:
// rd==x21, imm_val < 0, 
// opcode: jal; dest:x21; immval:0x55556; align:0
TEST_JAL_OP(x13, x21, 0x55556, 1b, x2, 0,0)

As far as I can see the "immval" (=0x55556) in this test case defines the "distance" or offset of the jump-and-link target by inserting NOPs. Thus, the generated program requires more than 400kB of program memory - mainly consisting of NOPs.

My question - part 1:
What is the purpose of this large offset? I think the actual functionality could also be tested even with smaller offsets.
Or is the intention to test all/most of JAL's 20-bit offset capabilities?

My question - part 2:
Is there any option to globally reduce these large offsets to generate a smaller program? For example if the targeted processor is memory-limited (let's say a small microcontroller with only a few kB of memory)?
Or is this not possible as the reference results include data words that are based on these (fixed?) large offsets?

[allenjbaum]

That's a good question. The intention is exactly as you surmise: - to test JALs architected 20-bit offset, - to ensure that the fullbranch address calculation is made, and - to ensure the immediate constant is properly constructed. The second part of your question is a lot more complicated to answer. In the short term: no. There is a fixed reference signature, and a fixed set of tests that are run. In the (slightly) longer term, framework 3.0 will take configuration parameters from a YAML encoded file, apply it to the golden reference model, and have it generate the signature - which could match. The test generator would need to accept not only any architectural configuration, but also the platform configuration, and only generate tests that don't accept platform limits. This is not simple for a variety of reasons, but a big one is that programs can be loaded at implementation specified locations, which makes configuring tests that exercise this kind of limit fairly tricky. We can compensate in Load/Store tests by pre-adjusting the base register, but JALRs have even worse issues than the +/-20bit JAL range. Before an IOT chip gets tested, the RTL or C-model that is used to implement it can be tested, and that doesn't have to have the memory limits that the chip has. But there is a basic question here; suppose it's an IOT chip with very limited memory, and so it only implements the low 16 bits for address calculations. Can that be considered a compatible implementation? IF the PMA limits access, any attempt to branch must trap with an illegal access exception - or we can't test it, and that isn't architecturally required Also, there is no standard way to begin to describe a PMA to the reference model; that would need to be invented. IF you can do that, it may architecturally compatible - but only if the full address calculation is made The test generator would need to be smart enough to generate branches and loads to illegal addresses, and not try to load anything at those illegal addresses. There is no simple solution.

…

On Tue, Dec 15, 2020 at 12:08 PM Stephan ***@***.***> wrote: I'm trying to port the new test framework to a custom RISC-V core. While testing each of the new tests I came across the jump/branch tests - for example the I/jal-01.S test: inst_0: // rd==x21, imm_val < 0, // opcode: jal; dest:x21; immval:0x55556; align:0 TEST_JAL_OP(x13, x21, 0x55556, 1b, x2, 0,0) As far as I can see the "immval" (=0x55556) in this test case defines the "distance" or offset of the jump-and-link target by inserting NOPs. Thus, the generated program requires more than 400kB of program memory - mainly consisting of NOPs. My question - part 1: What is the purpose of this large offset? I think the actual functionality could also be tested even with smaller offsets. Or is the intention to test all/most of JAL's 20-bit offset capabilities? My question - part 2: Is there any option to *globally* reduce these large offsets to generate a smaller program? For example if the targeted processor is memory-limited (let's say a small microcontroller with only a few kB of memory)? Or is this not possible as the reference results include data words that are based on these (fixed?) large offsets? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#157>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJWYAOVDLWITWPP5HDDSU663TANCNFSM4U442VNQ> .

[stnolting]

Thank you for the detailed answer!
I managed to use a simulation-only model of the memory to run the compliance tests overcoming the physical memory limitations. Now the whole test suite (v2.1) works like a charm 👍

I am wondering, is there something like an "official label" like RISC-V-Compliant that can be used for a core if it passes certain tests? 🤔

By the way, I have noticed the reference signatures use zero-padding to make the size of the signature always a multiple of 4 words (at least for rv32i). I cannot remember to have read that anywhere in the documentation, so it might be helpful to add a note somewhere.

But there is a basic question here; suppose it's an IOT chip with very
limited memory, and so it only implements the low 16 bits for address
calculations.
Can that be considered a compatible implementation?

Good question...
From my humble point of view a core can only be compliant/compatible/whatever to the standard if it fulfills the specifications. When the spec says N bit are used for certain things, N bits have to be supported/implemented. But of course any constraints of architecture-specific issues like address computations could be seen as "custom extension"... But how to check that?! I am stumped...
I'm looking forward to the next version of the framework. 😉

[allenjbaum]

fantastic - thanks for letting me know.

…

On Mon, Feb 15, 2021 at 9:55 AM stnolting ***@***.***> wrote: Closed #157 <#157>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#157 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJXZNRX3YDYMAKAHYDTS7FNYVANCNFSM4U442VNQ> .