Verify lengths of waits recorded correctly

[philipstarkey]

The length of waits should be recorded by the PrawnBlaster (to +/- 1 clock cycle accuracy).

This should be checked (while noting that the values returned by getwaits are actually the remainder of the timeout, not the length of the wait)

[dihm]

I have to wrap up my day now, but I have some very preliminary results on waits with timeouts. I'm using a standard pulseblaster to send a hwstart trigger pulse then a retriggering pulse at a hard-coded time. I manually program the prawnblaster to be waiting at approximately that time. It's janky because I have to manually start the prawnblaster, timetagger, and BLACS, but it appears to be OK for now.

I'm observing something where I can only specify a wait timeout up to ~1ms. Anything longer caps there. Anything shorter times out at the expected time.

I'm also having trouble getting the Prawnblaster to retrigger once it has hit a wait. This may very well be user error given the janky setup.

[dihm]

Now that I'm not rushed, I have sorted things out.

I'm not seeing the first issue anymore. Suspect I just had some stale data in the analysis somewhere. I can observe correct timing of timeout retriggering, and the wait measurement reports 0 as expected.

The second issue was definitely user error in programming the pulseblaster triggers. I have it sorted out now. The PB sends an initial trigger (for the hwstart) then a second trigger 15us later (each trigger is 150ns long thanks to the min trigger length of the PB itself). The PrawnBlaster instruction set I'm working with is quite simple.

set 0 100 4
set 1 50 2
set 2 1000 0
set 3 50 3
set 4 0 0

Here are three representative time traces taken with the timetagger. I've also set a second PB channel that pulses with the Prawnblaster triggers into the timetagger for reference (green traces). All times are referenced to the rising edge of the first PB trigger.

Here is the data for the prawnblaster edges from this plot. Including the data for the reference PB channel was annoying, but I've checked that the wait retriggering edge is within +/-0.1ns of 15us after the first pulse rising edge.

Three things to note:

1. It appears the hwstart results in a 9 clock cycle delay from the triggering rising edge. I assume this is due to 3 clock cycles for detecting the pulse and a further 6 to program the first output. There is also about 5ns of timing jitter, no doubt due to the trigger pulse starting during a clock cycle.
2. The wait retrigger appears to have a 3 cycle delay, as expected (once the 9 cycle delay from the first pulse is removed). Similar 5ns timing jitter is observed.
3. The reported wait is a rock-solid 508 (giving a 4920 ns delay, 5000 expected). This time tracks with small changes in the PB trigger timing within +/-1 clock cycle. The rest of the difference comes from the hwstart delay causing a slip relative to the PB timing that sets the retrigger pulse.

So I'd say that the timeout waits are working well and the timing seems accurate at first blush. How rigorously do we want to test the wait timing? I could probably dream up a more automated system but it would take some effort and different hardware than what I have right now.

[dihm]

I've also just done some testing with indefinite waits. They appear to work with 150ns retrigger pulse length just fine. The only thing is that I am getting a significantly longer retrigger delay (~180ns) than before. It's probably as expected, but I can't figure out the exact numerology.

I'm also finding that that getwait 0 is returning a number that is not zero (consistently 999508). getwait 1 does return 0. My slightly modified instruction set is

set 0 100 4
set 1 50 2
set 2 10 0
set 3 10 0
set 4 50 3
set 5 0 0

Edit: fixed sequence instructions.

[philipstarkey]

That's actually a bit confusing. I would have expected the hwstart (and indefinite wait) to be 2-3 clock cycles shorter than the retrigger (when not an indefinite wait). And the jitter on the retrigger (not indefinite wait) should be +/- 10ns (one assembly instruction)! So I'm surprised you're seeing the wait retrigger be shorter than the hwstart.

I can't reproduce the odd wait length you are seeing with indefinite waits either. When I ran the program you listed (with fixed instruction numbers since you are missing instruction address 2!), I got a wait length of 0 (or 10 if I continually held the retrigger pin high which is as expected). So not really sure what is going on there either!

When you said the retrigger delay was 180ns for an indefinite wait, what was that measured from? Once it's in the second wait (after the first wait timeout has elapsed) it should be as fast as the hwstart (it's using the same bit of assembly code). But diagnosing indefinite waits is probably not particularly important though come to think of it as I've just realised labscript requires a finite wait timeout anyway!

It would be good to get a definitive explanation for the hwstart and retrigger delays though as labscript requires that those delays are the same, which means I need to work out where to pad the assembly code to make them the same length!

Ohhhh....Just realised you are using a pulseblaster to do this, but you said the pulseblaster is running from it's crystal (and not externally referenced)? Would that maybe explain why your measured delays don't align with what I expected?

[philipstarkey]

Actually, looking at the raw data, I think it might be fine? The PrawnBlaster takes 100ns to respond to the first hardware start trigger. The second trigger occurs at 15us and the PrawnBlaster responds ~120ns later. I don't think you need to subtract off the initial hwstart trigger delay here because it was absorbed by the wait (and reported wait length). This puts the retrigger as 2 clock cycles longer than the hwstart. I suspect if you set the PB retrigger time to be 10ns later then we would see the retrigger time be 130ns (3 clock cycles longer - but not sure if the PulseBlaster has the resolution to check this).

I have a suspicion that the wait length is out by 20ns (in that I have an off-by-one error somewhere in my code). It would make more sense for it to be reported as 510 (4900ns wait) so that the initial start trigger + wait length was equal to the expected wait length. I'll see if I can figure out a logical explanation for this based on the assembly code.

[philipstarkey]

All I've found from looking at the assembly code is the wait should actually be 30ns longer than reported (because that's how long it takes before it's ready to respond to the trigger).

This is opposite to what I was hoping for! I think...let me write out things again because I'm getting confused.

The PrawnBlaster starts at 100ns (PulseBlaster time), and the program runs for 10us, so it enters the wait at 10.1us. There is a 30ns (0.03us) delay before it starts counting the wait. It reported 508 for the wait time, which means 4.92us. We should not count how long it takes the pico to detect the pulse because we only know what the assembly code sees, not what the internal buffer of the pico is doing. So it exits the wait at 15.05us (pulseblaster time) and begins the next pulse 70ns later. Huh...and there are exactly 7 asm instructions between the wait condition being met and the output being set high! So that is all reasonably consistent.

What I guess is not consistent is that we're saying it exits the wait at 15.05us when we expect that to be 15.0us. There is definitely not 50ns of buffering on the pico chip between the signal and the wait ending. There is 2 cycles of buffering in the pico chip. Maybe (up to) an additional 19ns in my assembly code loop if the trigger occurs at just the wrong moment. But let's consider the initial trigger again (the hwstart). This should only take 7 clock cycles (2 pico buffering cycles and 5 assembly instructions). But we see 10. If we were to assume that the PulseBlaster is actually triggering the PrawnBlaster at t=30ns, then that means we would expect to see the second trigger at 15.03us, which is 20ns prior to when the PrawnBlaster detects the wait ending - exactly the expected time the pico buffers the input.

So that would actually appear completely consistent with what I expect to happen, assuming that (a) the PulseBlaster clock is not losing synchronisation with the PrawnBlaster clock over 15us and (b) the first PulseBlaster trigger to the PrawnBlaster is occurring 30ns later than you think.

Not sure if that rambling makes a huge amount of sense, but, based on the analysis of my code, we should see:

1. The first PrawnBlaster pulse 70ns after trigger,
2. The measured wait be 30ns longer than reported by the PrawnBlaster
3. The PrawnBlaster to take 70ns between end of measured wait and start of next pulse (or 90ns after trigger)

And I think we do see this if the PulseBlaster triggers are actually occurring at 0.03us and 15.03us relative to the data you shared.

[dihm]

I'll admit I'm going to have to read all the above a few times to try and follow it. Following the timings here is a bit confusing. Just a couple notes to follow-up right now while you have time on the weekend.

1. Oops on the sequence. I have written code that automatically populates instruction numbers from an array of times and reps so I have to write these comments out by hand.
2. I am pretty confident the pulseblaster timing. I set a mirroring pulse channel to the triggering pulses that is simultaneously measured by the timetagger. Those green traces are measured (rather than just calculated like the orange dashed), and all times are referenced to the measured first rising edge of the pulseblaster. For better or worse, it is the absolute clock reference right now (and it is not externally clocked, being a USB enclosure version).
3. I think sub 10ns jitter makes sense under an implicit assumption I made about how the Prawnblaster detection works, so maybe you can correct my understanding here. I assume that if a pulseblaster trigger begins mid-clock cycle on the prawnblaster, it is unlikely for the prawnblaster to count that clock cycle in part of its trigger detection. Put another way, the 3 cycle detection circuit requires 3 complete (or nearly complete) cycles with trigger high. Since the clocks are uncorrelated, an observed diff in times basically measures the difference and should be expected.

In any case, I'm thinking I might need to modify the test rig a bit to better automate things and try to remove some of these ambiguities. If you have any particular suggestions, by all means. Assume you have a very well stocked lab with any piece of equipment you could imagine and I can work from there.

[philipstarkey]

• I am pretty confident the pulseblaster timing. I set a mirroring pulse channel to the triggering pulses that is simultaneously measured by the timetagger. Those green traces are measured (rather than just calculated like the orange dashed), and all times are referenced to the measured first rising edge of the pulseblaster. For better or worse, it is the absolute clock reference right now (and it is not externally clocked, being a USB enclosure version).

Hmm. I might create a simplified firmware that will allow you to measure how long it takes the Pico to respond to a trigger. Your data seems to point to the trigger detection taking 3 cycles longer than I expect, which is odd, but I guess not out of the question.

• I think sub 10ns jitter makes sense under an implicit assumption I made about how the Prawnblaster detection works, so maybe you can correct my understanding here. I assume that if a pulseblaster trigger begins mid-clock cycle on the prawnblaster, it is unlikely for the prawnblaster to count that clock cycle in part of its trigger detection. Put another way, the 3 cycle detection circuit requires 3 complete (or nearly complete) cycles with trigger high. Since the clocks are uncorrelated, an observed diff in times basically measures the difference and should be expected.

There are several different components to the trigger detection for waits. The first is inside the RP2040 chip on the pico, which I believe buffers the signal for 2 cycles (to avoid detecting spurious edges). The second is inside my assembly code. The assembly code consists of 2 assembly instructions in a loop. The first jumps out of the loop if the input pin is high. The second jumps back to the first instruction as long as a counter variable is not 0 (and then subsequently decrements the counter regardless of whether it jumps or not...which I find odd but that's what the RP2040 designers decided to do!)

So if the trigger arrives just after a clock tick, then it would take 20ns-29.99ns to get through the RP2040 buffer. Assuming this buffer is synchronised to the PIO core running my code, then there is a potential 0-10ns delay depending on which instruction in my loop is currently being run. So that gives a possible 20-39.99ns delay range from what I can tell. But, if everything is sychronised, I would expect to see the same delay on every shot for the same instruction set. Slightly shifting the timing of the trigger pulse though may result in a bigger jump in observed delay if it wraps from 20ns to 39.99ns though. And this only applies to resuming from a finite wait, not an initial trigger (or indefinite wait)

None of this helps to explain the data though. It is very odd that the PrawnBlaster does not start until t=100ns. Once the trigger makes it to the PIO core there is only a delay of 5 clock cycles from assembly instructions between trigger and output. I think 2 more can be explained by the RP2040 buffering, but I don't understand the other 3. I think once we solve that then maybe we can make sense of the wait length (maybe...)

[dihm]

So I've been testing wait retriggering some more with the master branch (at d2adf1e). I'm also testing using all four clocks with the same program, and using the delay generator to send in retriggers after being triggered by the first rising edge of the prawnblaster (basically the same setup as in #9). This time I did some testing on the delay generator first. The way it works, I'm triggering the prawnblaster off the front panel outputs and using a combined output on the back into the time tagger to confirm timing. Turns out, there is ~8.3ns delay going from front to back, so I've subtracted that out from the delay generator reference pulses that I measure.

Anyway, my current results are (for 7,8,9,10 us retrigger times) that the prawnblaster restarts 119.4, 119.9, 121, 101 ns after the pulse. The self-reported waits are 684, 584, 484, 386. I've also checked that clock 3 always retriggers 20ns faster, even if I trigger it first or at the same time as all of the others.

Here is the raw data. Note that channels 6 and 11 are the same signal from the front and rear panel (which can be used to correct the measured retrigger pulses for channel 10).

So now not only am I confused on how long the retrigger delay should be, but it isn't even consistent across clocks. Great. Any specific requests for things to measure?

[philipstarkey]

Could you repeat again with the latest firmware in master? I've made a few changes (the changes in #12 that I merged in required some tweaking in how I calculate wait lengths

I think everything is lining up (for the old firmware) except for how long it takes to respond to a trigger. I think you might have a typo in your previous message as it looks like clock 3 restarts 92ns (not 101ns) after the 4th trigger pulse?

Anyway, working from your raw data (and reported wait values), we have the PrawnBlaster entering waits at 4.00us. The first wait is 3.16us long, second wait is 4.16us, third is 5.16us and forth is 6.14us (for clocks 0-3 respectively).

This gives the prawnblaster detecting the trigger at 7.16us, 8.16us, 9.16us and 10.14us respectively (according to the reported wait length at least)

I all cases the prawnblaster is resuming 60ns after this. Technically this should be 70ns (based on assembly instruction count) but the wait length is only accurate to 2 clock cycles I think. This discrepancy may be "fixed" in the latest firmware because instead of 3 clock cycles before trigger, and 7 clock cycles after trigger, it's now 4 and 8. Which is nicely divisible by 2 and shouldn't get mangled during divide/multiply by 2 that I have in the code.

That means it took the PrawnBlaster 60ns to detect the trigger, except for clock 3 which was 30ns....I can't really explain this!
(I guess technically if we assume it was 70ns between trigger detection and resume as the assembly instructions imply then these numbers would be 50ns and 20ns...).

Do you have the ability to offset all of the triggers by a nanosecond? Could be worth seeing how they all respond to slightly shifting the trigger times relative to the first PrawnBlaster pulse (e.g. all by 1ns, then 2ns, then 3ns, .... then 20ns) as well as testing with the latest firmware (which, unrelated, should now report timed out waits as 4294967295 to distinguish from waits resumed by trigger in the last iteration of the wait loop).

[philipstarkey]

I've also made #14 which might change things 🤷