[RFC] DI/OT FMC Carrier

[marmeladapk]

I'm designing FMC carrier board for DI/OT project for CERN. The specification is here. Basically it's a board in the same ecosystem as the system board which Greg discussed with you earlier.

This board will have Kintex Ultrascale FPGA, DDR4 SODIMM slot and FMC HPC. I'll probably be able to also include SD card or eMMC storage (requested in spec). One of the points in spec which I need to clarify is presence of USB connector.

This board could be a good fit for Shuttler FMC.

Is there something else required for ARTIQ support other than Si549? It already has a MGT connection to system board.

@sbourdeauducq @jordens @dhslichter

[jordens]

Nice. I don't see anything that would be needed. And I think that the clock recovery mechanisms of wr and drtio should be unified.
Assuming I read the spec correctly, are you sure you can leave the vadj fpga banks unpowered?

[marmeladapk]

are you sure you can leave the vadj fpga banks unpowered?

I read further and found that in principle VCCO is not strictly required for unused banks but

The IO protection ESD structures are designed to work while powered (best protection).

In other ARs and in UG they advised against leaving VCCO floating and didn't write about connecting it to GND, but I guess it's not worth the risk. Thanks for the tip!

[jordens]

Right. That was also my conclusion for the potentially unused bank on Phaser.

[marmeladapk]

@jordens @sbourdeauducq Would it be ok if I connect output from main Si549 to one of inputs of Si5341? In DIOT system board CERN is doing the same with their main VCXO - its output is connected to the input of Si5341.

[jordens]

To save a mux/fanout? Fine with me. Noise and determinism should not be affected, but that needs to be confirmed by someone else.

What's the CERN rationale for the Si5341? Is it a backup/alternative for WR clock recovery or just a synthesizer/buffer/clock fanout/divider?

[marmeladapk]

Yes, it's to simplify the design.

Si5341 seems to be all in one solution for clocks, Silabs even advertises it that way "clock tree on a chip". Clock from backplane is an input to Si5341.

[jordens]

di/ot zu clocking

They have no plan to use it as an alternative/successor for WR clock recovery?
Ultimately you'd hope that the WR recovery scheme can be replaced by a smaller/cheaper/better integrated solution. For us the only real reasons were deterministic phase which wasn't available and maybe a bit the performance hit. The phase issue looks solved with the zero delay mode and the synchronized dividers. One could even consider putting more of the tree into the feedback loop. Performance-wise it looks better than the WR-VCXO.

[jordens]

@marmeladapk Skipping Kasli-SoC in favor of this would be very valuable IMO. Or at least making the transition from Kasli-SOC easy for those that want to stick to ribbon cables for now.
Is there any effort on that compatibility front yet?

[marmeladapk]

@jordens Do you mean skipping KAsli-SoC in favor of DIOT System board?

Is there any effort on that compatibility front yet?

In what way?

@gkasprow consulted system board with you and there are Si549 oscillators there as well. Each slot gets 15 LVDS pairs + MGT in CERN variant or 16 LVDS pairs in Sinara variant. @gkasprow also designed CPCIS adapters for existing EEMs. What else do you have in mind?

[jordens]

@jordens Do you mean skipping KAsli-SoC in favor of DIOT System board?

Yes. Transition to cPCIS and SoC at the same time.

Is there any effort on that compatibility front yet?

In what way?
@gkasprow consulted system board with you and there are Si549 oscillators there as well. Each slot gets 15 LVDS pairs + MGT in CERN variant or 16 LVDS pairs in Sinara variant. @gkasprow also designed CPCIS adapters for existing EEMs. What else do you have in mind?

The way I see it we might be able to still make small changes to your DI/OT FMC carrier but we might be able to make bigger changes to Kasli-SoC to align them better. I'd like profit from the CERN involvement (review, development, characterization, testing, volume, cost). Pretty sure that Kasli-SoC will not profit much from that.

• Clocking: IMO Kasli-SoC should use the DI/OT clocking tree (add 4 MMCX). It support all options (WR CR with VCXO, DCXO, and Si5x-based recovery).
• I guess the FMC on DI/OT could be used with 4 MGTs to 4xSFP for DRTIO. 1.8 Vadj is not a problem there?
• DDR4 might be a bit of a challenge. But certainly a step forward.
• Same for the Ethernet section. We'd have to go pass Ethernet through an SFP+MGT and PL to PS, AFAICT? Might even be preferrable compared to the copper PHY.
• I2C bus tree differences? Looks like bigger delta but less critical.
• Any chance to get an FTDI for JTAG on the DI/OT board? What does JTAG over backplane look like in the ZU case?
• Do we get a front panel clock input to the tree?

(Looking at the ZU DI/OT board)

[sbourdeauducq]

Yes. Transition to cPCIS and SoC at the same time.

I do not like this idea; the Kasli-SoC design is almost ready and it can support the existing cards already.

I'd like profit from the CERN involvement (review, development, characterization, testing, volume, cost).

There are also disadvantages to it, such has having even less control over the designs than we have currently.

[hartytp]

Without wanting to make a judgement on what anyone else should/will do, I agree that Kasli-SoC is a good next step for us.

A backplane is at best a nice to have for us (personally I don't think the ribbon cables are that big a deal). One of the lessons I took from the original Kasli is that keeping the designs minimal complexity and totally under our control was a real benefit in terms of getting working hardware. I'm apprehensive about coupling our use-cases to those of a big organisation like CERN.

[hartytp]

I'll be interested to follow this project, but we'll keep pushing Kasli-SoC forward for the foreseeable.

[marmeladapk]

Any chance to get an FTDI for JTAG on the DI/OT board? What does JTAG over backplane look like in the ZU case?

Unfortunately it's too late for that, the system board is already routed. Maybe in the next revision if there is one.
I pasted FTDI schematics from Kasli/Sayma in my FMC carrier, we'll see if it sticks.

What does JTAG over backplane look like in the ZU case?

AFAICT DI/OT SB is the source for JTAG for other cards. When it asserts SERVMOD_N signal then cards should connect some BP signals to JTAG chain and also connect their I2C to BP. (see crate monitoring)

There are also disadvantages to it, such has having even less control over the designs than we have currently.

It's open hardware, so if something doesn't work we can modify the card to our liking. Even if we modify the card, we could benefit greatly from common BoM.

Anyway my goal is to look for possible changes that could be beneficial for us and neutral for CERN. I wouldn't axe Kasli-SoC, since we don't even have ARTIQ support for ZU.

[hartytp]

Anyway my goal is to look for possible changes that could be beneficial for us and neutral for CERN. I wouldn't axe Kasli-SoC, since we don't even have ARTIQ support for ZU.

I completely agree with this. It's definitely worth us having input on these projects and steering them where we can. I just wouldn't want that to be at the detriment of Kasli-SoC.