What is a reference implementation?

[benjagm]

Context

In the last OCWM #907 , we discussed about what project to fund in the future and there was an idea to fund Reference Implementations. But what is a reference implementation?

We need to come up with a definition that clearly defines what a reference implementation is and complement it with clear eligibility criteria, commitments, and quantitative compliance data from Bowtie.

Some definitions:

• Wikipedia

Expectations for this issue:

• Provide definition
• Provide the eligibility criteria:
  • What makes those implementations different from others?
  • Can we have more than one by language?
  • Are they committed to implementing experimental capabilities?
• Define the bowtie compliance value

Next steps:

• Agree on the criteria.
• Highlight reference implementations.
• Support implementations:
  • Funding.
  • Promotion.
  • Others.

[jviotti]

I think the options that were discussed in the call were:

• Either the implementations made/maintained by the TSC, under the argument that a TSC member either knows what they are doing, or if a non-TSC member really knows what they are doing, they will likely get elected into the TSC
• Or implementations with a perfect or close to perfect score in Bowtie

I personally like option 1 more. The official test suite is not super vast (you can still get it to 100% coverage with various issues), and if we consider reference implementations to serve as a reference of how things should be done, there is no way that thought leadership will come from outside the TSC.

[gregsdennis]

if we consider reference implementations to serve as a reference of how things should be done

A reference implementation isn't an example of how things should be done. To the contrary, generally reference implementations are simplistic examples that serve as explanations of concepts without optimizations. 100% accuracy is their goal, not performance, and most cases they are not viable for use in production environments because of this.

[benjagm]

What could the requirements be?

• XX% bowtie coverage in the modern drafts?
• Implements the experimental capabilities?
• Provide specific documentation to guide/support other implementers?
• Member of the TSC?