Proposal to add support for libp2p-record in py-libp2p

[seetadev]

We propose implementing a new libp2p-record module in py-libp2p that follows the structure and behavior of the Go and JS versions, providing:

1. Record Abstraction

• Define a Record class that represents a typed, serializable key-value record with optional metadata (e.g., sequence number, TTL, validity, signature).
• Include serialization and deserialization methods (e.g., using protobuf).
• Implement compatibility with the protobuf schema used in go-libp2p.

2. Validator Interface

• Define a generic Validator interface:

  class Validator:
      def validate(self, key: bytes, value: bytes) -> bool: ...

• Support built-in validators for:

  • pk — validates a public key record
  • ipns — validates IPNS records (including public key extraction and signature verification)

• Allow developers to register custom validators by record namespace (e.g., /ipns/, /pk/, etc.).

3. Selector Interface

• Define a Selector interface to choose the best record from a set of candidates:

  class Selector:
      def select(self, key: bytes, records: List[bytes]) -> bytes: ...

• Implement default IPNS selector logic:

  • Choose the record with the highest sequence number and valid signature

4. Validator Registry

• Create a central registry or map to associate namespaces with their validator and selector functions, similar to:

  registry = {
      "ipns": (IPNSValidator(), IPNSSelector()),
      "pk": PKValidator()
  }

5. Protobuf and Crypto Integration

• Port or generate protobuf definitions for Record, IPNSEntry, etc., based on go-libp2p.

• Use existing crypto primitives in py-libp2p or py-ipfs for:

  • Keypair handling
  • Signature verification (e.g., RSA, Ed25519)
  • Peer ID recovery from public keys

6. Integration with DHT

• Integrate the libp2p-record module with the DHT once implemented or during the refactor.

• Add logic to:

  • Validate a record before storing it
  • Select the best record when multiple responses are received
  • Reject invalid or outdated records

7. Testing & Interop

• Unit tests for each validator and selector

• Tests for roundtrip serialization/deserialization of records

• Interoperability tests using go-libp2p or js-libp2p nodes for:

  • Storing and retrieving IPNS records
  • Public key resolution

Alternatives Considered

1. Ad-Hoc Record Validation Inside DHT Only

One option was to implement custom validation logic directly inside the DHT module. However, this approach:

• Couples DHT behavior tightly to record structure
• Reduces modularity and reusability across other protocols (like IPNS or PubSub)
• Makes testing and extending record validation logic much harder

Why rejected:
The libp2p ecosystem has standardized on a modular libp2p-record system used by DHT, IPNS, and other protocols. Following this pattern ensures long-term maintainability and interop.

2. Relying on go-libp2p or js-libp2p as a Backend

Another idea was to offload record validation to external libp2p daemons via RPC, avoiding implementation work in Python.

Why rejected:

• Adds complexity in cross-language communication
• Introduces runtime dependency on other binaries
• Violates the principle of py-libp2p as a standalone, native Python libp2p implementation

3. Limited Validator Support (Only for IPNS)

A minimal version could implement just enough for IPNS resolution and ignore generalized record validation.

Why rejected:
While sufficient for one use case, this would fragment the architecture and require rewrites later when other protocols (e.g., PubSub validators) are introduced.

Tasks / Checklist

Design & Core Implementation

• Create record/ module in the py-libp2p codebase
• Define Record class with serialization and deserialization (Protobuf)
• Add public key and IPNS-specific protobuf schemas
• Implement a Validator interface
• Implement a Selector interface
• Implement default PKValidator and IPNSValidator
• Implement default IPNSSelector

Crypto and Utility Support

• Add cryptographic verification utilities (e.g., signature check, peer ID recovery)
• Integrate key formats (RSA, Ed25519) already used in py-libp2p

DHT & Protocol Integration

• Add logic to validate records before storing in DHT
• Add record selection logic when resolving conflicting DHT values
• Prepare stub integration for IPNS

Testing & Validation

• Write unit tests for Record, validators, selectors
• Add serialization/deserialization roundtrip tests
• Add negative test cases (e.g., invalid signatures, bad TTL)
• Add interop test with go-libp2p record validation and selection logic

Documentation & Examples

• Add module documentation with usage examples
• Document how to register custom validators/selectors
• Add integration notes for developers building IPNS / PubSub features