protobuf error with test go-ipfs-repo

[AdamStone]

I'm noticing some weirdness with the cli test repo. The object at QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG contains links to the starting materials (about, contact, help, quick-start, readme, security-notes). When trying to get some of the linked objects such as readme (QmPZ9gcCEpqKTo6aq61g2nXGUhM4iCL3ewB6LDXZCtioEB) from within the cli tests (which are based on a copy of go-ipfs-repo), I'm getting an error "Groups are not supported".

Apparently this error is associated with protocol buffers. It's occurring at node.unMarshal(block.data) in the dag service, but I think the problem is with the block data rather than the dag service because I don't get this error using the same console command against the ~/.ipfs repo (node ./src/cli/bin.js object get QmPZ9gcCEpqKTo6aq61g2nXGUhM4iCL3ewB6LDXZCtioEB).

Example of a test that shows this:

nexpect.spawn('node', [process.cwd() + '/src/cli/bin.js', 'object', 'get', 'QmPZ9gcCEpqKTo6aq61g2nXGUhM4iCL3ewB6LDXZCtioEB'], {env: env, stream: 'all'})
  .run((err, stdout, exitcode) => {
    console.log(stdout)
    done()
  })

[daviddias]

I believe this is because js-ipfs-unixfs-engine doesn't support sharding or node wrapping like go-ipfs does. Making it so that some files added through go-ipfs not readable on js-ipfs (for now)

ipfs-inactive/js-ipfs-unixfs-engine#45

@whyrusleeping are we settling on the trickle-dag for future versions of go-ipfs ipfs add? (ref: ipfs/specs#98)

[whyrusleeping]

@diasdavid I think that the trickledag code is irrelevant to what is going on here.

[daviddias]

I mean, in js-unixfs-engine we don't 'understand' (yet) all the dag structures you do in go-ipfs to represent files, just the simplest one. I believe that is the issue here.

[daviddias]

Follow up on ipfs-inactive/js-ipfs-unixfs-engine#45

@AdamStone would you like to tackle and implement this one?

[dignifiedquire]

To make sure we can handle all different versions of things that go-ipfs stores @whyrusleeping could you generate a small repo, with a set of all the different shardings that go-ipfs currently supports please?

[daviddias]

• + list the hashes of which is which :)

[AdamStone]

Would you prefer I work on this next, or the datastore? I can take a closer look at either, I'd like to get the pinning implementation finalized first though.

[daviddias]

@whyrusleeping could you provide #300 (comment)

[daviddias]

@whyrusleeping see: #300 (comment)

To make sure we can handle all different versions of things that go-ipfs stores @whyrusleeping could you generate a small repo, with a set of all the different shardings that go-ipfs currently supports please?

[daviddias]

Is this still an issue?

[whyrusleeping]

I don't think i have anything to do here.

a set of all the different shardings that go-ipfs currently supports

This set is infinite, i can do the file layout any number of ways, and the parser code should have no trouble dealing with any of it.

[daviddias]

Our unixfs-engine is now 'complete' thanks to @pgte and can handle all the DAGBuilders. I believe this isn't an issue anymore, @AdamStone can you provide a test case so that we validate it?

Thank you!

[daviddias]

Closing this issue, @AdamStone let me know if you manage to reproduce the problem.