Skip to content

Conversation

@ariel-miculas
Copy link
Contributor

It is similar to BufferSegments, but it avoids any allocations.

It is similar to BufferSegments, but it avoids any allocations.
@ariel-miculas
Copy link
Contributor Author

With this PR I can use capnp in the linux kernel, I've tested it and it works (with the honorable mention of removing the f32/f64 match arms in stringify.rs). See https://github.com/ariel-miculas/linux/tree/puzzlefs_rfc for details.

ariel-miculas added a commit to ariel-miculas/linux that referenced this pull request Jul 19, 2023
…etadata

Puzzlefs uses capnproto for storing the filesystem metadata, defined in
manifest.capnp and metadata.capnp. The files manifest_capnp.rs and
metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4):
```
capnp compile -orust manifest.capnp
capnp compile -orust metadata.capnp
```
, formatted with rustfmt and I have also added the following lines:
```
```
to avoid warnings.

Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically
generated at build time. However, this means the capnp binary becomes a
dependency for the build and I didn't want this. Besides, the metadata
schemas are not expected to change frequently, so it's acceptable to
manually regenerate the two rust files when this happens.

The code is adapted from the puzzlefs FUSE driver [1]. The data
structures required for the filesystem metadata are defined in types.rs.
Each structure has a `from_capnp` method used to convert between the
capnp memory layout and the native Rust structures.  This leaves room
for improvement [2].

The deserializing code also uses `NoAllocBufferSegments` which is not
yet merged in capnproto-rust upstream [3]. When a new release is
published, then we'll be able to use capnproto-rust in the kernel with
only a few patches to it.

inode.rs implements the PuzzleFS structure which contains a list of
metadata layers.

Link: https://github.com/project-machine/puzzlefs [1]
Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2]
Link: capnproto/capnproto-rust#423 [3]

Signed-off-by: Ariel Miculas <[email protected]>
@ariel-miculas
Copy link
Contributor Author

@dwrensha when could you take a look at this?

@dwrensha
Copy link
Member

dwrensha commented Aug 8, 2023

I expect to have time in about two weeks.

@dwrensha dwrensha merged commit 491a5b1 into capnproto:master Aug 21, 2023
@dwrensha
Copy link
Member

Thanks!

@dwrensha
Copy link
Member

I made NoAllocSliceSegments a type alias for `NoAllocBufferSegments<&[u8]> in 4015fa0 .

@dwrensha
Copy link
Member

@ariel-miculas in #475 I change the Deref<Target=[u8]> to AsRef<[u8]>. Please let me know if you foresee any problems with that. I don't expect any problems, because if a type implements the former then it should be easy to add an impl for the latter (if it does not yet already exist).

@ariel-miculas
Copy link
Contributor Author

Looks straightforward to me, thanks for the heads-up.

ariel-miculas added a commit to ariel-miculas/linux that referenced this pull request Mar 26, 2024
…etadata

Puzzlefs uses capnproto for storing the filesystem metadata, defined in
manifest.capnp and metadata.capnp. The files manifest_capnp.rs and
metadata_capnp.rs were generated with (Cap'n Proto version 0.10.4):
```
capnp compile -orust manifest.capnp
capnp compile -orust metadata.capnp
```
, formatted with rustfmt and I have also added the following lines:
```
```
to avoid warnings.

Ideally, manifest_capnp.rs and metadata_capnp.rs should be automatically
generated at build time. However, this means the capnp binary becomes a
dependency for the build and I didn't want this. Besides, the metadata
schemas are not expected to change frequently, so it's acceptable to
manually regenerate the two rust files when this happens.

The code is adapted from the puzzlefs FUSE driver [1]. The data
structures required for the filesystem metadata are defined in types.rs.
Each structure has a `from_capnp` method used to convert between the
capnp memory layout and the native Rust structures.  This leaves room
for improvement [2].

The deserializing code also uses `NoAllocBufferSegments` which is not
yet merged in capnproto-rust upstream [3]. When a new release is
published, then we'll be able to use capnproto-rust in the kernel with
only a few patches to it.

inode.rs implements the PuzzleFS structure which contains a list of
metadata layers.

Link: https://github.com/project-machine/puzzlefs [1]
Link: https://lore.kernel.org/rust-for-linux/[email protected]/ [2]
Link: capnproto/capnproto-rust#423 [3]

Signed-off-by: Ariel Miculas <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants