safety: introduce pointer types and their restrictions #208

muzarski · 2024-11-26T15:47:47Z

This is a follow-up to: #207.

It defines a CassPtr type, wrapper over Option<NonNull<T>>. The pointer is parameterized by:

Lifetime

Thanks to that, we can distinguish between owned (should be freed) and borrowed pointers.

Properties

There are currently two properties:

Ownership - it tells whether the pointer is mutable from Rust perspective. In other words, Exclusive = mutable pointer, Shared = immutable pointer.
CMutability - the semantics of this one are very simple. It just tells whether the pointer is T* or const T* in C API definition. It's introduced so we can distinguish shared, logically mutable types (Mutability::Const and CMutability::CMut) from shared, logically immutable types (Mutability::Const and CMutability::CConst). This property will help to write safe unit tests where we play a role of C API user.

During review, please pay attention to that if some pointer was originally const T* (or *const T in Rust), then it is CConst after this PR. Analogously, if it was T* (or *mut T in Rust), then it should be CMut.

Pre-review checklist

I have split my patch into logically separate commits.
All commit messages clearly explain what they change and why.
PR description sums up the changes and reasons why they should be introduced.
I have implemented Rust unit tests for the features/changes introduced.
~~[ ] I have enabled appropriate tests in .github/workflows/build.yml in gtest_filter.~~
~~[ ] I have enabled appropriate tests in .github/workflows/cassandra.yml in gtest_filter.~~

wprzytula

The approach is very impressive.
I couldn't check all the changes, there were too much of them; I just peeked through them. I trust your meticulousness :)

scylla-rust-wrapper/src/collection.rs

scylla-rust-wrapper/src/metadata.rs

scylla-rust-wrapper/src/argconv.rs

muzarski · 2024-12-02T15:05:58Z

Rebased on master

muzarski · 2024-12-02T19:32:07Z

v1.1: Addressed @wprzytula comments

wprzytula · 2024-12-04T12:09:41Z

Waiting with review for a rebase on #207.

scylla-rust-wrapper/src/argconv.rs

muzarski · 2024-12-04T12:38:42Z

Waiting with review for a rebase on #207.

Everything is up to date

scylla-rust-wrapper/src/argconv.rs

muzarski · 2024-12-04T18:09:34Z

v2: Improved pointer API based on @wprzytula suggestions. Thank you!

removed Copy and Clone implementations for CassPtr (currently, it's only defined for test purposes. I need to think how to address it some other way)
Left CassPtr::into_ref() as is. As discussed in the comment, it is required to "erase" the lifetime of the returned reference. This method consumes the pointer.
Introduced CassPtr::as_ref() method. It accepts a reference to the pointer. The lifetime of reference is inherited from pointer's lifetime.
Changed CassExclusiveMutPtr::into_mut_ref() to [...]::as_mut_ref(). It accepts &mut CassExclusiveMutPtr. The lifetime of returned mutable reference is inherited from mutable borrow of the pointer. This is really cool, because this ensures aliasing ^ mutability at borrow-checker level.
CassPtr::is_null() now borrows the pointer, instead of consuming it
CassFuture::set_callback(), and CassFutureCallback now expect a raw pointer. Safe CassPtr::as_raw() method is introduced for this use-case

I still need to think how to address the issue in tests. It's not possible to reuse the pointer, which is not Copy, while passing it to multiple API functions that consume the pointer.

wprzytula · 2024-12-04T20:30:57Z

I still need to think how to address the issue in tests. It's not possible to reuse the pointer, which is not Copy, while passing it to multiple API functions that consume the pointer.

You could have an unsafe clone() method for tests only, then this would not break soundness guarantees of safe code.

muzarski · 2024-12-04T22:43:47Z

I still need to think how to address the issue in tests. It's not possible to reuse the pointer, which is not Copy, while passing it to multiple API functions that consume the pointer.

You could have an unsafe clone() method for tests only, then this would not break soundness guarantees of safe code.

Applied the suggestion. At first, I did not understand how is this different from implementing Copy and Clone for test only. But this part: "this would not break soundness guarantees of safe code" convinced me.

scylla-rust-wrapper/src/argconv.rs

scylla-rust-wrapper/src/batch.rs

muzarski · 2024-12-09T12:46:02Z

Rebased on master

muzarski · 2024-12-09T12:54:07Z

v2.1: improved docstrings as suggested by @wprzytula

Before this commit, the pointer was obtained from a valid reference &CassFuture, which is totally fine. However, I want to reduce the ways one can obtain such pointer. For ArcFFI (shared pointers), I want them to be obtainable only in two ways: - `ArcFFI::as_ptr()` which accepts an &Arc - from the user, as a function parameter This way, we are guaranteed that the pointer comes from a valid Arc allocation (unless user provided pointer to some garbage, but there is no much we can do about it). If we assume that user provides a pointer returned from some prior call to API, we are guaranteed that it is valid, and comes from an Arc allocation (or is null). I don't want to allow ArcFFI api to create a pointer from a refernce, to prevent creating a pointer, from example from stack allocated object: ``` let future = CassFuture { ... }; let future_ptr = ArcFFI::as_ptr(&future); ``` Since some methods started accepting a pointer, they are marked as unsafe, because they assume that pointer is valid. We will be able to remove unsafes once CassPtr is introduced - the type will give enough guarantees for these methods to be safe. In this PR, there was a question regarding CassFuture::set_callback: Q: Here self_ptr should refer to the same CassFuture as self, right? Can you instead make the receiver of this function an arc? self: &Arc<CassFuture> A: I don't see a good reason behind it. We get a pointer in cass_future_set_callback, then we would need to convert it to Arc (increase a refcount), just to then convert it back to pointer (decrease refcount) and pass the ponter to callback.

muzarski · 2025-03-17T12:18:25Z

v4:

Introduced CMutability property - it simply tells whether the pointer was T* or const T* - as defined in C API. It tells about logical mutability of the pointer.
Rebased on main
Updated cover letter

scylla-rust-wrapper/src/argconv.rs

scylla-rust-wrapper/src/batch.rs

scylla-rust-wrapper/src/binding.rs

scylla-rust-wrapper/src/lib.rs

muzarski · 2025-03-21T14:12:06Z

v4.1: addressed @wprzytula comments

Lorak-mmk

💯 Really great work! I no longer see any big problems, only left some minor comments.

I'm not a fan of the workaround in cass_future_set_callback, but I don't have any good ideas on how to improve it.

Lorak-mmk · 2025-03-28T16:01:48Z

scylla-rust-wrapper/src/argconv.rs

+mod sealed {
+    pub trait Sealed {}
+}
+
+/// A trait representing (Rust) mutability of the pointer.
+///
+/// Pointer can either be [`Const`] or [`Mut`].
+///
+/// ## Const pointers
+/// Const pointers can only be converted to **immutable** Rust referential types.
+/// There is no way to obtain a mutable reference from such pointer.
+///
+/// In some cases, we need to be able to mutate the data behind a shared pointer.
+/// There is an example of such use case - namely [`crate::cass_types::CassDataType`].
+/// argconv API does not provide a way to mutate such pointer - one can only convert the pointer
+/// to [`Arc`] or &. It is the API user's responsibility to implement sound interior mutability
+/// pattern in such case. This is what we currently do - CassDataType wraps CassDataTypeInner
+/// inside an `UnsafeCell` to implement interior mutability for the type.
+/// Other example is [`crate::future::CassFuture`] which uses Mutex.
+///
+/// ## Mut pointers
+/// Mut pointers can be converted to both immutable and mutable Rust referential types.
+pub trait Mutability: sealed::Sealed {}
+
+/// Represents immutable pointer.
+pub struct Const;
+impl sealed::Sealed for Const {}
+impl Mutability for Const {}
+
+/// Represents mutable pointer.
+pub struct Mut;
+impl sealed::Sealed for Mut {}
+impl Mutability for Mut {}


Mutability and CMutability are quite similar names. The names of specific structs are also similar.

WDYT about
Mutability -> Ownership
Const -> Shared
Mut -> Exclusive

I think it better reflects what this property is about, and reflects the imo better way to think about rust references.

I definitely agree with Exclusive and Shared names. Not sure about Ownership. For me, Ownership property is currently represented via lifetimes (Owned vs Borrowed).

OTOH, Mutability name is not perfect either. I'll try to come up with some reasonable name. If I fail to do so, I'll apply your suggestion and rename the trait to Ownership.

I couldn't come up with a better name. Renamed the trait and corresponding struct as you suggested.

scylla-rust-wrapper/src/argconv.rs

This commit introduces a `CassPtr` type, generic over pointer `Properties`. It allows specific pointer-to-reference conversions based on the guarantees provided by the pointer type.

Existing `Ref/Box/Arc`FFIs are adjusted, so they now allow interaction with new pointer type. You can say, that for a user of `argconv` API, new type is opaque. The only way to operate on it is to use the corresponding ffi API.

Disallow implementing two different APIs for specific type.

muzarski · 2025-03-30T21:14:45Z

v4.2: Addressed @Lorak-mmk comments.

wprzytula · 2025-03-31T07:44:23Z

Hmm, why did cassandra tests fail?

wprzytula · 2025-03-31T07:47:26Z

The cover letter needs to be adjusted for the recent Mutability -> Ownership change.

muzarski · 2025-03-31T08:31:01Z

Hmm, why did cassandra tests fail?

Due to the recent change on scylla-ccm's master. I'll be investivating this, but the failure is not related to this PR.

muzarski · 2025-03-31T11:56:52Z

Hmm, why did cassandra tests fail?

Due to the recent change on scylla-ccm's master. I'll be investivating this, but the failure is not related to this PR.

@wprzytula @Lorak-mmk I opened an issue on scylla-ccm: scylladb/scylla-ccm#646. As a temporary fix, should I set the ccm version in CI to the version before the change that introduced the bug?

Lorak-mmk · 2025-03-31T12:06:16Z

Hmm, why did cassandra tests fail?

Due to the recent change on scylla-ccm's master. I'll be investivating this, but the failure is not related to this PR.

@wprzytula @Lorak-mmk I opened an issue on scylla-ccm: scylladb/scylla-ccm#646. As a temporary fix, should I set the ccm version in CI to the version before the change that introduced the bug?

Yes

See the issue linked in the comment.

muzarski · 2025-03-31T12:48:41Z

CI passed - I am merging

muzarski marked this pull request as draft November 26, 2024 15:47

muzarski force-pushed the pointer-kinds branch 2 times, most recently from f433610 to 9dadad8 Compare November 26, 2024 17:35

muzarski self-assigned this Nov 26, 2024

muzarski force-pushed the pointer-kinds branch from 9dadad8 to 7b8bb5a Compare November 29, 2024 12:44

muzarski requested review from Lorak-mmk and wprzytula November 29, 2024 12:47

muzarski marked this pull request as ready for review November 29, 2024 12:47

wprzytula requested changes Dec 1, 2024

View reviewed changes

muzarski force-pushed the pointer-kinds branch from 7b8bb5a to e8d2d93 Compare December 2, 2024 15:05

muzarski force-pushed the pointer-kinds branch 2 times, most recently from 98fbac9 to aa90f5a Compare December 2, 2024 19:30

muzarski requested a review from wprzytula December 2, 2024 19:32

wprzytula reviewed Dec 4, 2024

View reviewed changes

scylla-rust-wrapper/src/argconv.rs Outdated Show resolved Hide resolved

wprzytula requested changes Dec 4, 2024

View reviewed changes

scylla-rust-wrapper/src/argconv.rs Outdated Show resolved Hide resolved

scylla-rust-wrapper/src/argconv.rs Outdated Show resolved Hide resolved

muzarski force-pushed the pointer-kinds branch from aa90f5a to 6c2a061 Compare December 4, 2024 18:00

muzarski requested a review from wprzytula December 4, 2024 18:37

muzarski force-pushed the pointer-kinds branch from 6c2a061 to 3686e0d Compare December 4, 2024 22:39

wprzytula approved these changes Dec 6, 2024

View reviewed changes

scylla-rust-wrapper/src/argconv.rs Outdated Show resolved Hide resolved

scylla-rust-wrapper/src/argconv.rs Outdated Show resolved Hide resolved

scylla-rust-wrapper/src/batch.rs Outdated Show resolved Hide resolved

muzarski force-pushed the pointer-kinds branch from 3686e0d to f341734 Compare December 9, 2024 12:45

muzarski force-pushed the pointer-kinds branch from f341734 to 8dfff0a Compare December 9, 2024 12:53

muzarski force-pushed the pointer-kinds branch 2 times, most recently from 59cfb50 to ce3896f Compare March 14, 2025 12:38

muzarski requested review from Lorak-mmk and wprzytula March 17, 2025 12:15

muzarski marked this pull request as ready for review March 17, 2025 12:16

muzarski mentioned this pull request Mar 18, 2025

deser: lazy CassRow construction #213

Merged

3 tasks

wprzytula requested changes Mar 20, 2025

View reviewed changes

muzarski force-pushed the pointer-kinds branch from ce3896f to 149fe70 Compare March 21, 2025 14:11

muzarski requested a review from wprzytula March 21, 2025 14:12

wprzytula approved these changes Mar 25, 2025

View reviewed changes

Lorak-mmk requested changes Mar 28, 2025

View reviewed changes

muzarski added 3 commits March 30, 2025 23:05

argconv: introduce CassPtr

5e22a9c

This commit introduces a `CassPtr` type, generic over pointer `Properties`. It allows specific pointer-to-reference conversions based on the guarantees provided by the pointer type.

argconv: adjust FFI apis to new pointer type

b98c969

Existing `Ref/Box/Arc`FFIs are adjusted, so they now allow interaction with new pointer type. You can say, that for a user of `argconv` API, new type is opaque. The only way to operate on it is to use the corresponding ffi API.

argconv: make FFI implementation mutually exclusive

4727e75

Disallow implementing two different APIs for specific type.

muzarski force-pushed the pointer-kinds branch from 149fe70 to 4727e75 Compare March 30, 2025 21:10

muzarski requested a review from Lorak-mmk March 30, 2025 21:14

CI: Point ccm to commit before the C* bug was introduced.

d5df8a1

See the issue linked in the comment.

muzarski merged commit 283fd5f into scylladb:master Mar 31, 2025
11 checks passed

muzarski deleted the pointer-kinds branch March 31, 2025 12:48

muzarski mentioned this pull request Apr 14, 2025

Release v0.4.0 #245

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

safety: introduce pointer types and their restrictions #208

safety: introduce pointer types and their restrictions #208

muzarski commented Nov 26, 2024 •

edited

Loading

wprzytula left a comment

muzarski commented Dec 2, 2024

muzarski commented Dec 2, 2024

wprzytula commented Dec 4, 2024

muzarski commented Dec 4, 2024

muzarski commented Dec 4, 2024 •

edited

Loading

wprzytula commented Dec 4, 2024

muzarski commented Dec 4, 2024

muzarski commented Dec 9, 2024

muzarski commented Dec 9, 2024

muzarski commented Mar 17, 2025

muzarski commented Mar 21, 2025

Lorak-mmk left a comment

Lorak-mmk Mar 28, 2025

muzarski Mar 28, 2025

muzarski Mar 30, 2025

muzarski commented Mar 30, 2025

wprzytula commented Mar 31, 2025

wprzytula commented Mar 31, 2025

muzarski commented Mar 31, 2025

muzarski commented Mar 31, 2025 •

edited

Loading

Lorak-mmk commented Mar 31, 2025

muzarski commented Mar 31, 2025

safety: introduce pointer types and their restrictions #208

safety: introduce pointer types and their restrictions #208

Conversation

muzarski commented Nov 26, 2024 • edited Loading

Lifetime

Properties

Pre-review checklist

wprzytula left a comment

Choose a reason for hiding this comment

muzarski commented Dec 2, 2024

muzarski commented Dec 2, 2024

wprzytula commented Dec 4, 2024

muzarski commented Dec 4, 2024

muzarski commented Dec 4, 2024 • edited Loading

wprzytula commented Dec 4, 2024

muzarski commented Dec 4, 2024

muzarski commented Dec 9, 2024

muzarski commented Dec 9, 2024

muzarski commented Mar 17, 2025

muzarski commented Mar 21, 2025

Lorak-mmk left a comment

Choose a reason for hiding this comment

Lorak-mmk Mar 28, 2025

Choose a reason for hiding this comment

muzarski Mar 28, 2025

Choose a reason for hiding this comment

muzarski Mar 30, 2025

Choose a reason for hiding this comment

muzarski commented Mar 30, 2025

wprzytula commented Mar 31, 2025

wprzytula commented Mar 31, 2025

muzarski commented Mar 31, 2025

muzarski commented Mar 31, 2025 • edited Loading

Lorak-mmk commented Mar 31, 2025

muzarski commented Mar 31, 2025

muzarski commented Nov 26, 2024 •

edited

Loading

muzarski commented Dec 4, 2024 •

edited

Loading

muzarski commented Mar 31, 2025 •

edited

Loading