pwrite64() is missing for uclibc targets

[drmikehenry]

When I upgraded to a recent toolchain (2021-03-25) and updated some crates to today's latest as a result, I started getting errors about missing libc::pwrite64() for my uclibc-based target. I haven't tracked it all the way back to see when the error first started happening, but I think the issue is that the standard library source requires pwrite64() for non-Android Linux:
https://github.com/rust-lang/rust/blob/48691ea6e639640f110b43e33d4aba1f07e7415c/library/std/src/sys/unix/fd.rs#L167-L183

#[cfg(target_os = "linux")]
use libc::pwrite64;
cvt(pwrite64(fd, buf, count, offset))

Whereas pwrite64() is disallowed for uclibc in the libc crate:

libc/src/unix/linux_like/mod.rs

Lines 1671 to 1685 in f913fc5

	cfg_if! {
	if #[cfg(not(target_env = "uclibc"))] {
	extern "C" {
	pub fn preadv64(
	fd: ::c_int,
	iov: *const ::iovec,
	iovcnt: ::c_int,
	offset: ::off64_t,
	) -> ::ssize_t;
	pub fn pwrite64(
	fd: ::c_int,
	buf: *const ::c_void,
	count: ::size_t,
	offset: off64_t,
	) -> ::ssize_t;

However, the companion function pread64() is permitted for uclibc:

libc/src/unix/linux_like/mod.rs

Lines 1543 to 1548 in f913fc5

	pub fn pread64(
	fd: ::c_int,
	buf: *mut ::c_void,
	count: ::size_t,
	offset: off64_t,
	) -> ::ssize_t;

The actual pwrite64() function is available on my uclibc target. Considering that libc contains a SYS_pwrite64 value for uclibc targets, I suspect this is just an oversight in the libc crate, and that pwrite64() was not intended to be caveated by cfg(not(target_env = "uclibc")). Unless there is some other reason for disallowing pwrite64() on uclibc-based Linux targets, could this function be promoted to work for uclibc variants?

[JohnTitor]

Thanks for reporting, yeah, I think there's no reason to disable pwrite64 on uclibc. Submitted #2131 as a fix.

[drmikehenry]

I tested with a custom copy of std using this new libc and it works great. Thanks for the lightning-fast fix.

[JohnTitor]

Anytime :)

[zvirja]

@drmikehenry Could you please share an example of how you consumed this fix? I tried playing with xargo and the patch feature, but no luck 😟

I created a PR to rust to update the libc dep, but want to consume it in the meanwhile.

Thanks

[drmikehenry]

@zvirja I'd be happy to post the steps I followed once I retrieve them from the office early next week. Mostly I just followed somebody's tutorial on re-compiling the Rust standard library, but there are indeed a number of steps involved and without notes I'd be lost :-)

[drmikehenry]

@zvirja I'm having trouble replicating this change with my home setup, but it appears that your pull request has gone through already, so perhaps you can use a new nightly toolchain to get the libc updates.

[drmikehenry]

@zvirja You probably won't need this if you use the very latest Rust sources; commit e41f378f825488a537b024fc3ed599d9c12fda96 bumped the libc version dependency to 0.2.93. But in case you want to use an older toolchain with the new libc, here are the steps I used to test libc-0.2.92.

First, install a nightly toolchain and make it the default:

rustup install nightly-2021-03-01
rustup default nightly-2021-03-01

Clone Rust sources and enter rust directory:

mkdir -p ~/projects/custom-std
cd ~/projects/custom-std
git clone https://github.com/rust-lang/rust
cd rust

Optionally, checkout a branch or commit hash as the desired starting point, perhaps using the hash from the active compiler:

rustc -vV | grep commit-hash

with output:

commit-hash: e37a13cc3594004663738bd18d8100e6db9666cf

Checkout this commit:

git checkout e37a13cc3594004663738bd18d8100e6db9666cf

Initialize and update the submodules:

git submodule init
git submodule update

For use with Cargo's build-std feature, the Rust sources need to live in the expected location below ~/.rustup. Using this example toolchain (nightly-2021-03-01), enter the toolchain's rustlib directory, create src (if necessary), and enter src:

cd ~/.rustup/toolchains/nightly-2021-03-01-x86_64-unknown-linux-gnu/lib/rustlib
mkdir -p src
cd src

If there's already a rust/ directory, it must be deleted or renamed out of the way:

mv rust rust.orig

Create a symlink to the cloned Rust sources:

ln -s ~/projects/custom-std/rust rust

Create an example project to test:

cd ~/projects
cargo new hello-custom-std
cd hello-custom-std

Add the following to Cargo.toml:

[profile.release]
panic = "abort"

At this point, Cargo's build-std feature should work:

cargo build \
  -Z build-std=panic_abort,std \
  -Z build-std-features=panic_immediate_abort \
  --target x86_64-unknown-linux-gnu \
  --release

Building with Xargo should work as well, if a Xargo.toml is created, e.g.:

[dependencies]
std = {default-features=false, features=["panic_immediate_abort"]}

Then install Xargo and build:

cargo install xargo
xargo build \
  --target x86_64-unknown-linux-gnu \
  --release

With Xargo, it's not necessary to place the sources below ~/.rustup; instead, an environment variable can be set to point directly to the cloned sources, e.g.:

export XARGO_RUST_SRC=$HOME/projects/custom-std/rust/library

You might need to remove ~/.xargo to force a rebuild of everything.

To use libc-0.2.92, clone the libc source and checkout the 0.2.92 branch:

cd ~/projects
git clone https://github.com/rust-lang/libc
cd libc
git checkout 0.2.92

Edit the Rust standard library's Cargo.toml file to use this new version; the changes are shown via this command:

cd ~/projects/custom-std/rust
git diff library/std/Cargo.toml

with sample output:

diff --git a/library/std/Cargo.toml b/library/std/Cargo.toml
index 275fcc4c292..8a82c66b250 100644
--- a/library/std/Cargo.toml
+++ b/library/std/Cargo.toml
@@ -16,7 +16,7 @@ cfg-if = { version = "0.1.8", features = ['rustc-dep-of-std'] }
 panic_unwind = { path = "../panic_unwind", optional = true }
 panic_abort = { path = "../panic_abort" }
 core = { path = "../core" }
-libc = { version = "0.2.85", default-features = false, features = ['rustc-dep-of-std'] }
+libc = { version = "=0.2.92", default-features = false, features = ['rustc-dep-of-std'], path = "../../../../libc" }
 compiler_builtins = { version = "0.1.39" }
 profiler_builtins = { path = "../profiler_builtins", optional = true }
 unwind = { path = "../unwind" }

The change involves requesting version = "=0.2.92" (matching the version in libc/Cargo.toml) and setting the path to point to the libc working copy (note that relative paths are taken relative to the directory containing library/std/Cargo.toml).

It may be necessary to clean the workspace and remove the ~/.xargo tree before rebuilding::

rm -rf ~/.xargo
cargo clean
xargo build \
  --target x86_64-unknown-linux-gnu \
  --release

The indication that the local copies are being used is in these lines of output:

   Compiling libc v0.2.92 (/home/USERNAME/projects/libc)
   Compiling std v0.0.0 (/home/USERNAME/projects/custom-std/rust/library/std)

The reason I couldn't replicate this earlier is that I'd been using version = "0.2.92" to request version 0.2.92. I'd mistakenly thought this would request the version exactly, but it turns out that this is equivalent to a "caret requirement" "^0.2.92", which asks Cargo to use the newest version of the form 0.2.x for x >= 92. The day I initially tried this, the newest version on crates.io was 0.2.92, so that's the version Cargo selected. Since I'd placed version 0.2.92 at the path location, Cargo used my local working copy. But when I later tried to replicate this, libc-0.2.93 had been published. Cargo selected 0.2.93 as the best version, then skipped over my local working copy of libc because it contained the wrong version (0.2.92).