liballoc: rewrite on motivation and alloc crate requirements

* Largely rewrite the motivation section to concentrate on the impact
  on the (crates.io) library ecosystem.

* Mention the assumptions/requirements made by the alloc crate:
  allocator and OOM handler, but not randomness.

* Add stabilization of `alloc::prelude`

* Discuss alternatives for HashMap in no_std

Author: SimonSapin

text/0000-liballoc.md

@@ -1,16 +1,16 @@
 - Feature Name: `liballoc`
 - Start Date: 2018-06-14
-- RFC PR:
+- RFC PR: [rust-lang/rfcs#2480](https://github.com/rust-lang/rfcs/pull/2480)
 - Rust Issue: [rust-lang/rust#27783](https://github.com/rust-lang/rust/issues/27783)
 
 # Summary
 [summary]: #summary
 
-Stabilize the `alloc` crate, with a module structure matching `std`.
+Stabilize the `alloc` crate.
 
 This crate provides the subset of the standard library’s functionality that requires
-a global allocator (unlike the `core` crate) but not other operating system
-capabilities (unlike the `std` crate).
+a global allocator (unlike the `core` crate) and an allocation error handler,
+but not other operating system capabilities (unlike the `std` crate).
 
 
 # Motivation
@@ -23,34 +23,49 @@ micro-controllers that don’t have an operating system at all, kernel-space cod
 The `#![no_std]` attribute allows a crate to not link to `std` implicitly,
 using `core` instead with only the subset of functionality that doesn’t have a runtime dependency.
 
-## Use case 1: pushing `no_std` programs toward stable Rust
+## `no_std` with an allocator
 
-Programs (or `staticlib`s) that do not link `std` at all may still want to use `Vec<T>`
-or other functionality that requires a memory allocator.
-Blockers for doing so on stable Rust are diminishing:
+The `core` crate does not assume even the presence of heap memory,
+and so it excludes standard library types like `Vec<T>`.
+However some environments do have a heap memory allocator
+(possibly as `malloc` and `free` C functions),
+even if they don’t have files or threads
+or something that could be called an operating system or kernel.
+Or one could be defined [in a Rust library][wee-alloc]
+ultimately backed by fixed-size static byte array.
 
-* [The `#[global_allocator]` attribute][global_allocator] to specify an allocator
-  and remove the need for an operating-system-provided one is stable since Rust 1.28.
-* [Tracking issue #51540] discusses how to stabilize a way to specify
-  the handling of OOM (allocation failure) conditions when `std` is not available,
-  instead of the unstable `oom` lang item.
+An intermediate subset of the standard library smaller than “all of `std`”
+but larger than “only `core`” can serve such environments.
 
-With this, the only allocation-related blocker is being able to import `Vec` in the first place.
-This RFC proposes stabilizing the current unstable way to do it: `extern crate alloc;`
+[wee-alloc]: https://github.com/rustwasm/wee_alloc
 
-[global_allocator]: https://doc.rust-lang.org/nightly/std/alloc/#the-global_allocator-attribute
-[Tracking issue #51540]: https://github.com/rust-lang/rust/issues/51540
+## Libraries
+
+In 2018 there is a [coordinated push]
+toward making `no_std` application compatible with Stable Rust.
+As of this writing not all of that work is completed yet.
+For example, [`#[panic_implementation]`][panic-impl] is required for `no_std` but still unstable.
+So it may seem that this RFC does not unlock anything new,
+as `no_std` application still need to be on Nigthly anyway.
 
-## Use case 2: making stable libraries `no_std`
+[coordinated push]: https://github.com/rust-lang-nursery/embedded-wg/issues/42
+[panic-impl]: https://github.com/rust-lang/rust/issues/44489
 
-Even if a `no_std` program might still require other features that are still unstable,
-it is very common to use libraries from crates.io that have other users.
-Such a library might support stable Rust and use `Vec<T>`
-(or something else that requires a memory allocator)
-but not other operating-sytem functionality.
+The immediate impact can be found in the library ecosystem.
+Many general-purpose libraries today are compatible with Stable Rust
+and also have potential users who ask for them to be compatible with `no_std` environments.
 
-Today, making such a library possible to use without `std` without breaking stable users
-requires a compile-time flag:
+For a library that is fundamentally about using for example TCP sockets or threads,
+this may not be possible.
+
+For a library that happens to only use parts of `std` that are also in `core`
+(and are willing to commit to keep doing so), this is relatively easy:
+add `#![no_std]` to the crate, and change `std::` in paths to `core::`.
+
+And here again, there is the intermediate case of a library that needs `Vec<T>`
+or something else that involves heap memory, but not other parts of `std` that are not in `core`.
+Today, in order to not lose compatibility with Stable,
+such a library needs to make compatibility with `no_std` an opt-in feature flag:
 
 ```rust
 #![no_std]
@@ -61,7 +76,11 @@ requires a compile-time flag:
 use alloc::vec::Vec;
 ```
 
-With this RFC, this can be simplified to:
+But publishing a library that uses unstable features, even optionally,
+comes with the expectation that it will be promptly updated whenever those features change.
+Some maintainers are not willing to commit to this.
+
+With this RFC, the library’s code can be simplified to:
 
 ```rust
 #![no_std]
@@ -71,9 +90,15 @@ extern crate alloc;
 use alloc::vec::Vec;
 ```
 
+… and perhaps more importantly,
+maintainers can rely on the stability promise made by the Rust project.
+
+
 # Guide-level explanation
 [guide-level-explanation]: #guide-level-explanation
 
+## For libraries
+
 When using `#![no_std]` in a crate, that crate does not implicitly depend on `std`
 but depends on `core` instead. For example:
 
@@ -86,41 +111,105 @@ APIs that require a memory allocator are not available in `core`.
 In order to use them, `no_std` Rust code must explicitly depend on the `alloc` crate:
 
 ```rust
-extern crate alloc;
+#[macro_use] extern crate alloc;
 
 use core::cell::RefCell;
 use alloc::rc::Rc;
 ```
 
+Note: `#[macro_use]` imports the [`vec!`] and [`format!`] macros.
+
+[`vec!`]: https://doc.rust-lang.org/alloc/macro.vec.html
+[`format!`]: https://doc.rust-lang.org/alloc/macro.format.html
+
 Like `std` and `core`, this dependency does not need to be declared in `Cargo.toml`
 since `alloc` is part of the standard library and distributed with Rust.
 
-The prelude (set of items that are implicitly in scope) for `#![no_std]` crates
+The implicit prelude (set of items that are automatically in scope) for `#![no_std]` crates
 does not assume the presence of the `alloc` crate, unlike the default prelude.
-So in such crates, items from `alloc` always need to be imported explicitly.
+So such crates may need to import either that prelude or specific items explicitly.
 For example:
 
 ```rust
+use alloc::prelude::*;
+
+// Or
+
+use alloc::string::ToString;
 use alloc::vec::Vec;
 ```
 
+## For programs¹
+
+[¹] … and other roots of a dependency graph, such as `staticlib`s.
+
+Compared to `core`, the `alloc` crate makes two additional requirements:
+
+* A global heap memory allocator.
+
+* An allocation error handler (that is not allowed to return).
+  This is called for example by `Vec::push`, whose own API is infallible,
+  when the allocator fails to allocate memory.
+
+`std` provides both of these. So as long as it is present in the dependency graph,
+nothing else is required even if some crates of the graph use `alloc` without `std`.
+
+If `std` is not present they need to be defined explicitly,
+somewhere in the dependency graph (not necessarily in the root crate).
+
+* [The `#[global_allocator]` attribute][global_allocator], on a `static` item
+  of a type that implements the `GlobalAlloc` trait,
+  defines the global allocator. It is stable in Rust 1.28.
+
+* [Tracking issue #51540] propose the `#[alloc_error_handler]` attribute
+  for a function with signature `fn foo(_: Layout) -> !`.
+  As of this writing this attribute is implemented but unstable.
+
+[global_allocator]: https://doc.rust-lang.org/nightly/std/alloc/#the-global_allocator-attribute
+[Tracking issue #51540]: https://github.com/rust-lang/rust/issues/51540
+
 
 # Reference-level explanation
 [reference-level-explanation]: #reference-level-explanation
 
 The `alloc` crate already exists (marked unstable),
 and every public API in it is already available in `std`.
 
-[PR #51569] moves them around so that the module structure matches that of `std`,
-and the public APIs become a subset:
-any path that starts with `alloc::` should still be valid and point to the same item
+Except for the `alloc::prelude` module, since [PR #51569] the module structure is a subset
+of that of `std`: every path that starts with `alloc::` is still valid and point to the same item
 after replacing that prefix with `std::` (assuming both crates are available).
 
-All that remains is stabilizing the `alloc` crate itself and tweaking its doc-comment.
-(In particular, removing the “not intended for general usage” sentence
-and mention `no_std` instead.)
-Since it is the only remaining unstable crate tracked by [tracking issue #27783],
-that issue can be closed after this RFC is implemented.
+The concrete changes proposed by this RFC are:
+
+* Stabilize `extern crate alloc;`
+  (that is, change `#![unstable]` to `#![stable]` near the top of `src/liballoc/lib.rs`).
+
+* Stabilize the `alloc::prelude` module and its contents
+  (which is only re-exports of items that are themselves already stable).
+
+* Stabilize the fact that the crate makes no more and no less than
+  the two requirements/assumptions of a global allocator and an allocation error handler
+  being provided for it, as described above.
+
+  The exact mechanism for [providing the allocation error handler][Tracking issue #51540]
+  is not stabilized by this RFC.
+
+  In particular, this RFC proposes that the presence of a source of randomness
+  is *not* a requirement that the `alloc` crate can make.
+  This is contrary to what [PR #51846] proposed,
+  and means that `std::collections::hash_map::RandomState` cannot be moved into `alloc`.
+
+[Tracking issue #27783] tracks “the `std` facade”:
+crates whose contents are re-exported in `std` but also exist separately.
+Other such crates have already been moved, merged, or stabilized,
+such that `alloc` is the only remaining unstable one.
+Therefore #27783 can serve as the tracking issue for this RFC
+and can be closed once it is implemented.
+
+[PR #51569]: https://github.com/rust-lang/rust/pull/51569
+[PR #51846]: https://github.com/rust-lang/rust/pull/51846
+[Tracking issue #27783]: https://github.com/rust-lang/rust/issues/27783
+
 
 The structure of the standard library is therefore:
 
@@ -134,8 +223,6 @@ The structure of the standard library is therefore:
 * `proc-macro`: depends on parts of the compiler, typically only used at build-time
   (in procedural macro crates or Cargo build scripts).
 
-[PR #51569]: https://github.com/rust-lang/rust/pull/51569
-[tracking issue #27783]: https://github.com/rust-lang/rust/issues/27783
 
 
 # Drawbacks
@@ -148,22 +235,24 @@ for the standard library, given infinite time and resources.
 
 In particular, could we have a single crate with some mechanism for selectively disabling
 or enabling some of the crate’s components, depending on which runtime dependencies
-are available in targetted environments?
+are available in targeted environments?
 In that world, the `no_std` attribute and standard library crates other than `std`
-would be unecessary.
+would be unnecessary.
 
 By stabilizing the `alloc` crate, we commit to having it − and its public API − exist “forever”.
 
 
 # Rationale and alternatives
 [alternatives]: #alternatives
 
+## Single-crate standard library
+
 The `core` and the `no_std` attribute are already stable,
 so in a sense it’s already too late for the “pure” version of the vision described above
 where `std` really is the only standard library crate that exists.
 
 It may still be [desirable] to regroup the standard library into one crate,
-and it is proably still possible.
+and it is probably still possible.
 The `core` crate could be replaced with a set of `pub use` reexport
 to maintained compatibility with existing users.
 Whatever the eventual status is for `core` is,
@@ -174,10 +263,34 @@ While we want to leave the possibility open for it,
 at the time of this writing there are no concrete plans
 for implementing such a standard library crates unification any time soon.
 So the only alternative to this RFC seems to be
-leaving heap allocation for `no_std` in unstable limbo for the forseeable future.
+leaving heap allocation for `no_std` in unstable limbo for the foreseeable future.
 
 [desirable]: https://aturon.github.io/2018/02/06/portability-vision/#the-vision
 
+## Require randomness
+
+[PR #51569] proposed adding a source of randomness to the other requirements
+made by the `alloc` crate.
+This would allow moving `std::collections::hash_map::RandomState`,
+and therefore `HashMap` (which has `RandomState` as a default type parameter),
+into `alloc`.
+
+This RFC chooses not to do this because it would make it difficult to use for example `Vec<T>`
+in environments where a source of randomness is not easily available.
+
+I hope that the language will eventually make it possible to have `HashMap` in `alloc`
+without a default hasher type parameter, and have the same type in `std` with its current default.
+
+Although I am not necessarily in favor
+of continuing the increase of the number of crates in the standard library,
+another solution for `HashMap` in `no_std` might be another intermediate crate
+that depends on `alloc` and adds the randomness source requirement.
+
+Additionally, with this RFC it should be possible to make https://crates.io/crates/hashmap_core
+compatible with Stable Rust.
+The downside of that crate is that although based on a copy of the same code,
+it is a different type incompatible in the type system with `std::collections::HashMap`.
+
 
 # Prior art
 [prior-art]: #prior-art
@@ -202,3 +315,9 @@ It does provide a memory allocator through `malloc` and related functions, uncon
   What really characterizes it is the assumption that a global allocator is available.
   The name `global_alloc` was proposed.
   (Although the crate doesn’t only contain the global allocator itself.)
+
+* Should the `alloc::prelude` module be moved to `alloc::prelude::v1`?
+  This would make the `alloc` module structure a subset of `std` without exception.
+  However, since this prelude is not inserted automatically,
+  it is less likely that we’ll ever have a second version of it.
+  In that sense it is closer to `std::io::prelude` than `std::prelude::v1`.