|
1 | 1 | # Data interchange mechanisms
|
| 2 | + |
| 3 | +This section discusses the mechanism to convert one type of array into another. |
| 4 | +As discussed in the :ref:`assumptions-dependencies <Assumptions>` section, |
| 5 | +_functions_ provided by an array library are not expected to operate on |
| 6 | +_array types_ implemented by another library. Instead, the array can be |
| 7 | +converted to a "native" array type. |
| 8 | + |
| 9 | +The interchange mechanism must offer the following: |
| 10 | + |
| 11 | +1. Data access via a protocol that describes the memory layout of the array |
| 12 | + in an implementation-independent manner. |
| 13 | + _Rationale: any number of libraries must be able to exchange data, and no |
| 14 | + particular package must be needed to do so._ |
| 15 | +2. Support for all dtypes in this API standard (see :ref:`data-types`). |
| 16 | +3. Device support. It must be possible to determine on what device the array |
| 17 | + that is to be converted lives. |
| 18 | + _Rationale: there are CPU-only, GPU-only, and multi-device array types; |
| 19 | + it's best to support these with a single protocol (with separate |
| 20 | + per-device protocols it's hard to figure out unambiguous rules for which |
| 21 | + protocol gets used, and the situation will get more complex over time |
| 22 | + as TPU's and other accelerators become more widely available)._ |
| 23 | +4. Moving data from one device to another must raise by default, and be |
| 24 | + explicitly enabled by the user. |
| 25 | + _Rationale: data transfer is typically very expensive, and hence must not |
| 26 | + happen silently._ |
| 27 | +5. Zero-copy semantics where possible, making a copy only if needed (e.g. |
| 28 | + when data is not contiguous in memory). |
| 29 | + _Rationale: performance._ |
| 30 | +6. A stable C ABI. |
| 31 | + _Rationale: all prominent existing array libraries are implemented in |
| 32 | + C/C++, and are released independently from each other. Hence a stable C |
| 33 | + ABI is required for packages to work well together._ |
| 34 | + |
| 35 | +The best candidate for this protocol is DLPack. See the |
| 36 | +[RFC to adopt DLPack](https://github.com/data-apis/consortium-feedback/issues/1) |
| 37 | +for details. |
| 38 | + |
| 39 | +TODO: design an appropriate Python API for DLPACK (`to_dlpack` followed by `from_dlpack` is a little clunky, we'd like it to work more like the buffer protocol does on CPU, with a single constructor function). |
| 40 | + |
| 41 | +TODO: specify the expected behaviour with copy/view/move/shared-memory semantics in detail. |
0 commit comments