From 556509c5d4f0e43cf4b472da03d0291d0d40870e Mon Sep 17 00:00:00 2001 From: Colin McDonnell Date: Wed, 22 Feb 2023 19:36:21 -0800 Subject: [PATCH 1/5] Add documentation --- docs/api/console.md | 38 ++ docs/api/dns.md | 41 ++ docs/api/ffi.md | 516 +++++++++++++++++++++++++ docs/api/file-io.md | 256 +++++++++++++ docs/api/file-system-router.md | 111 ++++++ docs/api/file.md | 19 + docs/api/globals.md | 381 ++++++++++++++++++ docs/api/html-rewriter.md | 31 ++ docs/api/http.md | 257 +++++++++++++ docs/api/node-api.md | 16 + docs/api/spawn.md | 336 ++++++++++++++++ docs/api/sqlite.md | 411 ++++++++++++++++++++ docs/api/tcp.md | 198 ++++++++++ docs/api/test.md | 1 + docs/api/transpiler.md | 275 +++++++++++++ docs/api/utils.md | 120 ++++++ docs/api/websockets.md | 471 +++++++++++++++++++++++ docs/benchmarks.md | 121 ++++++ docs/bun-flavored-toml.md | 2 - docs/cli/bun-completions.md | 3 + docs/cli/bun-dev.md | 21 + docs/cli/bun-init.md | 20 + docs/cli/bun-install.md | 257 +++++++++++++ docs/cli/bun-upgrade.md | 39 ++ docs/cli/bundler.md | 116 ++++++ docs/cli/create.md | 232 +++++++++++ docs/cli/install.md | 344 +++++++++++++++++ docs/cli/run.md | 156 ++++++++ docs/cli/test.md | 224 +++++++++++ docs/dev/bundev.md | 11 + docs/dev/cra.md | 31 ++ docs/dev/css.md | 77 ++++ docs/dev/discord.md | 26 ++ docs/dev/frameworks.md | 151 ++++++++ docs/dev/nextjs.md | 33 ++ docs/index.md | 64 ++++ docs/installation.md | 111 ++++++ docs/nav.ts | 94 +++++ docs/project/configuration.md | 182 +++++++++ docs/project/developing.md | 227 +++++++++++ docs/project/licensing.md | 207 ++++++++++ docs/project/profiling.md | 193 ++++++++++ docs/project/roadmap.md | 87 +++++ docs/quickstart.md | 117 ++++++ docs/runtime/hot.md | 93 +++++ docs/runtime/index.md | 286 ++++++++++++++ docs/runtime/loaders.md | 28 ++ docs/runtime/modules.md | 259 +++++++++++++ docs/runtime/nodejs.md | 682 +++++++++++++++++++++++++++++++++ docs/runtime/plugins.md | 248 ++++++++++++ docs/runtime/streams.md | 5 - docs/runtime/web-apis.md | 300 +++++++++++++++ docs/troubleshooting.md | 59 +++ docs/typescript.md | 87 +++++ docs/upgrading-webkit.md | 12 +- 55 files changed, 8669 insertions(+), 14 deletions(-) create mode 100644 docs/api/console.md create mode 100644 docs/api/dns.md create mode 100644 docs/api/ffi.md create mode 100644 docs/api/file-io.md create mode 100644 docs/api/file-system-router.md create mode 100644 docs/api/file.md create mode 100644 docs/api/globals.md create mode 100644 docs/api/html-rewriter.md create mode 100644 docs/api/http.md create mode 100644 docs/api/node-api.md create mode 100644 docs/api/spawn.md create mode 100644 docs/api/sqlite.md create mode 100644 docs/api/tcp.md create mode 100644 docs/api/test.md create mode 100644 docs/api/transpiler.md create mode 100644 docs/api/utils.md create mode 100644 docs/api/websockets.md create mode 100644 docs/benchmarks.md create mode 100644 docs/cli/bun-completions.md create mode 100644 docs/cli/bun-dev.md create mode 100644 docs/cli/bun-init.md create mode 100644 docs/cli/bun-install.md create mode 100644 docs/cli/bun-upgrade.md create mode 100644 docs/cli/bundler.md create mode 100644 docs/cli/create.md create mode 100644 docs/cli/install.md create mode 100644 docs/cli/run.md create mode 100644 docs/cli/test.md create mode 100644 docs/dev/bundev.md create mode 100644 docs/dev/cra.md create mode 100644 docs/dev/css.md create mode 100644 docs/dev/discord.md create mode 100644 docs/dev/frameworks.md create mode 100644 docs/dev/nextjs.md create mode 100644 docs/index.md create mode 100644 docs/installation.md create mode 100644 docs/nav.ts create mode 100644 docs/project/configuration.md create mode 100644 docs/project/developing.md create mode 100644 docs/project/licensing.md create mode 100644 docs/project/profiling.md create mode 100644 docs/project/roadmap.md create mode 100644 docs/quickstart.md create mode 100644 docs/runtime/hot.md create mode 100644 docs/runtime/index.md create mode 100644 docs/runtime/loaders.md create mode 100644 docs/runtime/modules.md create mode 100644 docs/runtime/nodejs.md create mode 100644 docs/runtime/plugins.md delete mode 100644 docs/runtime/streams.md create mode 100644 docs/runtime/web-apis.md create mode 100644 docs/troubleshooting.md create mode 100644 docs/typescript.md diff --git a/docs/api/console.md b/docs/api/console.md new file mode 100644 index 00000000000000..49b8a1b79722e5 --- /dev/null +++ b/docs/api/console.md @@ -0,0 +1,38 @@ +{% callout %} +**Note** — Bun provides a browser- and Node.js-compatible [console](https://developer.mozilla.org/en-US/docs/Web/API/console) global. This page only documents Bun-native APIs. +{% /callout %} + +In Bun, the `console` object can be used as an `AsyncIterable` to sequentially read lines from `process.stdin`. + +```ts +for await (const line of console) { + console.log(line); +} +``` + +This is useful for implementing interactive programs, like the following addition calculator. + +```ts#adder.ts +console.log(`Let's add some numbers!`); +console.write(`Count: 0\n> `); + +let count = 0; +for await (const line of console) { + count += Number(line); + console.write(`Count: ${count}\n> `); +} +``` + +To run the file: + +```bash +$ bun adder.ts +Let's add some numbers! +Count: 0 +> 5 +Count: 5 +> 5 +Count: 10 +> 5 +Count: 15 +``` diff --git a/docs/api/dns.md b/docs/api/dns.md new file mode 100644 index 00000000000000..d3d06527c4475f --- /dev/null +++ b/docs/api/dns.md @@ -0,0 +1,41 @@ +Bun implements the `node:dns` module. + +```ts +import * as dns from "node:dns"; + +const addrs = await dns.promises.resolve4("bun.sh", { ttl: true }); +console.log(addrs); +// => [{ address: "172.67.161.226", family: 4, ttl: 0 }, ...] +``` + + diff --git a/docs/api/ffi.md b/docs/api/ffi.md new file mode 100644 index 00000000000000..13b50f9c7fbec4 --- /dev/null +++ b/docs/api/ffi.md @@ -0,0 +1,516 @@ +Use the built-in `bun:ffi` module to efficiently call native libraries from JavaScript. It works with languages that support the C ABI (Zig, Rust, C/C++, C#, Nim, Kotlin, etc). + +To print the version number of `sqlite3`: + +```ts +import { dlopen, FFIType, suffix } from "bun:ffi"; + +// `suffix` is either "dylib", "so", or "dll" depending on the platform +// you don't have to use "suffix", it's just there for convenience +const path = `libsqlite3.${suffix}`; + +const { + symbols: { + sqlite3_libversion, // the function to call + }, +} = dlopen( + path, // a library name or file path + { + sqlite3_libversion: { + // no arguments, returns a string + args: [], + returns: FFIType.cstring, + }, + }, +); + +console.log(`SQLite 3 version: ${sqlite3_libversion()}`); +``` + +## Performance + +According to [our benchmark](https://github.com/oven-sh/bun/tree/main/bench/ffi), `bun:ffi` is roughly 2-6x faster than Node.js FFI via `Node-API`. + +{% image src="/images/ffi.png" height="400" /%} + +Bun generates & just-in-time compiles C bindings that efficiently convert values between JavaScript types and native types. To compile C, Bun embeds [TinyCC](https://github.com/TinyCC/tinycc), a small and fast C compiler. + +## Usage + +### Zig + +```zig +// add.zig +pub export fn add(a: i32, b: i32) i32 { + return a + b; +} +``` + +To compile: + +```bash +$ zig build-lib add.zig -dynamic -OReleaseFast +``` + +Pass a path to the shared library and a map of symbols to import into `dlopen`: + +```ts +import { dlopen, FFIType, suffix } from "bun:ffi"; + +const path = `libadd.${suffix}`; + +const lib = dlopen(path, { + add: { + args: [FFIType.i32, FFIType.i32], + returns: FFIType.i32, + }, +}); + +lib.symbols.add(1, 2); +``` + +### Rust + +```rust +// add.rs +#[no_mangle] +pub extern "C" fn add(a: isize, b: isize) -> isize { + a + b +} +``` + +To compile: + +```bash +$ rustc --crate-type cdylib add.rs +``` + +## FFI types + +The following `FFIType` values are supported. + +| `FFIType` | C Type | Aliases | +| --------- | -------------- | --------------------------- | +| cstring | `char*` | | +| function | `(void*)(*)()` | `fn`, `callback` | +| ptr | `void*` | `pointer`, `void*`, `char*` | +| i8 | `int8_t` | `int8_t` | +| i16 | `int16_t` | `int16_t` | +| i32 | `int32_t` | `int32_t`, `int` | +| i64 | `int64_t` | `int64_t` | +| i64_fast | `int64_t` | | +| u8 | `uint8_t` | `uint8_t` | +| u16 | `uint16_t` | `uint16_t` | +| u32 | `uint32_t` | `uint32_t` | +| u64 | `uint64_t` | `uint64_t` | +| u64_fast | `uint64_t` | | +| f32 | `float` | `float` | +| f64 | `double` | `double` | +| bool | `bool` | | +| char | `char` | | + +## Strings + +JavaScript strings and C-like strings are different, and that complicates using strings with native libraries. + +{% details summary="How are JavaScript strings and C strings different?" %} +JavaScript strings: + +- UTF16 (2 bytes per letter) or potentially latin1, depending on the JavaScript engine & what characters are used +- `length` stored separately +- Immutable + +C strings: + +- UTF8 (1 byte per letter), usually +- The length is not stored. Instead, the string is null-terminated which means the length is the index of the first `\0` it finds +- Mutable + +{% /details %} + +To solve this, `bun:ffi` exports `CString` which extends JavaScript's built-in `String` to support null-terminated strings and add a few extras: + +```ts +class CString extends String { + /** + * Given a `ptr`, this will automatically search for the closing `\0` character and transcode from UTF-8 to UTF-16 if necessary. + */ + constructor(ptr: number, byteOffset?: number, byteLength?: number): string; + + /** + * The ptr to the C string + * + * This `CString` instance is a clone of the string, so it + * is safe to continue using this instance after the `ptr` has been + * freed. + */ + ptr: number; + byteOffset?: number; + byteLength?: number; +} +``` + +To convert from a null-terminated string pointer to a JavaScript string: + +```ts +const myString = new CString(ptr); +``` + +To convert from a pointer with a known length to a JavaScript string: + +```ts +const myString = new CString(ptr, 0, byteLength); +``` + +The `new CString()` constructor clones the C string, so it is safe to continue using `myString` after `ptr` has been freed. + +```ts +my_library_free(myString.ptr); + +// this is safe because myString is a clone +console.log(myString); +``` + +When used in `returns`, `FFIType.cstring` coerces the pointer to a JavaScript `string`. When used in `args`, `FFIType.cstring` is identical to `ptr`. + +## Function pointers + +{% callout %} + +**Note** — Async functions are not yet supported. + +{% /callout %} + +To call a function pointer from JavaScript, use `CFunction`. This is useful if using Node-API (napi) with Bun, and you've already loaded some symbols. + +```ts +import { CFunction } from "bun:ffi"; + +let myNativeLibraryGetVersion = /* somehow, you got this pointer */ + +const getVersion = new CFunction({ + returns: "cstring", + args: [], + ptr: myNativeLibraryGetVersion, +}); +getVersion(); +``` + +If you have multiple function pointers, you can define them all at once with `linkSymbols`: + +```ts +import { linkSymbols } from "bun:ffi"; + +// getVersionPtrs defined elsewhere +const [majorPtr, minorPtr, patchPtr] = getVersionPtrs(); + +const lib = linkSymbols({ + // Unlike with dlopen(), the names here can be whatever you want + getMajor: { + returns: "cstring", + args: [], + + // Since this doesn't use dlsym(), you have to provide a valid ptr + // That ptr could be a number or a bigint + // An invalid pointer will crash your program. + ptr: majorPtr, + }, + getMinor: { + returns: "cstring", + args: [], + ptr: minorPtr, + }, + getPatch: { + returns: "cstring", + args: [], + ptr: patchPtr, + }, +}); + +const [major, minor, patch] = [ + lib.symbols.getMajor(), + lib.symbols.getMinor(), + lib.symbols.getPatch(), +]; +``` + +## Callbacks + +Use `JSCallback` to create JavaScript callback functions that can be passed to C/FFI functions. The C/FFI function can call into the JavaScript/TypeScript code. This is useful for asynchronous code or whenever you want to call into JavaScript code from C. + +```ts +import { dlopen, JSCallback, ptr, CString } from "bun:ffi"; + +const { + symbols: { search }, + close, +} = dlopen("libmylib", { + search: { + returns: "usize", + args: ["cstring", "callback"], + }, +}); + +const searchIterator = new JSCallback( + (ptr, length) => /hello/.test(new CString(ptr, length)), + { + returns: "bool", + args: ["ptr", "usize"], + }, +); + +const str = Buffer.from("wwutwutwutwutwutwutwutwutwutwutut\0", "utf8"); +if (search(ptr(str), searchIterator)) { + // found a match! +} + +// Sometime later: +setTimeout(() => { + searchIterator.close(); + close(); +}, 5000); +``` + +When you're done with a JSCallback, you should call `close()` to free the memory. + +{% callout %} + +**⚡️ Performance tip** — For a slight performance boost, directly pass `JSCallback.prototype.ptr` instead of the `JSCallback` object: + +```ts +const onResolve = new JSCallback((arg) => arg === 42, { + returns: "bool", + args: ["i32"], +}); +const setOnResolve = new CFunction({ + returns: "bool", + args: ["function"], + ptr: myNativeLibrarySetOnResolve, +}); + +// This code runs slightly faster: +setOnResolve(onResolve.ptr); + +// Compared to this: +setOnResolve(onResolve); +``` + +{% /callout %} + +## Pointers + +Bun represents [pointers]() as a `number` in JavaScript. + +{% details summary="How does a 64 bit pointer fit in a JavaScript number?" %} +64-bit processors support up to [52 bits of addressable space](https://en.wikipedia.org/wiki/64-bit_computing#Limits_of_processors). [JavaScript numbers](https://en.wikipedia.org/wiki/Double-precision_floating-point_format#IEEE_754_double-precision_binary_floating-point_format:_binary64) support 53 bits of usable space, so that leaves us with about 11 bits of extra space. + +**Why not `BigInt`?** `BigInt` is slower. JavaScript engines allocate a separate `BigInt` which means they can't fit into a regular JavaScript value. If you pass a `BigInt` to a function, it will be converted to a `number` +{% /details %} + +To convert from a `TypedArray` to a pointer: + +```ts +import { ptr } from "bun:ffi"; +let myTypedArray = new Uint8Array(32); +const myPtr = ptr(myTypedArray); +``` + +To convert from a pointer to an `ArrayBuffer`: + +```ts +import { ptr, toArrayBuffer } from "bun:ffi"; +let myTypedArray = new Uint8Array(32); +const myPtr = ptr(myTypedArray); + +// toArrayBuffer accepts a `byteOffset` and `byteLength` +// if `byteLength` is not provided, it is assumed to be a null-terminated pointer +myTypedArray = new Uint8Array(toArrayBuffer(myPtr, 0, 32), 0, 32); +``` + +To read data from a pointer, you have two options. For long-lived pointers, use a `DataView`: + +```ts +import { toArrayBuffer } from "bun:ffi"; +let myDataView = new DataView(toArrayBuffer(myPtr, 0, 32)); + +console.log( + myDataView.getUint8(0, true), + myDataView.getUint8(1, true), + myDataView.getUint8(2, true), + myDataView.getUint8(3, true), +); +``` + +For short-lived pointers, use `read`: + +```ts +import { read } from "bun:ffi"; + +console.log( + // ptr, byteOffset + read.u8(myPtr, 0), + read.u8(myPtr, 1), + read.u8(myPtr, 2), + read.u8(myPtr, 3), +); +``` + +The `read` function behaves similarly to `DataView`, but it's usually faster because it doesn't need to create a `DataView` or `ArrayBuffer`. + +| `FFIType` | `read` function | +| --------- | --------------- | +| ptr | `read.ptr` | +| i8 | `read.i8` | +| i16 | `read.i16` | +| i32 | `read.i32` | +| i64 | `read.i64` | +| u8 | `read.u8` | +| u16 | `read.u16` | +| u32 | `read.u32` | +| u64 | `read.u64` | +| f32 | `read.f32` | +| f64 | `read.f64` | + +### Memory management + +`bun:ffi` does not manage memory for you. You must free the memory when you're done with it. + +#### From JavaScript + +If you want to track when a `TypedArray` is no longer in use from JavaScript, you can use a [FinalizationRegistry](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/FinalizationRegistry). + +#### From C, Rust, Zig, etc + +{% callout %} +**Note** — Available in Bun v0.1.8 and later. +{% /callout %} + +If you want to track when a `TypedArray` is no longer in use from C or FFI, you can pass a callback and an optional context pointer to `toArrayBuffer` or `toBuffer`. This function is called at some point later, once the garbage collector frees the underlying `ArrayBuffer` JavaScript object. + +The expected signature is the same as in [JavaScriptCore's C API](https://developer.apple.com/documentation/javascriptcore/jstypedarraybytesdeallocator?language=objc): + +```c +typedef void (*JSTypedArrayBytesDeallocator)(void *bytes, void *deallocatorContext); +``` + +```ts +import { toArrayBuffer } from "bun:ffi"; + +// with a deallocatorContext: +toArrayBuffer( + bytes, + byteOffset, + + byteLength, + + // this is an optional pointer to a callback + deallocatorContext, + + // this is a pointer to a function + jsTypedArrayBytesDeallocator, +); + +// without a deallocatorContext: +toArrayBuffer( + bytes, + byteOffset, + + byteLength, + + // this is a pointer to a function + jsTypedArrayBytesDeallocator, +); +``` + +### Memory safety + +Using raw pointers outside of FFI is extremely not recommended. A future version of Bun may add a CLI flag to disable `bun:ffi`. + +### Pointer alignment + +If an API expects a pointer sized to something other than `char` or `u8`, make sure the `TypedArray` is also that size. A `u64*` is not exactly the same as `[8]u8*` due to alignment. + +### Passing a pointer + +Where FFI functions expect a pointer, pass a `TypedArray` of equivalent size: + +```ts +import { dlopen, FFIType } from "bun:ffi"; + +const { + symbols: { encode_png }, +} = dlopen(myLibraryPath, { + encode_png: { + // FFIType's can be specified as strings too + args: ["ptr", "u32", "u32"], + returns: FFIType.ptr, + }, +}); + +const pixels = new Uint8ClampedArray(128 * 128 * 4); +pixels.fill(254); +pixels.subarray(0, 32 * 32 * 2).fill(0); + +const out = encode_png( + // pixels will be passed as a pointer + pixels, + + 128, + 128, +); +``` + +The [auto-generated wrapper](https://github.com/oven-sh/bun/blob/6a65631cbdcae75bfa1e64323a6ad613a922cd1a/src/bun.js/ffi.exports.js#L180-L182) converts the pointer to a `TypedArray`. + +{% details summary="Hardmode" %} + +If you don't want the automatic conversion or you want a pointer to a specific byte offset within the `TypedArray`, you can also directly get the pointer to the `TypedArray`: + +```ts +import { dlopen, FFIType, ptr } from "bun:ffi"; + +const { + symbols: { encode_png }, +} = dlopen(myLibraryPath, { + encode_png: { + // FFIType's can be specified as strings too + args: ["ptr", "u32", "u32"], + returns: FFIType.ptr, + }, +}); + +const pixels = new Uint8ClampedArray(128 * 128 * 4); +pixels.fill(254); + +// this returns a number! not a BigInt! +const myPtr = ptr(pixels); + +const out = encode_png( + myPtr, + + // dimensions: + 128, + 128, +); +``` + +{% /details %} + +### Reading pointers + +```ts +const out = encode_png( + // pixels will be passed as a pointer + pixels, + + // dimensions: + 128, + 128, +); + +// assuming it is 0-terminated, it can be read like this: +let png = new Uint8Array(toArrayBuffer(out)); + +// save it to disk: +await Bun.write("out.png", png); +``` diff --git a/docs/api/file-io.md b/docs/api/file-io.md new file mode 100644 index 00000000000000..a79f6ad80ef17e --- /dev/null +++ b/docs/api/file-io.md @@ -0,0 +1,256 @@ +{% callout %} +**Note** — Bun also provides an [almost complete](/docs/runtime/nodejs) implementation of the Node.js `fs` module, documented [here](https://nodejs.org/api/fs.html). This page only documents Bun-native APIs. +{% /callout %} + +Bun provides a set of optimized APIs for reading and writing files. + +## Reading files + +`Bun.file(path): BunFile` + +Create a `BunFile` instance with the `Bun.file(path)` function. A `BunFile` represents represents a lazily-loaded file; initializing it does not actually read the file from disk. + +```ts +const foo = Bun.file("foo.txt"); // relative to cwd +foo.size; // number of bytes +foo.type; // MIME type +``` + +The reference conforms to the [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob) interface, so the contents can be read in various formats. + +```ts +const foo = Bun.file("foo.txt"); + +await foo.text(); // contents as a string +await foo.stream(); // contents as ReadableStream +await foo.arrayBuffer(); // contents as ArrayBuffer +``` + +File references can also be created using numerical [file descriptors](https://en.wikipedia.org/wiki/File_descriptor) or `file://` URLs. + +```ts +Bun.file(1234); +Bun.file(new URL(import.meta.url)); // reference to the current file +``` + +A `BunFile` can point to a location on disk where a file does not exist. + +```ts +const notreal = Bun.file("notreal.txt"); +notreal.size; // 0 +notreal.type; // "text/plain;charset=utf-8" +``` + +The default MIME type is `text/plain;charset=utf-8`, but it can be overridden by passing a second argument to `Bun.file`. + +```ts +const notreal = Bun.file("notreal.json", { type: "application/json" }); +notreal.type; // => "text/plain;charset=utf-8" +``` + +For convenience, Bun exposes `stdin`, `stdout` and `stderr` as instances of `BunFile`. + +```ts +Bun.stdin; // readonly +Bun.stdout; +Bun.stderr; +``` + +## Writing files + +`Bun.write(destination, data): Promise` + +The `Bun.write` function is a multi-tool for writing payloads of all kinds to disk. + +The first argument is the `destination` which can have any of the following types: + +- `string`: A path to a location on the file system. Use the `"path"` module to manipulate paths. +- `URL`: A `file://` descriptor. +- `BunFile`: A file reference. + +The second argument is the data to be written. It can be any of the following: + +- `string` +- `Blob` (including `BunFile`) +- `ArrayBuffer` or `SharedArrayBuffer` +- `TypedArray` (`Uint8Array`, et. al.) +- `Response` + +All possible permutations are handled using the fastest available system calls on the current platform. + +{% details summary="See syscalls" %} + +{% table %} + +- Output +- Input +- System call +- Platform + +--- + +- file +- file +- copy_file_range +- Linux + +--- + +- file +- pipe +- sendfile +- Linux + +--- + +- pipe +- pipe +- splice +- Linux + +--- + +- terminal +- file +- sendfile +- Linux + +--- + +- terminal +- terminal +- sendfile +- Linux + +--- + +- socket +- file or pipe +- sendfile (if http, not https) +- Linux + +--- + +- file (doesn't exist) +- file (path) +- clonefile +- macOS + +--- + +- file (exists) +- file +- fcopyfile +- macOS + +--- + +- file +- Blob or string +- write +- macOS + +--- + +- file +- Blob or string +- write +- Linux + +{% /table %} + +{% /details %} + +To write a string to disk: + +```ts +const data = `It was the best of times, it was the worst of times.`; +await Bun.write("output.txt", data); +``` + +To copy a file to another location on disk: + +```js +const input = Bun.file("input.txt"); +const output = Bun.file("output.txt"); // doesn't exist yet! +await Bun.write(output, input); +``` + +To write a byte array to disk: + +```ts +const encoder = new TextEncoder(); +const data = encoder.encode("datadatadata"); // Uint8Array +await Bun.write("output.txt", data); +``` + +To write a file to `stdout`: + +```ts +const input = Bun.file("input.txt"); +await Bun.write(Bun.stdout, input); +``` + +To write an HTTP response to disk: + +```ts +const response = await fetch("https://bun.sh"); +await Bun.write("index.html", response); +``` + +## Benchmarks + +The following is a 3-line implementation of the Linux `cat` command. + +```ts#cat.ts +// Usage +// $ bun ./cat.ts ./path-to-file + +import { resolve } from "path"; + +const path = resolve(process.argv.at(-1)); +await Bun.write(Bun.stdout, Bun.file(path)); +``` + +To run the file: + +```bash +$ bun ./cat.ts ./path-to-file +``` + +It runs 2x faster than GNU `cat` for large files on Linux. + +{% image src="/images/cat.jpg" /%} + +## Reference + +```ts +interface Bun { + stdin: BunFile; + stdout: BunFile; + stderr: BunFile; + + file(path: string | number | URL, options?: { type?: string }): BunFile; + + write( + destination: string | number | FileBlob | URL, + input: + | string + | Blob + | ArrayBuffer + | SharedArrayBuffer + | TypedArray + | Response, + ): Promise; +} + +interface BunFile { + readonly size: number; + readonly type: string; + + text(): Promise; + stream(): Promise; + arrayBuffer(): Promise; + json(): Promise; +} +``` diff --git a/docs/api/file-system-router.md b/docs/api/file-system-router.md new file mode 100644 index 00000000000000..e553764464b398 --- /dev/null +++ b/docs/api/file-system-router.md @@ -0,0 +1,111 @@ +Bun provides a fast API for resolving routes against file-system paths. This API is primarily intended by library authors. At the moment only Next.js-style file-system routing is supported, but other styles may be added in the future. + +## Next.js-style + +The `FileSystemRouter` class can resolve routes against a `pages` directory. (The Next.js 13 `app` directory is not yet supported.) Consider the following `pages` directory: + +```txt +pages +├── index.tsx +├── settings.tsx +├── blog +│   ├── [slug].tsx +│   └── index.tsx +└── [[...catchall]].tsx +``` + +The `FileSystemRouter` can be used to resolve routes against this directory: + +```ts +const router = new Bun.FileSystemRouter({ + style: "nextjs", + dir: "./pages", + origin: "https://mydomain.com", + assetPrefix: "_next/static/" +}); +router.match("/"); + +// => +{ + filePath: "/path/to/pages/index.tsx", + kind: "exact", + name: "/", + pathname: "/", + src: "https://mydomain.com/_next/static/pages/index.tsx" +} +``` + +Query parameters will be parsed and returned in the `query` property. + +```ts +router.match("/settings?foo=bar"); + +// => +{ + filePath: "/Users/colinmcd94/Documents/bun/fun/pages/settings.tsx", + kind: "dynamic", + name: "/settings", + pathname: "/settings?foo=bar", + src: "https://mydomain.com/_next/static/pages/settings.tsx" + query: { + foo: "bar" + } +} +``` + +The router will automatically parse URL parameters and return them in the `params` property: + +```ts +router.match("/blog/my-cool-post"); + +// => +{ + filePath: "/Users/colinmcd94/Documents/bun/fun/pages/blog/[slug].tsx", + kind: "dynamic", + name: "/blog/[slug]", + pathname: "/blog/my-cool-post", + src: "https://mydomain.com/_next/static/pages/blog/[slug].tsx" + params: { + slug: "my-cool-post" + } +} +``` + +The `.match()` method also accepts `Request` and `Response` objects. The `url` property will be used to resolve the route. + +```ts +router.match(new Request("https://example.com/blog/my-cool-post")); +``` + +The router will read the directory contents on initialization. To re-scan the files, use the `.reload()` method. + +```ts +router.reload(); +``` + +## Reference + +```ts +interface Bun { + class FileSystemRouter { + constructor(params: { + dir: string; + style: "nextjs"; + origin?: string; + assetPrefix?: string; + }); + + reload(): void; + + match(path: string | Request | Response): { + filePath: string; + kind: "exact" | "catch-all" | "optional-catch-all" | "dynamic"; + name: string; + pathname: string; + src: string; + params?: Record; + query?: Record; + } | null + } +} +``` diff --git a/docs/api/file.md b/docs/api/file.md new file mode 100644 index 00000000000000..16e5359017d2b1 --- /dev/null +++ b/docs/api/file.md @@ -0,0 +1,19 @@ +Bun.js has fast paths for common use cases that make Web APIs live up to the performance demands of servers and CLIs. + +`Bun.file(path)` returns a [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob) that represents a lazily-loaded file. + +When you pass a file blob to `Bun.write`, Bun automatically uses a faster system call: + +```js +const blob = Bun.file("input.txt"); +await Bun.write("output.txt", blob); +``` + +On Linux, this uses the [`copy_file_range`](https://man7.org/linux/man-pages/man2/copy_file_range.2.html) syscall and on macOS, this becomes `clonefile` (or [`fcopyfile`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/copyfile.3.html)). + +`Bun.write` also supports [`Response`](https://developer.mozilla.org/en-US/docs/Web/API/Response) objects. It automatically converts to a [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob). + +```js +// Eventually, this will stream the response to disk but today it buffers +await Bun.write("index.html", await fetch("https://example.com")); +``` diff --git a/docs/api/globals.md b/docs/api/globals.md new file mode 100644 index 00000000000000..d966c1861d9e09 --- /dev/null +++ b/docs/api/globals.md @@ -0,0 +1,381 @@ +Bun implements the following globals. + +{% table %} + +- Global +- Source +- Notes + +--- + +- [`AbortController`](https://developer.mozilla.org/en-US/docs/Web/API/AbortController) +- Web +-   + +--- + +- [`AbortSignal`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal) +- Web +-   + +--- + +- [`alert`](https://developer.mozilla.org/en-US/docs/Web/API/Window/alert) +- Web +- Intended for command-line tools + +--- + +- [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob) +- Web +-   + +--- + +- [`Buffer`](https://nodejs.org/api/buffer.html#class-buffer) +- Node.js +- See [Node.js > `Buffer`](/docs/runtime/nodejs#node_buffer) + +--- + +- [`Bun`](https://nodejs.org/api/buffer.html#class-buffer) +- Bun +- Subject to change as additional APIs are added + +--- + +- [`ByteLengthQueuingStrategy`](https://developer.mozilla.org/en-US/docs/Web/API/ByteLengthQueuingStrategy) +- Web +-   + +--- + +- [`confirm`](https://developer.mozilla.org/en-US/docs/Web/API/Window/confirm) +- Web +- Intended for command-line tools + +--- + +- [`__dirname`](https://nodejs.org/api/globals.html#__dirname) +- Node.js +-   + +--- + +- [`__filename`](https://nodejs.org/api/globals.html#__filename) +- Node.js +-   + +--- + +- [`atob()`](https://developer.mozilla.org/en-US/docs/Web/API/atob) +- Web +-   + +--- + +- [`btoa()`](https://developer.mozilla.org/en-US/docs/Web/API/btoa) +- Web +-   + +--- + +- `BuildError` +- Bun +-   + +--- + +- [`clearImmediate()`](https://developer.mozilla.org/en-US/docs/Web/API/Window/clearImmediate) +- Web +-   + +--- + +- [`clearInterval()`](https://developer.mozilla.org/en-US/docs/Web/API/Window/clearInterval) +- Web +-   + +--- + +- [`clearTimeout()`](https://developer.mozilla.org/en-US/docs/Web/API/Window/clearTimeout) +- Web +-   + +--- + +- [`console`](https://developer.mozilla.org/en-US/docs/Web/API/console) +- Web +-   + +--- + +- [`CountQueuingStrategy`](https://developer.mozilla.org/en-US/docs/Web/API/CountQueuingStrategy) +- Web +-   + +--- + +- [`Crypto`](https://developer.mozilla.org/en-US/docs/Web/API/Crypto) +- Web +-   + +--- + +- [`crypto`](https://developer.mozilla.org/en-US/docs/Web/API/crypto) +- Web +-   + +--- + +- [`CryptoKey`](https://developer.mozilla.org/en-US/docs/Web/API/CryptoKey) +- Web +-   + +--- + +- [`CustomEvent`](https://developer.mozilla.org/en-US/docs/Web/API/CustomEvent) +- Web +-   + +--- + +- [`Event`](https://developer.mozilla.org/en-US/docs/Web/API/Event) +- Web +- Also [`ErrorEvent`](https://developer.mozilla.org/en-US/docs/Web/API/ErrorEvent) [`CloseEvent`](https://developer.mozilla.org/en-US/docs/Web/API/CloseEvent) [`MessageEvent`](https://developer.mozilla.org/en-US/docs/Web/API/MessageEvent). + +--- + +- [`EventTarget`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget) +- Web +-   + +--- + +- [`exports`](https://nodejs.org/api/globals.html#exports) +- Node.js +-   + +--- + +- [`fetch`](https://developer.mozilla.org/en-US/docs/Web/API/fetch) +- Web +-   + +--- + +- [`FormData`](https://developer.mozilla.org/en-US/docs/Web/API/FormData) +- Web +-   + +--- + +- [`global`](https://nodejs.org/api/globals.html#global) +- Node.js +- See [Node.js > `global`](/docs/runtime/nodejs#node_global). + +--- + +- [`globalThis`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/globalThis) +- Cross-platform +- Aliases to `global` + +--- + +- [`Headers`](https://developer.mozilla.org/en-US/docs/Web/API/Headers) +- Web +-   + +--- + +- [`HTMLRewriter`](/docs/api/html-rewriter) +- Cloudflare +-   + +--- + +- [`MessageEvent`](https://developer.mozilla.org/en-US/docs/Web/API/MessageEvent) +- Web +-   + +--- + +- [`module`](https://nodejs.org/api/globals.html#module) +- Node.js +-   + +--- + +- [`performance`](https://developer.mozilla.org/en-US/docs/Web/API/performance) +- Web +-   + +--- + +- [`process`](https://nodejs.org/api/process.html) +- Node.js +- See [Node.js > `process`](/docs/runtime/nodejs#node_process) + +--- + +- [`prompt`](https://developer.mozilla.org/en-US/docs/Web/API/Window/prompt) +- Web +- Intended for command-line tools + +--- + +- [`queueMicrotask()`](https://developer.mozilla.org/en-US/docs/Web/API/queueMicrotask) +- Web +-   + +--- + +- [`ReadableByteStreamController`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableByteStreamController) +- Web +-   + +--- + +- [`ReadableStream`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream) +- Web +-   + +--- + +- [`ReadableStreamDefaultController`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamDefaultController) +- Web +-   + +--- + +- [`ReadableStreamDefaultReader`](https://developer.mozilla.org/en-US/docs/Web/API/ReadableStreamDefaultReader) +- Web +-   + +--- + +- [`reportError`](https://developer.mozilla.org/en-US/docs/Web/API/reportError) +- Web +-   + +--- + +- [`require()`](https://nodejs.org/api/globals.html#require) +- Node.js +-   + +--- + +- `ResolveError` +- Bun +-   + +--- + +- [`Response`](https://developer.mozilla.org/en-US/docs/Web/API/Response) +- Web +-   + +--- + +- [`Request`](https://developer.mozilla.org/en-US/docs/Web/API/Request) +- Web +-   + +--- + +- [`setImmediate()`](https://developer.mozilla.org/en-US/docs/Web/API/Window/setImmediate) +- Web +-   + +--- + +- [`setInterval()`](https://developer.mozilla.org/en-US/docs/Web/API/Window/setInterval) +- Web +-   + +--- + +- [`setTimeout()`](https://developer.mozilla.org/en-US/docs/Web/API/Window/setTimeout) +- Web +-   + +--- + +- [`ShadowRealm`](https://github.com/tc39/proposal-shadowrealm) +- Web +- Stage 3 proposal + +--- + +- [`SubtleCrypto`](https://developer.mozilla.org/en-US/docs/Web/API/SubtleCrypto) +- Web +-   + +--- + +- [`DOMException`](https://developer.mozilla.org/en-US/docs/Web/API/DOMException) +- Web +-   + +--- + +- [`TextDecoder`](https://developer.mozilla.org/en-US/docs/Web/API/TextDecoder) +- Web +-   + +--- + +- [`TextEncoder`](https://developer.mozilla.org/en-US/docs/Web/API/TextEncoder) +- Web +-   + +--- + +- [`TransformStream`](https://developer.mozilla.org/en-US/docs/Web/API/TransformStream) +- Web +-   + +--- + +- [`TransformStreamDefaultController`](https://developer.mozilla.org/en-US/docs/Web/API/TransformStreamDefaultController) +- Web +-   + +--- + +- [`URL`](https://developer.mozilla.org/en-US/docs/Web/API/URL) +- Web +-   + +--- + +- [`URLSearchParams`](https://developer.mozilla.org/en-US/docs/Web/API/URLSearchParams) +- Web +-   + +--- + +- [`WebAssembly`](https://nodejs.org/api/globals.html#webassembly) +- Web +-   + +--- + +- [`WritableStream`](https://developer.mozilla.org/en-US/docs/Web/API/WritableStream) +- Web +-   + +--- + +- [`WritableStreamDefaultController`](https://developer.mozilla.org/en-US/docs/Web/API/WritableStreamDefaultController) +- Web +-   + +--- + +- [`WritableStreamDefaultWriter`](https://developer.mozilla.org/en-US/docs/Web/API/WritableStreamDefaultWriter) +- Web +-   + +{% /table %} diff --git a/docs/api/html-rewriter.md b/docs/api/html-rewriter.md new file mode 100644 index 00000000000000..f67577b135aea5 --- /dev/null +++ b/docs/api/html-rewriter.md @@ -0,0 +1,31 @@ +Bun provides a fast native implementation of the `HTMLRewriter` pattern developed by Cloudflare. It provides a convenient, `EventListener`-like API for traversing and transforming HTML documents. + +```ts +const rewriter = new HTMLRewriter(); + +rewriter.on("*", { + element(el) { + console.log(el.tagName); // "body" | "div" | ... + }, +}); +``` + +To parse and/or transform the HTML: + +```ts#rewriter.ts +rewriter.transform( + new Response(` + + + + + My First HTML Page + + +

My First Heading

+

My first paragraph.

+ +`)); +``` + +View the full documentation on the [Cloudflare website](https://developers.cloudflare.com/workers/runtime-apis/html-rewriter/). diff --git a/docs/api/http.md b/docs/api/http.md new file mode 100644 index 00000000000000..d0569155611191 --- /dev/null +++ b/docs/api/http.md @@ -0,0 +1,257 @@ +{% callout %} +**Note** — Bun provides an [almost complete](/docs/runtime/nodejs#node_http) implementation of the Node.js [`http`](https://nodejs.org/api/http.html) and [`https`](https://nodejs.org/api/https.html) modules. This page only documents Bun-native APIs. +{% /callout %} + +## Start a server + +`Bun.serve(options) => Server` + +Start an HTTP server in Bun with `Bun.serve`. + +```ts +Bun.serve({ + fetch(req) { + return new Response(`Bun!`); + }, +}); +``` + +The `fetch` handler handles incoming requests. It receives a [`Request`](https://developer.mozilla.org/en-US/docs/Web/API/Request) object and returns a [`Response`](https://developer.mozilla.org/en-US/docs/Web/API/Response) or `Promise`. + +```ts +Bun.serve({ + fetch(req) { + const url = new URL(req.url); + if (url.pathname === "/") return new Response(`Home page!`); + if (url.pathname === "/blog") return new Response("Blog!"); + return new Response(`404!`); + }, +}); +``` + +To configure which port and hostname the server will listen on: + +```ts +Bun.serve({ + port: 8080, // defaults to $PORT, then 3000 + hostname: "mydomain.com", // defaults to "0.0.0.0" + fetch(req) { + return new Response(`404!`); + }, +}); +``` + +## Error handling + +To activate development mode, set `development: true`. By default, development mode is _enabled_ unless `NODE_ENV` is `production`. + +```ts +Bun.serve({ + development: true, + fetch(req) { + throw new Error("woops!"); + }, +}); +``` + +In development mode, Bun will surface errors in-browser with a built-in error page. + +{% image src="/images/exception_page.png" caption="Bun's built-in 500 page" /%} + +To handle server-side errors, implement an `error` handler. This function should return a `Response` to served to the client when an error occurs. This response will supercede Bun's default error page in `development` mode. + +```ts +Bun.serve({ + fetch(req) { + throw new Error("woops!"); + }, + error(error: Error) { + return new Response(`
${error}\n${error.stack}
`, { + headers: { + "Content-Type": "text/html", + }, + }); + }, +}); +``` + +{% callout %} +**Note** — Full debugger support is planned. +{% /callout %} + +The call to `Bun.serve` returns a `Server` object. To stop the server, call the `.stop()` method. + +```ts +const server = Bun.serve({ + fetch() { + return new Response("Bun!"); + }, +}); + +server.stop(); +``` + +## TLS + +Bun supports TLS out of the box, powered by [OpenSSL](https://www.openssl.org/). Enable TLS by passing in a value for `keyFile` and `certFile`; both are required to enable TLS. If needed, supply a `passphrase` to decrypt the `keyFile`. + +```ts +Bun.serve({ + fetch(req) { + return new Response("Hello!!!"); + }, + keyFile: "./key.pem", // path to TLS key + certFile: "./cert.pem", // path to TLS cert + passphrase: "super-secret", // optional passphrase +}); +``` + +The root CA and Diffie-Helman parameters can be configured as well. + +```ts +Bun.serve({ + fetch(req) { + return new Response("Hello!!!"); + }, + keyFile: "./key.pem", // path to TLS key + certFile: "./cert.pem", // path to TLS cert + caFile: "./ca.pem", // path to root CA certificate + dhParamsFile: "./dhparams.pem", // Diffie Helman parameters +}); +``` + +## Hot reloading + +Thus far, the examples on this page have used the explicit `Bun.serve` API. Bun also supports an alternate syntax. + +```ts#server.ts +export default { + fetch(req) { + return new Response(`Bun!`); + }, +}; +``` + +Instead of passing the server options into `Bun.serve`, export it. This file can be executed as-is; when Bun runs a file with a `default` export containing a `fetch` handler, it passes it into `Bun.serve` under the hood. + +This syntax has one major advantage: it is hot-reloadable out of the box. When any source file is changed, Bun will reload the server with the updated code _without restarting the process_. This makes hot reloads nearly instantaneous. Use the `--hot` flag when starting the server to enable hot reloading. + +```bash +$ bun --hot server.ts +``` + +It's possible to configure hot reloading while using the explicit `Bun.serve` API; for details refer to [Runtime > Hot reloading](/docs/runtime/hot). + +## Streaming files + +To stream a file, return a `Response` object with a `BunFile` object as the body. + +```ts +import { serve, file } from "bun"; + +serve({ + fetch(req) { + return new Response(Bun.file("./hello.txt")); + }, +}); +``` + +{% callout %} +⚡️ **Speed** — Bun automatically uses the [`sendfile(2)`](https://man7.org/linux/man-pages/man2/sendfile.2.html) system call when possible, enabling zero-copy file transfers in the kernel—the fastest way to send files. +{% /callout %} + +**[v0.3.0+]** You can send part of a file using the [`slice(start, end)`](https://developer.mozilla.org/en-US/docs/Web/API/Blob/slice) method on the `Bun.file` object. This automatically sets the `Content-Range` and `Content-Length` headers on the `Response` object. + +```ts +Bun.serve({ + fetch(req) { + // parse `Range` header + const [start = 0, end = Infinity] = req.headers + .get("Range") // Range: bytes=0-100 + .split("=") // ["Range: bytes", "0-100"] + .at(-1) // "0-100" + .split("-") // ["0", "100"] + .map(Number); // [0, 100] + + // return a slice of the file + const bigFile = Bun.file("./big-video.mp4"); + return new Response(bigFile.slice(start, end)); + }, +}); +``` + +## Benchmarks + +Below are Bun and Node.js implementations of a simple HTTP server that responds `Bun!` to each incoming `Request`. + +{% codetabs %} + +```ts#Bun +Bun.serve({ + fetch(req: Request) { + return new Response(`Bun!`); + }, + port: 3000, +}); +``` + +```ts#Node +require("http") + .createServer((req, res) => res.end("Bun!")) + .listen(8080); +``` + +{% /codetabs %} +The `Bun.serve` server can handle roughly 2.5x more requests per second than Node.js on Linux. + +{% table %} + +- Runtime +- Requests per second + +--- + +- Node 16 +- ~64,000 + +--- + +- Bun +- ~160,000 + +{% /table %} + +{% image width="499" alt="image" src="https://user-images.githubusercontent.com/709451/162389032-fc302444-9d03-46be-ba87-c12bd8ce89a0.png" /%} + +## Reference + +{% details summary="See TypeScript definitions" %} + +```ts +interface Bun { + serve(options: { + fetch: (req: Request, server: Server) => Response | Promise; + hostname?: string; + port?: number; + development?: boolean; + error?: (error: Error) => Response | Promise; + keyFile?: string; + certFile?: string; + caFile?: string; + dhParamsFile?: string; + passphrase?: string; + maxRequestBodySize?: number; + lowMemoryMode?: boolean; + }): Server; +} + +interface Server { + development: boolean; + hostname: string; + port: number; + pendingRequests: number; + stop(): void; +} +``` + +{% /details %} diff --git a/docs/api/node-api.md b/docs/api/node-api.md new file mode 100644 index 00000000000000..09cc9117597871 --- /dev/null +++ b/docs/api/node-api.md @@ -0,0 +1,16 @@ +Node-API is an interface for building native add-ons to Node.js. Bun implements 95% of this interface from scratch, so most existing Node-API extensions will work with Bun out of the box. Track the completion status of it in [this issue](https://github.com/oven-sh/bun/issues/158). + +As in Node.js, `.node` files (Node-API modules) can be required directly in Bun. + +```js +const napi = require("./my-node-module.node"); +``` + +Alternatively, use `process.dlopen`: + +```js +let mod = { exports: {} }; +process.dlopen(mod, "./my-node-module.node"); +``` + +Bun polyfills the [`detect-libc`](https://npmjs.com/package/detect-libc) package, which is used by many Node-API modules to detect which `.node` binding to `require`. diff --git a/docs/api/spawn.md b/docs/api/spawn.md new file mode 100644 index 00000000000000..876a0577defa62 --- /dev/null +++ b/docs/api/spawn.md @@ -0,0 +1,336 @@ +Spawn child processes with `Bun.spawn` or `Bun.spawnSync`. + +## Spawn a process + +Provide a command as an array of strings. The result of `Bun.spawn()` is a `Bun.Subprocess` object. + +```ts +Bun.spawn(["echo", "hello"]); +``` + +The second argument to `Bun.spawn` is a parameters object that can be used ton configure the subprocess. + +```ts +const proc = Bun.spawn(["echo", "hello"], { + cwd: "./path/to/subdir", // specify a working direcory + env: { ...process.env, FOO: "bar" }, // specify environment variables + onExit(proc, exitCode, signalCode, error) { + // exit handler + }, +}); + +proc.pid; // process ID of subprocess +``` + +## Input stream + +By default, the input stream of the subprocess is undefined; it can be configured with the `stdin` parameter. + +```ts +const proc = Bun.spawn(["cat"], { + stdin: await fetch( + "https://raw.githubusercontent.com/oven-sh/bun/main/examples/hashing.js", + ), +}); + +const text = await new Response(proc.stdout).text(); +console.log(text); // "const input = "hello world".repeat(400); ..." +``` + +{% table %} + +--- + +- `null` +- **Default.** Provide no input to the subprocess + +--- + +- `"pipe"` +- Return a `FileSink` for fast incremental writing + +--- + +- `"inherit"` +- Inherit the `stdin` of the parent process + +--- + +- `Bun.file()` +- Read from the specified file. + +--- + +- `TypedArray | DataView` +- Use a binary buffer as input. + +--- + +- `Response` +- Use the response `body` as input. + +--- + +- `Request` +- Use the request `body` as input. + +--- + +- `number` +- Read from the file with a given file descriptor. + +{% /table %} + +The `"pipe"` option lets incrementally write to the subprocess's input stream from the parent process. + +```ts +const proc = Bun.spawn(["cat"], { + stdin: "pipe", // return a FileSink for writing +}); + +// enqueue string data +proc.stdin!.write("hello"); + +// enqueue binary data +const enc = new TextEncoder(); +proc.stdin!.write(enc.encode(" world!")); + +// send buffered data +proc.stdin!.flush(); + +// close the input stream +proc.stdin!.end(); +``` + +## Output streams + +You can read results from the subprocess via the `stdout` and `stderr` properties. By default these are instances of `ReadableStream`. + +```ts +const proc = Bun.spawn(["echo", "hello"]); +const text = await new Response(proc.stdout).text(); +console.log(text); // => "hello" +``` + +Configure the output stream by passing one of the following values to `stdout/stderr`: + +{% table %} + +--- + +- `"pipe"` +- **Default for `stdout`.** Pipe the output to a `ReadableStream` on the returned `Subprocess` object. + +--- + +- `"inherit"` +- **Default for `stderr`.** Inherit from the parent process. + +--- + +- `Bun.file()` +- Write to the specified file. + +--- + +- `null` +- Write to `/dev/null`. + +--- + +- `number` +- Write to the file with the given file descriptor. + +{% /table %} + +## Exit handling + +Use the `onExit` callback to listen for the process exiting or being killed. + +```ts +const proc = Bun.spawn(["echo", "hello"], { + onExit(proc, exitCode, signalCode, error) { + // exit handler + }, +}); +``` + +For convenience, the `exited` property is a `Promise` that resolves when the process exits. + +```ts +const proc = Bun.spawn(["echo", "hello"]); + +await proc.exited; // resolves when process exit +proc.killed; // boolean — was the process killed? +proc.exitCode; // null | number +proc.signalCode; // null | "SIGABRT" | "SIGALRM" | ... +``` + +To kill a process: + +```ts +const proc = Bun.spawn(["echo", "hello"]); +proc.kill(); +proc.killed; // true + +proc.kill(); // specify an exit code +``` + +The parent `bun` process will not terminate until all child processes have exited. Use `proc.unref()` to detach the child process from the parent. + +``` +const proc = Bun.spawn(["echo", "hello"]); +proc.unref(); +``` + +## Blocking API + +Bun provides a synchronous equivalent of `Bun.spawn` called `Bun.spawnSync`. This is a blocking API that supports the same inputs and parameters as `Bun.spawn`. It returns a `SyncSubprocess` object, which differs from `Subprocess` in a few ways. + +1. It contains a `success` property that indicates whether the process exited with a zero exit code. +2. The `stdout` and `stderr` properties are instances of `Buffer` instead of `ReadableStream`. +3. There is no `stdin` property. Use `Bun.spawn` to incrementally write to the subprocess's input stream. + +```ts +const proc = Bun.spawnSync(["echo", "hello"]); + +console.log(proc.stdout!.toString()); +// => "hello\n" +``` + +As a rule of thumb, the asynchronous `Bun.spawn` API is better for HTTP servers and apps, and `Bun.spawnSync` is better for building command-line tools. + +## Benchmarks + +{%callout%} +⚡️ Under the hood, `Bun.spawn` and `Bun.spawnSync` use [`posix_spawn(3)`](https://man7.org/linux/man-pages/man3/posix_spawn.3.html). +{%/callout%} + +Bun's `spawnSync` spawns processes 60% faster than the Node.js `child_process` module. + +```bash +$ bun spawn.mjs +cpu: Apple M1 Max +runtime: bun 0.2.0 (arm64-darwin) + +benchmark time (avg) (min … max) p75 p99 p995 +--------------------------------------------------------- ----------------------------- +spawnSync echo hi 888.14 µs/iter (821.83 µs … 1.2 ms) 905.92 µs 1 ms 1.03 ms +$ node spawn.node.mjs +cpu: Apple M1 Max +runtime: node v18.9.1 (arm64-darwin) + +benchmark time (avg) (min … max) p75 p99 p995 +--------------------------------------------------------- ----------------------------- +spawnSync echo hi 1.47 ms/iter (1.14 ms … 2.64 ms) 1.57 ms 2.37 ms 2.52 ms +``` + +## Reference + +```ts +interface Bun { + spawn(command: string[], options?: SpawnOptions): Subprocess; + spawnSync(command: string[], options?: SpawnOptions): SyncSubprocess; +} + +interface SpawnOptions { + cwd?: string; + env?: Record; + stdin?: + | "pipe" + | "inherit" + | "ignore" + | ReadableStream + | BunFile + | Blob + | Response + | Request + | number + | null; + stdout?: + | "pipe" + | "inherit" + | "ignore" + | BunFile + | TypedArray + | DataView + | null; + stderr?: + | "pipe" + | "inherit" + | "ignore" + | BunFile + | TypedArray + | DataView + | null; + onExit?: ( + proc: Subprocess, + exitCode: number | null, + signalCode: string | null, + error: Error | null, + ) => void; +} + +interface Subprocess { + readonly pid: number; + readonly stdin?: number | ReadableStream | FileSink; + readonly stdout?: number | ReadableStream; + readonly stderr?: number | ReadableStream; + + readonly exited: Promise; + + readonly exitCode: number | undefined; + readonly signalCode: Signal | null; + readonly killed: boolean; + + ref(): void; + unref(): void; + kill(code?: number): void; +} + +interface SyncSubprocess { + readonly pid: number; + readonly success: boolean; + readonly stdout: Buffer; + readonly stderr: Buffer; +} + +type Signal = + | "SIGABRT" + | "SIGALRM" + | "SIGBUS" + | "SIGCHLD" + | "SIGCONT" + | "SIGFPE" + | "SIGHUP" + | "SIGILL" + | "SIGINT" + | "SIGIO" + | "SIGIOT" + | "SIGKILL" + | "SIGPIPE" + | "SIGPOLL" + | "SIGPROF" + | "SIGPWR" + | "SIGQUIT" + | "SIGSEGV" + | "SIGSTKFLT" + | "SIGSTOP" + | "SIGSYS" + | "SIGTERM" + | "SIGTRAP" + | "SIGTSTP" + | "SIGTTIN" + | "SIGTTOU" + | "SIGUNUSED" + | "SIGURG" + | "SIGUSR1" + | "SIGUSR2" + | "SIGVTALRM" + | "SIGWINCH" + | "SIGXCPU" + | "SIGXFSZ" + | "SIGBREAK" + | "SIGLOST" + | "SIGINFO"; +``` diff --git a/docs/api/sqlite.md b/docs/api/sqlite.md new file mode 100644 index 00000000000000..9e1668bf6640f7 --- /dev/null +++ b/docs/api/sqlite.md @@ -0,0 +1,411 @@ +Bun natively implements a high-performance [SQLite3](https://www.sqlite.org/) driver. To use it import from the built-in `bun:sqlite` module. + +```ts +import { Database } from "bun:sqlite"; + +const db = new Database(":memory:"); +const query = db.query("select 'Hello world' as message;"); +query.get(); // => { message: "Hello world" } +``` + +The API is simple, synchronous, and fast. Credit to [better-sqlite3](https://github.com/JoshuaWise/better-sqlite3) and its contributors for inspiring the API of `bun:sqlite`. + +Features include: + +- Transactions +- Parameters (named & positional) +- Prepared statements +- Datatype conversions (`BLOB` becomes `Uint8Array`) +- The fastest performance of any SQLite driver for JavaScript + +The `bun:sqlite` module is roughly 3-6x faster than `better-sqlite3` and 8-9x faster than `deno.land/x/sqlite` for read queries. Each driver was benchmarked against the [Northwind Traders](https://github.com/jpwhite3/northwind-SQLite3/blob/46d5f8a64f396f87cd374d1600dbf521523980e8/Northwind_large.sqlite.zip) dataset. View and run the [benchmark source](<[./bench/sqlite](https://github.com/oven-sh/bun/tree/main/bench/sqlite)>). + +{% image width="738" alt="SQLite benchmarks for Bun, better-sqlite3, and deno.land/x/sqlite" src="https://user-images.githubusercontent.com/709451/168459263-8cd51ca3-a924-41e9-908d-cf3478a3b7f3.png" caption="Benchmarked on an M1 MacBook Pro (64GB) running macOS 12.3.1" /%} + +## Database + +To open or create a SQLite3 database: + +```ts +import { Database } from "bun:sqlite"; + +const db = new Database("mydb.sqlite"); +``` + +To open an in-memory database: + +```ts +import { Database } from "bun:sqlite"; + +// all of these do the same thing +const db = new Database(":memory:"); +const db = new Database(); +const db = new Database(""); +``` + +To open in `readonly` mode: + +```ts +import { Database } from "bun:sqlite"; +const db = new Database("mydb.sqlite", { readonly: true }); +``` + +To create the database if the file doesn't exist: + +```ts +import { Database } from "bun:sqlite"; +const db = new Database("mydb.sqlite", { create: true }); +``` + +### `.close()` + +To close a database: + +```ts +const db = new Database(); +db.close(); +``` + +Note: `close()` is called automatically when the database is garbage collected. It is safe to call multiple times but has no effect after the first. + +### `.serialize()` + +`bun:sqlite` supports SQLite's built-in mechanism for [serializing](https://www.sqlite.org/c3ref/serialize.html) and [deserializing](https://www.sqlite.org/c3ref/deserialize.html) databases to and from memory. + +```ts +const olddb = new Database("mydb.sqlite"); +const contents = db.serialize(); // => Uint8Array +const newdb = new Database(copy); +``` + +Internally, `.serialize()` calls [`sqlite3_serialize`](https://www.sqlite.org/c3ref/serialize.html). + +### `.query()` + +Use the `db.query()` method on your `Database` instance to [prepare](https://www.sqlite.org/c3ref/prepare.html) a SQL query. The result is a `Statement` instance that will be cached on the `Database` instance. _The query will not be executed._ + +```ts +const query = db.query(`select "Hello world" as message`); +``` + +{% callout %} + +**Note** — Use the `.prepare()` method to prepare a query _without_ caching it on the `Database` instance. + +```ts +// compile the prepared statement +const query = db.prepare("SELECT * FROM foo WHERE bar = ?"); +``` + +{% /callout %} + +## Statements + +A `Statement` is a _prepared query_, which means it's been parsed and compiled into an efficient binary form. It can be executed multiple times in a performant way. + +Create a statement with the `.query` method on your `Database` instance. + +```ts +const query = db.query(`select "Hello world" as message`); +``` + +Queries can contain parameters. These can be numerical (`?1`) or named (`$param` or `:param` or `@param`). + +```ts +const query = db.query(`SELECT ?1, ?2;`); +const query = db.query(`SELECT $param1, $param2;`); +``` + +Values are bound to these parameters when the query is executed. A `Statement` can be executed with several different methods, each returning the results in a different form. + +### `.all()` + +Use `.all()` to run a query and get back the results as an array of objects. + +```ts +const query = db.query(`select $message;`); +query.all({ $message: "Hello world" }); +// => [{ message: "Hello world" }] +``` + +Internally, this calls [`sqlite3_reset`](https://www.sqlite.org/capi3ref.html#sqlite3_reset) and repeatedly calls [`sqlite3_step`](https://www.sqlite.org/capi3ref.html#sqlite3_step) until it returns `SQLITE_DONE`. + +### `.get()` + +Use `.get()` to run a query and get back the first result as an object. + +```ts +const query = db.query(`select $message;`); +query.get({ $message: "Hello world" }); +// => { $message: "Hello world" } +``` + +Internally, this calls [`sqlite3_reset`](https://www.sqlite.org/capi3ref.html#sqlite3_reset) and calls [`sqlite3_step`](https://www.sqlite.org/capi3ref.html#sqlite3_step) once. Stepping through all the rows is not necessary when you only want the first row. + +### `.run()` + +Use `.run()` to run a query and get back `undefined`. This is useful for queries schema-modifying queries (e.g. `CREATE TABLE`) or bulk write operations. + +```ts +const query = db.query(`create table foo;`); +query.run(); +// => undefined +``` + +Internally, this calls [`sqlite3_reset`](https://www.sqlite.org/capi3ref.html#sqlite3_reset) and calls [`sqlite3_step`](https://www.sqlite.org/capi3ref.html#sqlite3_step) once. Stepping through all the rows is not necessary when you don't care about the results. + +### `.values()` + +Use `values()` to run a query and get back all results as an array of arrays. + +```ts +const query = db.query(`select $message;`); +query.values({ $message: "Hello world" }); + +query.values(2); +// [ +// [ "Iron Man", 2008 ], +// [ "The Avengers", 2012 ], +// [ "Ant-Man: Quantumania", 2023 ], +// ] +``` + +Internally, this calls [`sqlite3_reset`](https://www.sqlite.org/capi3ref.html#sqlite3_reset) and repeatedly calls [`sqlite3_step`](https://www.sqlite.org/capi3ref.html#sqlite3_step) until it returns `SQLITE_DONE`. + +### `.finalize()` + +Use `.finalize()` to destroy a `Statement` and free any resources associated with it. Once finalized, a `Statement` cannot be executed again. Typically, the garbage collector will do this for you, but explicit finalization may be useful in performance-sensitive applications. + +```ts +const query = db.query("SELECT title, year FROM movies"); +const movies = query.all(); +query.finalize(); +``` + +### `.toString()` + +Calling `toString()` on a `Statement` instance prints the expanded SQL query. This is useful for debugging. + +```ts +import { Database } from "bun:sqlite"; + +// setup +const query = db.query("SELECT $param;"); + +console.log(query.toString()); // => "SELECT NULL" + +query.run(42); +console.log(query.toString()); // => "SELECT 42" + +query.run(365); +console.log(query.toString()); // => "SELECT 365" +``` + +Internally, this calls [`sqlite3_expanded_sql`](https://www.sqlite.org/capi3ref.html#sqlite3_expanded_sql). The parameters are expanded using the most recently bound values. + +## Parameters + +Queries can contain parameters. These can be numerical (`?1`) or named (`$param` or `:param` or `@param`). Bind values to these parameters when executing the query: + +{% codetabs %} + +```ts#Query +const query = db.query("SELECT * FROM foo WHERE bar = $bar"); +const results = await query.all({ + $bar: "bar", +}); +``` + +```json#Results +[ + { "$bar": "bar" } +] +``` + +{% /codetabs %} + +Numbered (positional) parameters work too: + +{% codetabs %} + +```ts#Query +const query = db.query("SELECT ?1, ?2"); +const results = await query.all("hello", "goodbye"); +``` + +```ts#Results +[ + { + "?1": "hello", + "?2": "goodbye" + } +] +``` + +{% /codetabs %} + +## Transactions + +Transactions are a mechanism for executing multiple queries in an _atomic_ way; that is, either all of the queries succeed or none of them do. Create a transaction with the `db.transaction()` method: + +```ts +const insertCat = db.prepare("INSERT INTO cats (name) VALUES ($name)"); +const insertCats = db.transaction((cats) => { + for (const cat of cats) insertCat.run(cat); +}); +``` + +At this stage, we haven't inserted any cats! The call to `db.transaction()` returns a new function (`insertCats`) that _wraps_ the function that executes the queries. + +To execute the transaction, call this function. All arguments will be passed through to the wrapped function; the return value of the wrapped function will be returned by the transaction function. The wrapped function also has access to the `this` context as defined where the transaction is executed. + +```ts +const insert = db.prepare("INSERT INTO cats (name) VALUES ($name)"); +const insertCats = db.transaction((cats) => { + for (const cat of cats) insert.run(cat); + return cats.length; +}); + +const count = insertCats([ + { $name: "Keanu" }, + { $name: "Salem" }, + { $name: "Crookshanks" }, +]); + +console.log(`Inserted ${count} cats`); +``` + +The driver will automatically [`begin`](https://www.sqlite.org/lang_transaction.html) a transaction when `insertCats` is called and `commit` it when the wrapped function returns. If an exception is thrown, the transaction will be rolled back. The exception will propagate as usual; it is not caught. + +{% callout %} +**Nested transactions** — Transaction functions can be called from inside other transaction functions. When doing so, the inner transaction becomes a [savepoint](https://www.sqlite.org/lang_savepoint.html). + +{% details summary="View nested transaction example" %} + +```ts +// setup +import { Database } from "bun:sqlite"; +const db = Database.open(":memory:"); +db.run( + "CREATE TABLE expenses (id INTEGER PRIMARY KEY AUTOINCREMENT, note TEXT, dollars INTEGER);", +); +db.run( + "CREATE TABLE cats (id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT UNIQUE, age INTEGER)", +); +const insertExpense = db.prepare( + "INSERT INTO expenses (note, dollars) VALUES (?, ?)", +); +const insert = db.prepare("INSERT INTO cats (name, age) VALUES ($name, $age)"); +const insertCats = db.transaction((cats) => { + for (const cat of cats) insert.run(cat); +}); + +const adopt = db.transaction((cats) => { + insertExpense.run("adoption fees", 20); + insertCats(cats); // nested transaction +}); + +adopt([ + { $name: "Joey", $age: 2 }, + { $name: "Sally", $age: 4 }, + { $name: "Junior", $age: 1 }, +]); +``` + +{% /details %} +{% /callout %} + +Transactions also come with `deferred`, `immediate`, and `exclusive` versions. + +```ts +insertCats(cats); // uses "BEGIN" +insertCats.deferred(cats); // uses "BEGIN DEFERRED" +insertCats.immediate(cats); // uses "BEGIN IMMEDIATE" +insertCats.exclusive(cats); // uses "BEGIN EXCLUSIVE" +``` + +### `.loadExtension()` + +To load a [SQLite extension](https://www.sqlite.org/loadext.html), call `.loadExtension(name)` on your `Database` instance + +```ts +import { Database } from "bun:sqlite"; + +const db = new Database(); +db.loadExtension("myext"); +``` + +{% details summary="For macOS users" %} +**MacOS users** By default, macOS ships with Apple's proprietary build of SQLite, which doesn't support extensions. To use extensions, you'll need to install a vanilla build of SQLite. + +```bash +$ brew install sqlite +$ which sqlite # get path to binary +``` + +To point `bun:sqlite` to the new build, call `Database.setCustomSQLite(path)` before creating any `Database` instances. (On other operating systems, this is a no-op.) Pass a path to the SQLite `.dylib` file, _not_ the executable. With recent versions of Homebrew this is something like `/opt/homebrew/Cellar/sqlite//libsqlite3.dylib`. + +```ts +import { Database } from "bun:sqlite"; + +Database.setCustomSQLite("/path/to/libsqlite.dylib"); + +const db = new Database(); +db.loadExtension("myext"); +``` + +{% /details %} + +## Reference + +```ts +class Database { + constructor( + filename: string, + options?: + | number + | { + readonly?: boolean; + create?: boolean; + readwrite?: boolean; + }, + ); + + query(sql: string): Statement; +} + +class Statement { + all(params: Params): ReturnType[]; + get(params: Params): ReturnType | undefined; + run(params: Params): void; + values(params: Params): unknown[][]; + + finalize(): void; // destroy statement and clean up resources + toString(): string; // serialize to SQL + + columnNames: string[]; // the column names of the result set + paramsCount: number; // the number of parameters expected by the statement + native: any; // the native object representing the statement +} + +type SQLQueryBindings = + | string + | bigint + | TypedArray + | number + | boolean + | null + | Record; +``` + +### Datatypes + +| JavaScript type | SQLite type | +| --------------- | ---------------------- | +| `string` | `TEXT` | +| `number` | `INTEGER` or `DECIMAL` | +| `boolean` | `INTEGER` (1 or 0) | +| `Uint8Array` | `BLOB` | +| `Buffer` | `BLOB` | +| `bigint` | `INTEGER` | +| `null` | `NULL` | diff --git a/docs/api/tcp.md b/docs/api/tcp.md new file mode 100644 index 00000000000000..5e04dd348225dd --- /dev/null +++ b/docs/api/tcp.md @@ -0,0 +1,198 @@ +Use Bun's native TCP API implement performance sensitive systems like database clients, game servers, or anything that needs to communicate over TCP (instead of HTTP). This is a low-level API intended for library authors and for advanced use cases. + +## Start a server + +To start a TCP server with `Bun.listen`: + +```ts +Bun.listen({ + hostname: "localhost", + port: 8080, + socket: { + data(socket, data) {}, // message received from client + open(socket) {}, // socket opened + close(socket) {}, // socket closed + drain(socket) {}, // socket ready for more data + error(socket, error) {}, // error handler + }, +}); +``` + +{% details summary="An API designed for speed" %} + +In Bun, a set of handlers are declared once per server instead of assigning callbacks to each socket, as with Node.js `EventEmitters` or the web-standard `WebSocket` API. + +```ts +Bun.listen({ + hostname: "localhost", + port: 8080, + socket: { + open(socket) {}, + data(socket, data) {}, + drain(socket) {}, + close(socket) {}, + error(socket, error) {}, + }, +}); +``` + +For performance-sensitive servers, assigning listeners to each socket can cause significant garbage collector pressure and increase memory usage. By contrast, Bun only allocates one handler function for each event and shares it among all sockets. This is a small optimization, but it adds up. + +{% /details %} + +Contextual data can be attached to a socket in the `open` handler. + +```ts +type SocketData = { sessionId: string }; + +Bun.listen({ + hostname: "localhost", + port: 8080, + socket: { + data(socket, data) { + socket.write(`${socket.data.sessionId}: ack`); + }, + open(socket) { + socket.data = { sessionId: "abcd" }; + }, + }, +}); +``` + +To enable TLS, pass a `tls` object containing `keyFile` and `certFile` properties. + +```ts +Bun.listen({ + hostname: "localhost", + port: 8080, + socket: { + data(socket, data) {}, + }, + tls: { + certFile: "cert.pem", + keyFile: "key.pem", + }, +}); +``` + +The result of `Bun.listen` is a server that conforms to the `TCPSocket` instance. + +```ts +const server = Bun.listen({ + /* config*/ +}); + +// stop listening +// parameter determines whether active connections are closed +server.stop(true); + +// let Bun process exit even if server is still listening +server.unref(); +``` + +## Create a connection + +Use `Bun.connect` to connect to a TCP server. Specify the server to connect to with `hostname` and `port`. TCP clients can define the same set of handlers as `Bun.listen`, plus a couple client-specific handlers. + +```ts +// The client +const socket = Bun.connect({ + hostname: "localhost", + port: 8080, + + socket: { + data(socket, data) {}, + open(socket) {}, + close(socket) {}, + drain(socket) {}, + error(socket, error) {}, + + // client-specific handlers + connectError(socket, error) {}, // connection failed + end(socket) {}, // connection closed by server + timeout(socket) {}, // connection timed out + }, +}); +``` + +To require TLS, specify `tls: true`. + +```ts +// The client +const socket = Bun.connect({ + // ... config + tls: true, +}); +``` + +## Hot reloading + +Both TCP servers and sockets can be hot reloaded with new handlers. + +{% codetabs %} + +```ts#Server +const server = Bun.listen({ /* config */ }) + +// reloads handlers for all active server-side sockets +server.reload({ + socket: + data(){ + // new 'data' handler + } + } +}) +``` + +```ts#Client +const socket = Bun.connect({ /* config */ }) +socket.reload({ + data(){ + // new 'data' handler + } +}) +``` + +{% /codetabs %} + +## Buffering + +Currently, TCP sockets in Bun do not buffer data. For performance-sensitive code, it's important to consider buffering carefully. For example, this: + +```ts +socket.write("h"); +socket.write("e"); +socket.write("l"); +socket.write("l"); +socket.write("o"); +``` + +...performs significantly worse than this: + +```ts +socket.write("hello"); +``` + +To simplify this for now, consider using Bun's `ArrayBufferSink` with the `{stream: true}` option: + +```ts +const sink = new ArrayBufferSink({ stream: true, highWaterMark: 1024 }); + +sink.write("h"); +sink.write("e"); +sink.write("l"); +sink.write("l"); +sink.write("o"); + +queueMicrotask(() => { + var data = sink.flush(); + if (!socket.write(data)) { + // put it back in the sink if the socket is full + sink.write(data); + } +}); +``` + +{% callout %} +**Corking** — Support for corking is planned, but in the meantime backpressure must be managed manually with the `drain` handler. +{% /callout %} diff --git a/docs/api/test.md b/docs/api/test.md new file mode 100644 index 00000000000000..6704d407d6d341 --- /dev/null +++ b/docs/api/test.md @@ -0,0 +1 @@ +See the [`bun test`](/docs/cli/test) documentation. diff --git a/docs/api/transpiler.md b/docs/api/transpiler.md new file mode 100644 index 00000000000000..184007212eea8b --- /dev/null +++ b/docs/api/transpiler.md @@ -0,0 +1,275 @@ +Bun exposes its internal transpiler via the `Bun.Transpiler` class. To create an instance of Bun's transpiler: + +```ts +const tx = new Bun.Transpiler({ + loader: "tsx", // "js | "jsx" | "ts" | "tsx" +}); +``` + +## `.transformSync()` + +Transpile code synchronously with the `.transformSync()` method. Modules are not resolved and the code is not executed. The result is a string of vanilla JavaScript code. + + + +{% codetabs %} + +```js#Example +const transpiler = new Bun.Transpiler({ + loader: 'tsx', +}); + +const code = ` +import * as whatever from "./whatever.ts" +export function Home(props: {title: string}){ + return

{props.title}

; +}`; + +const result = tx.transformSync(code); +``` + +```js#Result +import { __require as require } from "bun:wrap"; +import * as JSX from "react/jsx-dev-runtime"; +var jsx = require(JSX).jsxDEV; + +export default jsx( + "div", + { + children: "hi!", + }, + undefined, + false, + undefined, + this, +); +``` + +{% /codetabs %} + +To override the default loader specified in the `new Bun.Transpiler()` constructor, pass a second argument to `.transformSync()`. + +```ts +await transpiler.transform("
hi!
", "tsx"); +``` + +{% details summary="Nitty gritty" %} +When `.transformSync` is called, the transpiler is run in the same thread as the currently executed code. + +If a macro is used, it will be run in the same thread as the transpiler, but in a separate event loop from the rest of your application. Currently, globals between macros and regular code are shared, which means it is possible (but not recommended) to share states between macros and regular code. Attempting to use AST nodes outside of a macro is undefined behavior. +{% /details %} + +## `.transform()` + +The `transform()` method is an async version of `.transformSync()` that returns a `Promise`. + +```js +const transpiler = new Bun.Transpiler({ loader: "jsx" }); +const result = await transpiler.transform("
hi!
"); +console.log(result); +``` + +Unless you're transpiling _many_ large files, you should probably use `Bun.Transpiler.transformSync`. The cost of the threadpool will often take longer than actually transpiling code. + +```ts +await transpiler.transform("
hi!
", "tsx"); +``` + +{% details summary="Nitty gritty" %} +The `.tranform()` method runs the transpiler in Bun's worker threadpool, so if you run it 100 times, it will run it across `Math.floor($cpu_count * 0.8)` threads, without blocking the main JavaScript thread. + +If your code uses a macro, it will potentially spawn a new copy of Bun's JavaScript runtime environment in that new thread. +{% /details %} + +## `.scan()` + +The `Transpiler` instance can also scan some source code and return a list of its imports and exports, plus additional metadata about each one. [Type-only](https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-8.html#type-only-imports-and-export)imports and exports are ignored. + +{% codetabs %} + +```ts#Example +const transpiler = new Bun.Transpiler({ + loader: 'tsx', +}); + +const code = ` +import React from 'react'; +import type {ReactNode} from 'react'; +const val = require('./cjs.js') +import('./loader'); + +export const name = "hello"; +`; + +const result = transpiler.scan(code); +``` + +```json#Output +{ + "exports": [ + "name" + ], + "imports": [ + { + "kind": "import-statement", + "path": "react" + }, + { + "kind": "import-statement", + "path": "remix" + }, + { + "kind": "dynamic-import", + "path": "./loader" + } + ] +} +``` + +{% /codetabs %} + +Each import in the `imports` array has a `path` and `kind`. Bun categories imports into the following kinds: + +- `import-statement`: `import React from 'react'` +- `require-call`: `const val = require('./cjs.js')` +- `require-resolve`: `require.resolve('./cjs.js')` +- `dynamic-import`: `import('./loader')` +- `import-rule`: `@import 'foo.css'` +- `url-token`: `url('./foo.png')` + + +## `.scanImports()` + +For performance-sensitive code, you can use the `.scanImports()` method to get a list of imports. It's faster than `.scan()` (especially for large files) but marginally less accurate due to some performance optimizations. + +{% codetabs %} + +```ts#Example +const transpiler = new Bun.Transpiler({ + loader: 'tsx', +}); + +const code = ` +import React from 'react'; +import type {ReactNode} from 'react'; +const val = require('./cjs.js') +import('./loader'); + +export const name = "hello"; +`; + +const result = transpiler.scanImports(code); +`); +``` + +```json#Results +[ + { + kind: "import-statement", + path: "react" + }, { + kind: "require-call", + path: "./cjs.js" + }, { + kind: "dynamic-import", + path: "./loader" + } +] +``` + +{% /codetabs %} + +## Reference + +```ts +type Loader = "jsx" | "js" | "ts" | "tsx"; + +interface TranspilerOptions { + // Replace key with value. Value must be a JSON string. + // { "process.env.NODE_ENV": "\"production\"" } + define?: Record, + + // Default loader for this transpiler + loader?: Loader, + + // Default platform to target + // This affects how import and/or require is used + platform?: "browser" | "bun" | "macro" | "node", + + // Specify a tsconfig.json file as stringified JSON or an object + // Use this to set a custom JSX factory, fragment, or import source + // For example, if you want to use Preact instead of React. Or if you want to use Emotion. + tsconfig?: string | TSConfig, + + // Replace imports with macros + macro?: MacroMap, + + // Specify a set of exports to eliminate + // Or rename certain exports + exports?: { + eliminate?: string[]; + replace?: Record; + }, + + // Whether to remove unused imports from transpiled file + // Default: false + trimUnusedImports?: boolean, + + // Whether to enable a set of JSX optimizations + // jsxOptimizationInline ..., + + // Experimental whitespace minification + minifyWhitespace?: boolean, + + // Whether to inline constant values + // Typically improves performance and decreases bundle size + // Default: true + inline?: boolean, +} + +// Map import paths to macros +interface MacroMap { + // { + // "react-relay": { + // "graphql": "bun-macro-relay/bun-macro-relay.tsx" + // } + // } + [packagePath: string]: { + [importItemName: string]: string, + }, +} + +class Bun.Transpiler { + constructor(options: TranspilerOptions) + + transform(code: string, loader?: Loader): Promise + transformSync(code: string, loader?: Loader): string + + scan(code: string): {exports: string[], imports: Import} + scanImports(code: string): Import[] +} + +type Import = { + path: string, + kind: + // import foo from 'bar'; in JavaScript + | "import-statement" + // require("foo") in JavaScript + | "require-call" + // require.resolve("foo") in JavaScript + | "require-resolve" + // Dynamic import() in JavaScript + | "dynamic-import" + // @import() in CSS + | "import-rule" + // url() in CSS + | "url-token" + // The import was injected by Bun + | "internal"  + // Entry point (not common) + | "entry-point" +} + +const transpiler = new Bun.Transpiler({ loader: "jsx" }); +``` diff --git a/docs/api/utils.md b/docs/api/utils.md new file mode 100644 index 00000000000000..dcd9c01284f737 --- /dev/null +++ b/docs/api/utils.md @@ -0,0 +1,120 @@ +## `Bun.sleep` + +`Bun.sleep(ms: number)` (added in Bun v0.5.6) + +Returns a `Promise` that resolves after the given number of milliseconds. + +```ts +console.log("hello"); +await Bun.sleep(1000); +console.log("hello one second later!"); +``` + +Alternatively, pass a `Date` object to receive a `Promise` that resolves at that point in time. + +```ts +const oneSecondInFuture = new Date(Date.now() + 1000); + +console.log("hello"); +await Bun.sleep(oneSecondInFuture); +console.log("hello one second later!"); +``` + +## `Bun.which` + +`Bun.which(bin: string)` + +Find the path to an executable, similar to typing `which` in your terminal. + +```ts +const ls = Bun.which("ls"); +console.log(ls); // "/usr/bin/ls" +``` + +By default Bun looks at the current `PATH` environment variable to determine the path. To configure `PATH`: + +```ts +const ls = Bun.which("ls", { + PATH: "/usr/local/bin:/usr/bin:/bin", +}); +console.log(ls); // "/usr/bin/ls" +``` + +Pass a `cwd` option to resolve for executable from within a specific directory. + +```ts +const ls = Bun.which("ls", { + cwd: "/tmp", + PATH: "", +}); + +console.log(ls); // null +``` + +## `Bun.peek` + +`Bun.peek(prom: Promise)` (added in Bun v0.2.2) + +`Bun.peek` is a utility function that lets you read a promise's result without `await` or `.then`, but only if the promise has already fulfilled or rejected. + +```ts +import { peek } from "bun"; + +const promise = Promise.resolve("hi"); + +// no await! +const result = peek(promise); +console.log(result); // "hi" +``` + +This is important when attempting to reduce number of extraneous microticks in performance-sensitive code. It's an advanced API and you probably shouldn't use it unless you know what you're doing. + +```ts +import { peek } from "bun"; +import { expect, test } from "bun:test"; + +test("peek", () => { + const promise = Promise.resolve(true); + + // no await necessary! + expect(peek(promise)).toBe(true); + + // if we peek again, it returns the same value + const again = peek(promise); + expect(again).toBe(true); + + // if we peek a non-promise, it returns the value + const value = peek(42); + expect(value).toBe(42); + + // if we peek a pending promise, it returns the promise again + const pending = new Promise(() => {}); + expect(peek(pending)).toBe(pending); + + // If we peek a rejected promise, it: + // - returns the error + // - does not mark the promise as handled + const rejected = Promise.reject( + new Error("Successfully tested promise rejection"), + ); + expect(peek(rejected).message).toBe("Successfully tested promise rejection"); +}); +``` + +The `peek.status` function lets you read the status of a promise without resolving it. + +```ts +import { peek } from "bun"; +import { expect, test } from "bun:test"; + +test("peek.status", () => { + const promise = Promise.resolve(true); + expect(peek.status(promise)).toBe("fulfilled"); + + const pending = new Promise(() => {}); + expect(peek.status(pending)).toBe("pending"); + + const rejected = Promise.reject(new Error("oh nooo")); + expect(peek.status(rejected)).toBe("rejected"); +}); +``` diff --git a/docs/api/websockets.md b/docs/api/websockets.md new file mode 100644 index 00000000000000..f4556d46a204f7 --- /dev/null +++ b/docs/api/websockets.md @@ -0,0 +1,471 @@ +As of Bun v0.2.1, `Bun.serve()` supports server-side WebSockets, with on-the-fly compression, TLS support, and a Bun-native pubsub API. + +{% callout %} + +**⚡️ Speed** — Bun's WebSockets are fast. For a [simple chatroom](https://github.com/oven-sh/bun/tree/main/bench/websocket-server/README.md) on Linux x64, Bun can handle 7x more requests per second than Node.js + [`"ws"`](https://github.com/websockets/ws). + +| Messages sent per second | Runtime | Clients | +| ------------------------ | ------------------------------ | ------- | +| ~700,000 | (`Bun.serve`) Bun v0.2.1 (x64) | 16 | +| ~100,000 | (`ws`) Node v18.10.0 (x64) | 16 | + +Internally Bun's WebSocket implementation is built on [uWebSockets](https://github.com/uNetworking/uWebSockets). +{% /callout %} + +## Create a client + +To connect to an external socket server, create an instance of `WebSocket` with the constructor. + +```ts +const socket = new WebSocket("ws://localhost:8080"); +``` + +Bun supports setting custom headers. This is a Bun-specific extension of the `WebSocket` standard. + +```ts +const socket = new WebSocket("ws://localhost:8080", { + headers: { + // custom headers + }, +}); +``` + +To add event listeners to the socket: + +```ts +// message is received +socket.addEventListener("message", (event) => {}); + +// socket opened +socket.addEventListener("open", (event) => {}); + +// socket closed +socket.addEventListener("close", (event) => {}); + +// error handler +socket.addEventListener("error", (event) => {}); +``` + +## Create a server + +Below is a simple WebSocket server built with `Bun.serve`, in which all incoming requests are [upgraded](https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism) to WebSocket connections in the `fetch` handler. The socket handlers are declared in the `websocket` parameter. + +```ts +Bun.serve({ + fetch(req, server) { + // upgrade the request to a WebSocket + if (server.upgrade(req)) { + return; // do not return a Response + } + return new Response("Upgrade failed :(", { status: 500 }); + }, + websocket: {}, // handlers +}); +``` + +The following WebSocket event handlers are supported: + +```ts +Bun.serve({ + fetch(req, server) {}, // upgrade logic + websocket: { + message(ws, message) {}, // a message is received + open(ws) {}, // a socket is opened + close(ws, code, message) {}, // a socket is closed + drain(ws) {}, // the socket is ready to receive more data + }, +}); +``` + +{% details summary="An API designed for speed" %} + +In Bun, handlers are declared once per server, instead of per socket. + +`ServerWebSocket` expects you to pass a `WebSocketHandler` object to the `Bun.serve()` method which has methods for `open`, `message`, `close`, `drain`, and `error`. This is different than the client-side `WebSocket` class which extends `EventTarget` (onmessage, onopen, onclose), + +Clients tend to not have many socket connections open so an event-based API makes sense. + +But servers tend to have **many** socket connections open, which means: + +- Time spent adding/removing event listeners for each connection adds up +- Extra memory spent on storing references to callbacks function for each connection +- Usually, people create new functions for each connection, which also means more memory + +So, instead of using an event-based API, `ServerWebSocket` expects you to pass a single object with methods for each event in `Bun.serve()` and it is reused for each connection. + +This leads to less memory usage and less time spent adding/removing event listeners. +{% /details %} + +The first argument to each handler is the instance of `ServerWebSocket` handling the event. The `ServerWebSocket` class is a fast, Bun-native implementation of [`WebSocket`](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket) with some additional features. + +```ts +Bun.serve({ + fetch(req, server) {}, // upgrade logic + websocket: { + message(ws, message) { + ws.send(message); // echo back the message + }, + }, +}); +``` + +## Sending messages + +Each `ServerWebSocket` instance has a `.send()` method for sending messages to the client. It supports a range of input types. + +```ts +ws.send("Hello world"); // string +ws.send(response.arrayBuffer()); // ArrayBuffer +ws.send(new Uint8Array([1, 2, 3])); // TypedArray | DataView +``` + +## Headers + +Once the upgrade succeeds, Bun will send a `101 Switching Protocols` response per the [spec](https://developer.mozilla.org/en-US/docs/Web/HTTP/Protocol_upgrade_mechanism). Additional `headers` can be attched to this `Response` in the call to `server.upgrade()`. + +```ts +Bun.serve({ + fetch(req, server) { + const sessionId = await generateSessionId(); + server.upgrade(req, { + headers: { + "Set-Cookie": `SessionId=${sessionId}`, + }, + }); + }, + websocket: {}, // handlers +}); +``` + +## Contextual data + +Contextual `data` can be attached to a new WebSocket in the `.upgrade()` call. This data is made available on the `ws.data` property inside the WebSocket handlers. + +```ts +type WebSocketData = { + createdAt: number; + channelId: string; +}; + +// TypeScript: specify the type of `data` +Bun.serve({ + fetch(req, server) { + server.upgrade(req, { + // TS: this object must conform to WebSocketData + data: { + createdAt: Date.now(), + channelId: new URL(req.url).searchParams.get("channelId"), + }, + }); + + return undefined; + }, + websocket: { + // handler called when a message is received + async message(ws, message) { + ws.data; // WebSocketData + await saveMessageToDatabase({ + channel: ws.data.channelId, + message: String(message), + }); + }, + }, +}); +``` + +## Pub/Sub + +Bun's `ServerWebSocket` implementation implements a native publish-subscribe API for topic-based broadcasting. Individual sockets can `.subscribe()` to a topic (specified with a string identifier) and `.publish()` messages to all other subscribers to that topic. This topic-based broadcast API is similar to [MQTT](https://en.wikipedia.org/wiki/MQTT) and [Redis Pub/Sub](https://redis.io/topics/pubsub). + +```ts +const pubsubserver = Bun.serve<{username: string}>({ + fetch(req, server) { + if (req.url === '/chat') { + const cookies = getCookieFromRequest(req); + const success = server.upgrade(req, { + data: {username: cookies.username}, + }); + return success + ? undefined + : new Response('WebSocket upgrade error', {status: 400}); + } + + return new Response('Hello world'); + }, + websocket: { + open(ws) { + ws.subscribe('the-group-chat'); + ws.publish('the-group-chat', `${ws.data.username} has entered the chat`); + }, + message(ws, message) { + // this is a group chat + // so the server re-broadcasts incoming message to everyone + ws.publish('the-group-chat', `${ws.data.username}: ${message}`); + }, + close(ws) { + ws.unsubscribe('the-group-chat'); + ws.publish('the-group-chat', `${ws.data.username} has left the chat`); + }, +}); +``` + +## Compression + +Per-message [compression](https://websockets.readthedocs.io/en/stable/topics/compression.html) can be enabled with the `perMessageDeflate` parameter. + +```ts +Bun.serve({ + fetch(req, server) {}, // upgrade logic + websocket: { + // enable compression and decompression + perMessageDeflate: true, + }, +}); +``` + +Compression can be enabled for individual messages by passing a `boolean` as the second argument to `.send()`. + +```ts +ws.send("Hello world", true); +``` + +For fine-grained control over compression characteristics, refer to the [Reference](#reference). + +## Backpressure + +The `.send(message)` method of `ServerWebSocket` returns a `number` indicating the result of the operation. + +- `-1` — The message was enqueued but there is backpressure +- `0` — The message was dropped due to a connection issue +- `1+` — The number of bytes sent + +This gives you better control over backpressure in your server. + +## Reference + +```ts +namespace Bun { + export function serve(params: { + fetch: (req: Request, server: Server) => Response | Promise; + websocket?: { + message: ( + ws: ServerWebSocket, + message: string | ArrayBuffer | Uint8Array, + ) => void; + open?: (ws: ServerWebSocket) => void; + close?: (ws: ServerWebSocket) => void; + error?: (ws: ServerWebSocket, error: Error) => void; + drain?: (ws: ServerWebSocket) => void; + perMessageDeflate?: + | boolean + | { + compress?: boolean | Compressor; + decompress?: boolean | Compressor; + }; + }; + }): Server; +} + +type Compressor = + | `"disable"` + | `"shared"` + | `"dedicated"` + | `"3KB"` + | `"4KB"` + | `"8KB"` + | `"16KB"` + | `"32KB"` + | `"64KB"` + | `"128KB"` + | `"256KB"`; + +interface Server { + pendingWebsockets: number; + publish( + topic: string, + data: string | ArrayBufferView | ArrayBuffer, + compress?: boolean, + ): number; + upgrade( + req: Request, + options?: { + headers?: HeadersInit; + data?: any; + }, + ): boolean; +} + +interface ServerWebSocket { + readonly data: any; + readonly readyState: number; + readonly remoteAddress: string; + send(message: string | ArrayBuffer | Uint8Array, compress?: boolean): number; + close(code?: number, reason?: string): void; + subscribe(topic: string): void; + unsubscribe(topic: string): void; + publish(topic: string, message: string | ArrayBuffer | Uint8Array): void; + isSubscribed(topic: string): boolean; + cork(cb: (ws: ServerWebSocket) => void): void; +} +``` + + diff --git a/docs/benchmarks.md b/docs/benchmarks.md new file mode 100644 index 00000000000000..3d794342301449 --- /dev/null +++ b/docs/benchmarks.md @@ -0,0 +1,121 @@ +Bun.js focuses on performance, developer experience, and compatibility with the JavaScript ecosystem. + +## HTTP Requests + +```ts +// http.ts +export default { + port: 3000, + fetch(request: Request) { + return new Response("Hello World"); + }, +}; + +// bun ./http.ts +``` + +| Requests per second | OS | CPU | Bun version | +| ---------------------------------------------------------------------- | ----- | ------------------------------ | ----------- | +| [260,000](https://twitter.com/jarredsumner/status/1512040623200616449) | macOS | Apple Silicon M1 Max | 0.0.76 | +| [160,000](https://twitter.com/jarredsumner/status/1511988933587976192) | Linux | AMD Ryzen 5 3600 6-Core 2.2ghz | 0.0.76 | + +{% details summary="See benchmark details" %} +Measured with [`http_load_test`](https://github.com/uNetworking/uSockets/blob/master/examples/http_load_test.c) by running: + +```bash +$ ./http_load_test 20 127.0.0.1 3000 +``` + +{% /details %} + +## File System + +`cat` clone that runs [2x faster than GNU cat](https://twitter.com/jarredsumner/status/1511707890708586496) for large files on Linux + +```js +// cat.js +import { resolve } from "path"; +import { write, stdout, file, argv } from "bun"; + +const path = resolve(argv.at(-1)); + +await write( + // stdout is a Blob + stdout, + // file(path) returns a Blob - https://developer.mozilla.org/en-US/docs/Web/API/Blob + file(path), +); +``` + +Run this with `bun cat.js /path/to/big/file`. + +## Reading from standard input + +```ts +// As of Bun v0.3.0, console is an AsyncIterable +for await (const line of console) { + // line of text from stdin + console.log(line); +} +``` + +## React SSR + +```js +import { renderToReadableStream } from "react-dom/server"; + +const dt = new Intl.DateTimeFormat(); + +export default { + port: 3000, + async fetch(request: Request) { + return new Response( + await renderToReadableStream( + + + Hello World + + +

Hello from React!

+

The date is {dt.format(new Date())}

+ + , + ), + ); + }, +}; +``` + +Write to stdout with `console.write`: + +```js +// no trailing newline +// works with strings and typed arrays +console.write("Hello World!"); +``` + +There are some more examples in the [examples](./examples) folder. + +PRs adding more examples are very welcome! + +## Fast paths for Web APIs + +Bun.js has fast paths for common use cases that make Web APIs live up to the performance demands of servers and CLIs. + +`Bun.file(path)` returns a [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob) that represents a lazily-loaded file. + +When you pass a file blob to `Bun.write`, Bun automatically uses a faster system call: + +```js +const blob = Bun.file("input.txt"); +await Bun.write("output.txt", blob); +``` + +On Linux, this uses the [`copy_file_range`](https://man7.org/linux/man-pages/man2/copy_file_range.2.html) syscall and on macOS, this becomes `clonefile` (or [`fcopyfile`](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/copyfile.3.html)). + +`Bun.write` also supports [`Response`](https://developer.mozilla.org/en-US/docs/Web/API/Response) objects. It automatically converts to a [`Blob`](https://developer.mozilla.org/en-US/docs/Web/API/Blob). + +```js +// Eventually, this will stream the response to disk but today it buffers +await Bun.write("index.html", await fetch("https://example.com")); +``` diff --git a/docs/bun-flavored-toml.md b/docs/bun-flavored-toml.md index b041ba05586663..ab7beb5920cdf0 100644 --- a/docs/bun-flavored-toml.md +++ b/docs/bun-flavored-toml.md @@ -1,5 +1,3 @@ -# Bun-flavored TOML - [TOML](https://toml.io/) is a minimal configuration file format designed to be easy for humans to read. Bun implements a TOML parser with a few tweaks designed for better interoperability with INI files and with JavaScript. diff --git a/docs/cli/bun-completions.md b/docs/cli/bun-completions.md new file mode 100644 index 00000000000000..68dcb946d5a318 --- /dev/null +++ b/docs/cli/bun-completions.md @@ -0,0 +1,3 @@ +This command installs completions for `zsh` and/or `fish`. It runs automatically on every `bun upgrade` and on install. It reads from `$SHELL` to determine which shell to install for. It tries several common shell completion directories for your shell and OS. + +If you want to copy the completions manually, run `bun completions > path-to-file`. If you know the completions directory to install them to, run `bun completions /path/to/directory`. diff --git a/docs/cli/bun-dev.md b/docs/cli/bun-dev.md new file mode 100644 index 00000000000000..5d1adec5fe322e --- /dev/null +++ b/docs/cli/bun-dev.md @@ -0,0 +1,21 @@ +In your project folder root (where `package.json` is): + +```bash +$ bun bun ./entry-point-1.js ./entry-point-2.jsx +$ bun dev +``` + +By default, `bun dev` will look for any HTML files in the `public` directory and serve that. For browsers navigating to the page, the `.html` file extension is optional in the URL, and `index.html` will automatically rewrite for the directory. + +Here are examples of routing from `public/` and how they’re matched: +| Dev Server URL | File Path | +|----------------|-----------| +| /dir | public/dir/index.html | +| / | public/index.html | +| /index | public/index.html | +| /hi | public/hi.html | +| /file | public/file.html | +| /font/Inter.woff2 | public/font/Inter.woff2 | +| /hello | public/index.html | + +If `public/index.html` exists, it becomes the default page instead of a 404 page, unless that pathname has a file extension. diff --git a/docs/cli/bun-init.md b/docs/cli/bun-init.md new file mode 100644 index 00000000000000..3a0e6681511603 --- /dev/null +++ b/docs/cli/bun-init.md @@ -0,0 +1,20 @@ +`bun init` is a quick way to start a blank project with Bun. It guesses with sane defaults and is non-destructive when run multiple times. + +![Demo](https://user-images.githubusercontent.com/709451/183006613-271960a3-ff22-4f7c-83f5-5e18f684c836.gif) + +It creates: + +- a `package.json` file with a name that defaults to the current directory name +- a `tsconfig.json` file or a `jsconfig.json` file, depending if the entry point is a TypeScript file or not +- an entry point which defaults to `index.ts` unless any of `index.{tsx, jsx, js, mts, mjs}` exist or the `package.json` specifies a `module` or `main` field +- a `README.md` file + +If you pass `-y` or `--yes`, it will assume you want to continue without asking questions. + +At the end, it runs `bun install` to install `bun-types`. + +Added in Bun v0.1.7. + +#### How is `bun init` different than `bun create`? + +`bun init` is for blank projects. `bun create` applies templates. diff --git a/docs/cli/bun-install.md b/docs/cli/bun-install.md new file mode 100644 index 00000000000000..11cf3ee8134c11 --- /dev/null +++ b/docs/cli/bun-install.md @@ -0,0 +1,257 @@ +### `bun install` + +bun install is a fast package manager & npm client. + +bun install can be configured via `bunfig.toml`, environment variables, and CLI flags. + +#### Configuring `bun install` with `bunfig.toml` + +`bunfig.toml` is searched for in the following paths on `bun install`, `bun remove`, and `bun add`: + +1. `$XDG_CONFIG_HOME/.bunfig.toml` or `$HOME/.bunfig.toml` +2. `./bunfig.toml` + +If both are found, the results are merged together. + +Configuring with `bunfig.toml` is optional. Bun tries to be zero configuration in general, but that's not always possible. + +```toml +# Using scoped packages with bun install +[install.scopes] + +# Scope name The value can be a URL string or an object +"@mybigcompany" = { token = "123456", url = "https://registry.mybigcompany.com" } +# URL is optional and fallsback to the default registry + +# The "@" in the scope is optional +mybigcompany2 = { token = "123456" } + +# Environment variables can be referenced as a string that starts with $ and it will be replaced +mybigcompany3 = { token = "$npm_config_token" } + +# Setting username and password turns it into a Basic Auth header by taking base64("username:password") +mybigcompany4 = { username = "myusername", password = "$npm_config_password", url = "https://registry.yarnpkg.com/" } +# You can set username and password in the registry URL. This is the same as above. +mybigcompany5 = "https://username:password@registry.yarnpkg.com/" + +# You can set a token for a registry URL: +mybigcompany6 = "https://:$NPM_CONFIG_TOKEN@registry.yarnpkg.com/" + +[install] +# Default registry +# can be a URL string or an object +registry = "https://registry.yarnpkg.com/" +# as an object +#registry = { url = "https://registry.yarnpkg.com/", token = "123456" } + +# Install for production? This is the equivalent to the "--production" CLI argument +production = false + +# Don't actually install +dryRun = true + +# Install optionalDependencies (default: true) +optional = true + +# Install local devDependencies (default: true) +dev = true + +# Install peerDependencies (default: false) +peer = false + +# When using `bun install -g`, install packages here +globalDir = "~/.bun/install/global" + +# When using `bun install -g`, link package bins here +globalBinDir = "~/.bun/bin" + +# cache-related configuration +[install.cache] +# The directory to use for the cache +dir = "~/.bun/install/cache" + +# Don't load from the global cache. +# Note: Bun may still write to node_modules/.cache +disable = false + + +# Always resolve the latest versions from the registry +disableManifest = false + + +# Lockfile-related configuration +[install.lockfile] + +# Print a yarn v1 lockfile +# Note: it does not load the lockfile, it just converts bun.lockb into a yarn.lock +print = "yarn" + +# Path to read bun.lockb from +path = "bun.lockb" + +# Path to save bun.lockb to +savePath = "bun.lockb" + +# Save the lockfile to disk +save = true + +``` + +If it's easier to read as TypeScript types: + +```ts +export interface Root { + install: Install; +} + +export interface Install { + scopes: Scopes; + registry: Registry; + production: boolean; + dryRun: boolean; + optional: boolean; + dev: boolean; + peer: boolean; + globalDir: string; + globalBinDir: string; + cache: Cache; + lockfile: Lockfile; + logLevel: "debug" | "error" | "warn"; +} + +type Registry = + | string + | { + url?: string; + token?: string; + username?: string; + password?: string; + }; + +type Scopes = Record; + +export interface Cache { + dir: string; + disable: boolean; + disableManifest: boolean; +} + +export interface Lockfile { + print?: "yarn"; + path: string; + savePath: string; + save: boolean; +} +``` + +## Configuring with environment variables + +Environment variables have a higher priority than `bunfig.toml`. + +| Name | Description | +| -------------------------------- | ------------------------------------------------------------- | +| BUN_CONFIG_REGISTRY | Set an npm registry (default: ) | +| BUN_CONFIG_TOKEN | Set an auth token (currently does nothing) | +| BUN_CONFIG_LOCKFILE_SAVE_PATH | File path to save the lockfile to (default: bun.lockb) | +| BUN_CONFIG_YARN_LOCKFILE | Save a Yarn v1-style yarn.lock | +| BUN_CONFIG_LINK_NATIVE_BINS | Point `bin` in package.json to a platform-specific dependency | +| BUN_CONFIG_SKIP_SAVE_LOCKFILE | Don’t save a lockfile | +| BUN_CONFIG_SKIP_LOAD_LOCKFILE | Don’t load a lockfile | +| BUN_CONFIG_SKIP_INSTALL_PACKAGES | Don’t install any packages | + +Bun always tries to use the fastest available installation method for the target platform. On macOS, that’s `clonefile` and on Linux, that’s `hardlink`. You can change which installation method is used with the `--backend` flag. When unavailable or on error, `clonefile` and `hardlink` fallsback to a platform-specific implementation of copying files. + +Bun stores installed packages from npm in `~/.bun/install/cache/${name}@${version}`. Note that if the semver version has a `build` or a `pre` tag, it is replaced with a hash of that value instead. This is to reduce the chances of errors from long file paths, but unfortunately complicates figuring out where a package was installed on disk. + +When the `node_modules` folder exists, before installing, Bun checks if the `"name"` and `"version"` in `package/package.json` in the expected node_modules folder matches the expected `name` and `version`. This is how it determines whether it should install. It uses a custom JSON parser which stops parsing as soon as it finds `"name"` and `"version"`. + +When a `bun.lockb` doesn’t exist or `package.json` has changed dependencies, tarballs are downloaded & extracted eagerly while resolving. + +When a `bun.lockb` exists and `package.json` hasn’t changed, Bun downloads missing dependencies lazily. If the package with a matching `name` & `version` already exists in the expected location within `node_modules`, Bun won’t attempt to download the tarball. + +## Platform-specific dependencies? + +bun stores normalized `cpu` and `os` values from npm in the lockfile, along with the resolved packages. It skips downloading, extracting, and installing packages disabled for the current target at runtime. This means the lockfile won’t change between platforms/architectures even if the packages ultimately installed do change. + +## Peer dependencies? + +Peer dependencies are handled similarly to yarn. `bun install` does not automatically install peer dependencies and will try to choose an existing dependency. + +## Lockfile + +`bun.lockb` is Bun’s binary lockfile format. + +## Why is it binary? + +In a word: Performance. Bun’s lockfile saves & loads incredibly quickly, and saves a lot more data than what is typically inside lockfiles. + +## How do I inspect it? + +For now, the easiest thing is to run `bun install -y`. That prints a Yarn v1-style yarn.lock file. + +## What does the lockfile store? + +Packages, metadata for those packages, the hoisted install order, dependencies for each package, what packages those dependencies resolved to, an integrity hash (if available), what each package was resolved to and which version (or equivalent). + +## Why is it fast? + +It uses linear arrays for all data. [Packages](https://github.com/oven-sh/bun/blob/be03fc273a487ac402f19ad897778d74b6d72963/src/install/install.zig#L1825) are referenced by an auto-incrementing integer ID or a hash of the package name. Strings longer than 8 characters are de-duplicated. Prior to saving on disk, the lockfile is garbage-collected & made deterministic by walking the package tree and cloning the packages in dependency order. + +## Cache + +To delete the cache: + +```bash +$ rm -rf ~/.bun/install/cache +``` + +## Platform-specific backends + +`bun install` uses different system calls to install dependencies depending on the platform. This is a performance optimization. You can force a specific backend with the `--backend` flag. + +**`hardlink`** is the default backend on Linux. Benchmarking showed it to be the fastest on Linux. + +```bash +$ rm -rf node_modules +$ bun install --backend hardlink +``` + +**`clonefile`** is the default backend on macOS. Benchmarking showed it to be the fastest on macOS. It is only available on macOS. + +```bash +$ rm -rf node_modules +$ bun install --backend clonefile +``` + +**`clonefile_each_dir`** is similar to `clonefile`, except it clones each file individually per directory. It is only available on macOS and tends to perform slower than `clonefile`. Unlike `clonefile`, this does not recursively clone subdirectories in one system call. + +```bash +$ rm -rf node_modules +$ bun install --backend clonefile_each_dir +``` + +**`copyfile`** is the fallback used when any of the above fail, and is the slowest. on macOS, it uses `fcopyfile()` and on linux it uses `copy_file_range()`. + +```bash +$ rm -rf node_modules +$ bun install --backend copyfile +``` + +**`symlink`** is typically only used for `file:` dependencies (and eventually `link:`) internally. To prevent infinite loops, it skips symlinking the `node_modules` folder. + +If you install with `--backend=symlink`, Node.js won't resolve node_modules of dependencies unless each dependency has its own node_modules folder or you pass `--preserve-symlinks` to `node`. See [Node.js documentation on `--preserve-symlinks`](https://nodejs.org/api/cli.html#--preserve-symlinks). + +```bash +$ rm -rf node_modules +$ bun install --backend symlink +$ node --preserve-symlinks ./my-file.js # https://nodejs.org/api/cli.html#--preserve-symlinks +``` + +Bun's runtime does not currently expose an equivalent of `--preserve-symlinks`, though the code for it does exist. + +## npm registry metadata + +bun uses a binary format for caching NPM registry responses. This loads much faster than JSON and tends to be smaller on disk. +You will see these files in `~/.bun/install/cache/*.npm`. The filename pattern is `${hash(packageName)}.npm`. It’s a hash so that extra directories don’t need to be created for scoped packages. + +Bun's usage of `Cache-Control` ignores `Age`. This improves performance, but means bun may be about 5 minutes out of date to receive the latest package version metadata from npm. diff --git a/docs/cli/bun-upgrade.md b/docs/cli/bun-upgrade.md new file mode 100644 index 00000000000000..a9d0759b6a4c82 --- /dev/null +++ b/docs/cli/bun-upgrade.md @@ -0,0 +1,39 @@ +To upgrade Bun, run `bun upgrade`. + +It automatically downloads the latest version of Bun and overwrites the currently-running version. + +This works by checking the latest version of Bun in [bun-releases-for-updater](https://github.com/Jarred-Sumner/bun-releases-for-updater/releases) and unzipping it using the system-provided `unzip` library (so that Gatekeeper works on macOS) + +If for any reason you run into issues, you can also use the curl install script: + +```bash +$ curl https://bun.sh/install | bash +``` + +It will still work when Bun is already installed. + +Bun is distributed as a single binary file, so you can also do this manually: + +- Download the latest version of Bun for your platform in [bun-releases-for-updater](https://github.com/Jarred-Sumner/bun-releases-for-updater/releases/latest) (`darwin` == macOS) +- Unzip the folder +- Move the `bun` binary to `~/.bun/bin` (or anywhere) + +## `--canary` + +[Canary](https://github.com/oven-sh/bun/releases/tag/canary) builds are generated on every commit. + +To install a [canary](https://github.com/oven-sh/bun/releases/tag/canary) build of Bun, run: + +```bash +$ bun upgrade --canary +``` + +This flag is not persistent (though that might change in the future). If you want to always run the canary build of Bun, set the `BUN_CANARY` environment variable to `1` in your shell's startup script. + +This will download the release zip from https://github.com/oven-sh/bun/releases/tag/canary. + +To revert to the latest published version of Bun, run: + +```bash +$ bun upgrade +``` diff --git a/docs/cli/bundler.md b/docs/cli/bundler.md new file mode 100644 index 00000000000000..6900e92924eb70 --- /dev/null +++ b/docs/cli/bundler.md @@ -0,0 +1,116 @@ +Bundling is currently an important mechanism for building complex web apps. + +Modern apps typically consist of a large number of files and package dependencies. Despite the fact that modern browsers support [ES Module](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules) imports, it's still too slow to fetch each file via inidividual HTTP requests. _Bundling_ is the process of concatenating several source files into a single large file that can be loaded in a single request. + +{% callout %} +**On bundling** — Despite recent advances like [`modulepreload`](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel/modulepreload) and [HTTP/3](https://en.wikipedia.org/wiki/HTTP/3), bundling is still the most performant approach. +{% /callout %} + +## Bundling your app + +Bun's approach to bundling is a little different from other bundlers. Start by passing your app's entrypoint to `bun bun`. + +```bash +$ bun bun ./app.js +``` + +Your entrypoint can be any `js|jsx|ts|tsx|html` file. With this file as a starting point, Bun will construct a graph of imported files and packages, transpile everything, and generate a file called `node_modules.bun`. + +## What is `.bun`? + +{% callout %} +**Note** — [This format may change soon](https://github.com/oven-sh/bun/issues/121) +{% /callout %} + +A `.bun` file contains the pre-transpiled source code of your application, plus a bunch of binary-encoded metadata about your application's structure. as a contains: + +- all the bundled source code +- all the bundled source code metadata +- project metadata & configuration + +Here are some of the questions `.bun` files answer: + +- when I import `react/index.js`, where in the `.bun` is the code for that? (not resolving, just the code) +- what modules of a package are used? +- what framework is used? (e.g., Next.js) +- where is the routes directory? +- how big is each imported dependency? +- what is the hash of the bundle’s contents? (for etags) +- what is the name & version of every npm package exported in this bundle? +- what modules from which packages are used in this project? ("project" is defined as all the entry points used to generate the .bun) + +All in one file. + +It’s a little like a build cache, but designed for reuse across builds. + +{% details summary="Position-independent code" %} + +From a design perspective, the most important part of the `.bun` format is how code is organized. Each module is exported by a hash like this: + +```js +// preact/dist/preact.module.js +export var $eb6819b = $$m({ + "preact/dist/preact.module.js": (module, exports) => { + let n, l, u, i, t, o, r, f, e = {}, c = [], s = /acit|ex(?:s|g|n|p|$)|rph|grid|ows|mnc|ntw|ine[ch]|zoo|^ord|itera/i; + // ... rest of code +``` + +This makes bundled modules [position-independent](https://en.wikipedia.org/wiki/Position-independent_code). In theory, one could import only the exact modules in-use without reparsing code and without generating a new bundle. One bundle can dynamically become many bundles comprising only the modules in use on the webpage. Thanks to the metadata with the byte offsets, a web server can send each module to browsers [zero-copy](https://en.wikipedia.org/wiki/Zero-copy) using [sendfile](https://man7.org/linux/man-pages/man2/sendfile.2.html). Bun itself is not quite this smart yet, but these optimizations would be useful in production and potentially very useful for React Server Components. + +To see the schema inside, have a look at [`JavascriptBundleContainer`](./src/api/schema.d.ts#:~:text=export%20interface-,JavascriptBundleContainer,-%7B). You can find JavaScript bindings to read the metadata in [src/api/schema.js](./src/api/schema.js). This is not really an API yet. It’s missing the part where it gets the binary data from the bottom of the file. Someday, I want this to be usable by other tools too. +{% /details %} + +## Where is the code? + +`.bun` files are marked as executable. + +To print out the code, run `./node_modules.bun` in your terminal or run `bun ./path-to-node_modules.bun`. + +Here is a copy-pastable example: + +```bash +$ ./node_modules.bun > node_modules.js +``` + +This works because every `.bun` file starts with this: + +``` +#!/usr/bin/env bun +``` + +To deploy to production with Bun, you’ll want to get the code from the `.bun` file and stick that somewhere your web server can find it (or if you’re using Vercel or a Rails app, in a `public` folder). + +Note that `.bun` is a binary file format, so just opening it in VSCode or vim might render strangely. + +## Advanced + +By default, `bun bun` only bundles external dependencies that are `import`ed or `require`d in either app code or another external dependency. An "external dependency" is defined as, "A JavaScript-like file that has `/node_modules/` in the resolved file path and a corresponding `package.json`". + +To force Bun to bundle packages which are not located in a `node_modules` folder (i.e., the final, resolved path following all symlinks), add a `bun` section to the root project’s `package.json` with `alwaysBundle` set to an array of package names to always bundle. Here’s an example: + +```json +{ + "name": "my-package-name-in-here", + "bun": { + "alwaysBundle": ["@mybigcompany/my-workspace-package"] + } +} +``` + +Bundled dependencies are not eligible for Hot Module Reloading. The code is served to browsers & Bun.js verbatim. But, in the future, it may be sectioned off into only parts of the bundle being used. That’s possible in the current version of the `.bun` file (so long as you know which files are necessary), but it’s not implemented yet. Longer-term, it will include all `import` and `export` of each module inside. + +## What is the module ID hash? + +The `$eb6819b` hash used here: + +```js +export var $eb6819b = $$m({ +``` + +Is generated like this: + +1. Murmur3 32-bit hash of `package.name@package.version`. This is the hash uniquely identifying the npm package. +2. Wyhash 64 of the `package.hash` + `package_path`. `package_path` means "relative to the root of the npm package, where is the module imported?". For example, if you imported `react/jsx-dev-runtime.js`, the `package_path` is `jsx-dev-runtime.js`. `react-dom/cjs/react-dom.development.js` would be `cjs/react-dom.development.js` +3. Truncate the hash generated above to a `u32` + +The implementation details of this module ID hash will vary between versions of Bun. The important part is the metadata contains the module IDs, the package paths, and the package hashes, so it shouldn’t really matter in practice if other tooling wants to make use of any of this. diff --git a/docs/cli/create.md b/docs/cli/create.md new file mode 100644 index 00000000000000..8542db68b90f91 --- /dev/null +++ b/docs/cli/create.md @@ -0,0 +1,232 @@ +Template a new Bun project with `bun create`. + +```bash +$ bun create