Skip to content

Commit f90688c

Browse files
mhdawsontargos
mhdawson
authored and
targos
committed
tools: make undici updater build wasm from src
- modify updater to include all files required to rebuild wasm from what's stored in the deps/undici directory - modify updater to build wasm from source while updating Signed-off-by: Michael Dawson <[email protected]> PR-URL: #54128 Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Marco Ippolito <[email protected]> Reviewed-By: Robert Nagy <[email protected]>
1 parent b61a2f5 commit f90688c

28 files changed

+20603
-33
lines changed

‎deps/undici/src/.c8rc.json

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"all": true,
3+
"reporter": [
4+
"lcov",
5+
"text",
6+
"html",
7+
"text-summary"
8+
],
9+
"include": [
10+
"lib/**/*.js",
11+
"index.js"
12+
]
13+
}

‎deps/undici/src/.dockerignore

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# Ignore everything but the stuff following the `*` with the `!`
2+
# See https://docs.docker.com/engine/reference/builder/#dockerignore-file
3+
4+
*
5+
!package.json
6+
!lib
7+
!deps
8+
!build

‎deps/undici/src/.editorconfig

+9
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# https://editorconfig.org/
2+
3+
root = true
4+
5+
[*]
6+
indent_size = 2
7+
indent_style = space
8+
insert_final_newline = true
9+
trim_trailing_whitespace = true

‎deps/undici/src/.gitignore

+86
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
# Logs
2+
logs
3+
*.log
4+
npm-debug.log*
5+
yarn-debug.log*
6+
yarn-error.log*
7+
8+
# Runtime data
9+
pids
10+
*.pid
11+
*.seed
12+
*.pid.lock
13+
14+
# Directory for instrumented libs generated by jscoverage/JSCover
15+
lib-cov
16+
17+
# Coverage directory used by tools like istanbul
18+
coverage
19+
20+
# nyc test coverage
21+
.nyc_output
22+
23+
# Grunt intermediate storage (http://gruntjs.com/creating-plugins#storing-task-files)
24+
.grunt
25+
26+
# Bower dependency directory (https://bower.io/)
27+
bower_components
28+
29+
# node-waf configuration
30+
.lock-wscript
31+
32+
# Compiled binary addons (https://nodejs.org/api/addons.html)
33+
build/Release
34+
35+
# Dependency directories
36+
node_modules/
37+
jspm_packages/
38+
39+
# TypeScript v1 declaration files
40+
typings/
41+
42+
# Optional npm cache directory
43+
.npm
44+
45+
# Optional eslint cache
46+
.eslintcache
47+
48+
# Optional REPL history
49+
.node_repl_history
50+
51+
# Output of 'npm pack'
52+
*.tgz
53+
54+
# Yarn Integrity file
55+
.yarn-integrity
56+
57+
# dotenv environment variables file
58+
.env
59+
60+
# next.js build output
61+
.next
62+
63+
# lock files
64+
package-lock.json
65+
yarn.lock
66+
67+
# IDE files
68+
.idea
69+
.vscode
70+
71+
*0x
72+
*clinic*
73+
74+
# Fuzzing
75+
corpus/
76+
crash-*
77+
fuzz-results-*.json
78+
79+
# Bundle output
80+
undici-fetch.js
81+
/test/imports/undici-import.js
82+
83+
# .npmrc has platform specific value for windows
84+
.npmrc
85+
86+
.tap

‎deps/undici/src/.npmignore

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
*
2+
!lib/**/*
3+
!index.js
4+
!index-fetch.js
5+
6+
# The wasm files are stored as base64 strings in the corresponding .js files
7+
lib/llhttp/llhttp_simd.wasm
8+
lib/llhttp/llhttp.wasm
9+
10+
!types/**/*
11+
!index.d.ts
12+
!docs/docs/**/*
13+
!scripts/strip-comments.js

‎deps/undici/src/CODE_OF_CONDUCT.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# Code of Conduct
2+
3+
Undici is committed to upholding the Node.js Code of Conduct.
4+
5+
The Node.js Code of Conduct document can be found at
6+
https://github.com/nodejs/admin/blob/main/CODE_OF_CONDUCT.md

‎deps/undici/src/CONTRIBUTING.md

+213
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,213 @@
1+
# Contributing to Undici
2+
3+
* [Guides](#guides)
4+
* [Update `llhttp`](#update-llhttp)
5+
* [Lint](#lint)
6+
* [Test](#test)
7+
* [Coverage](#coverage)
8+
* [Releases](#releases)
9+
* [Update `WPTs`](#update-wpts)
10+
* [Building for externally shared node builtins](#external-builds)
11+
* [Benchmarks](#benchmarks
12+
* [Documentation](#documentation)
13+
* [Developer's Certificate of Origin 1.1](#developers-certificate-of-origin)
14+
* [Moderation Policy](#moderation-policy)
15+
16+
<a id="guides"></a>
17+
## Guides
18+
19+
This is a collection of guides on how to run and update `undici`, and how to run different parts of the project.
20+
21+
<a id="update-llhttp"></a>
22+
### Update `llhttp`
23+
24+
The HTTP parser used by `undici` is a WebAssembly build of [`llhttp`](https://github.com/nodejs/llhttp).
25+
26+
While the project itself provides a way to compile targeting WebAssembly, at the moment we embed the sources
27+
directly and compile the module in `undici`.
28+
29+
The `deps/llhttp/include` folder contains the C header files, while the `deps/llhttp/src` folder contains
30+
the C source files needed to compile the module.
31+
32+
The `lib/llhttp` folder contains the `.js` transpiled assets required to implement a parser.
33+
34+
The following are the steps required to perform an update.
35+
36+
#### Clone the [llhttp](https://github.com/nodejs/llhttp) project
37+
38+
```bash
39+
git clone [email protected]:nodejs/llhttp.git
40+
41+
cd llhttp
42+
```
43+
44+
#### Checkout a `llhttp` release
45+
46+
```bash
47+
git checkout <tag>
48+
```
49+
50+
#### Install the `llhttp` dependencies
51+
52+
```bash
53+
npm i
54+
```
55+
56+
#### Run the wasm build script
57+
58+
> This requires [docker](https://www.docker.com/) installed on your machine.
59+
60+
```bash
61+
npm run build-wasm
62+
```
63+
64+
#### Copy the sources to `undici`
65+
66+
```bash
67+
cp build/wasm/*.js <your-path-to-undici>/lib/llhttp/
68+
69+
cp build/wasm/*.js.map <your-path-to-undici>/lib/llhttp/
70+
71+
cp build/wasm/*.d.ts <your-path-to-undici>/lib/llhttp/
72+
73+
cp src/native/api.c src/native/http.c build/c/llhttp.c <your-path-to-undici>/deps/llhttp/src/
74+
75+
cp src/native/api.h build/llhttp.h <your-path-to-undici>/deps/llhttp/include/
76+
```
77+
78+
#### Build the WebAssembly module in `undici`
79+
80+
> This requires [docker](https://www.docker.com/) installed on your machine.
81+
82+
```bash
83+
cd <your-path-to-undici>
84+
85+
npm run build:wasm
86+
```
87+
88+
#### Commit the contents of lib/llhttp
89+
90+
Create a commit which includes all of the updated files in lib/llhttp.
91+
92+
<a id="update-wpts"></a>
93+
### Update `WPTs`
94+
95+
`undici` runs a subset of the [`web-platform-tests`](https://github.com/web-platform-tests/wpt).
96+
97+
### Requirements:
98+
- [Node core utils](https://github.com/nodejs/node-core-utils) setup with credentials.
99+
100+
To update every test, run the following commands. Typically you would only need to update the tests in a specific directory.
101+
102+
```bash
103+
git node wpt resources
104+
git node wpt interfaces
105+
git node wpt common
106+
git node wpt fetch
107+
git node wpt FileAPI
108+
git node wpt xhr
109+
git node wpt websockets
110+
git node wpt mimesniff
111+
git node wpt storage
112+
git node wpt service-workers
113+
git node wpt eventsource
114+
```
115+
116+
#### Run the tests
117+
118+
Run the tests to ensure that any new failures are marked as such.
119+
120+
You can mark tests as failing in their corresponding [status](./test/wpt/status) file.
121+
122+
```bash
123+
npm run test:wpt
124+
```
125+
126+
<a id="lint"></a>
127+
### Lint
128+
129+
```bash
130+
npm run lint
131+
```
132+
133+
<a id="test"></a>
134+
### Test
135+
136+
```bash
137+
npm run test
138+
```
139+
140+
<a id="coverage"></a>
141+
### Coverage
142+
143+
```bash
144+
npm run coverage
145+
```
146+
147+
<a id="releases"></a>
148+
### Issuing Releases
149+
150+
Release is automatic on commit to main which bumps the package.json version field.
151+
Use the "Create release PR" github action to generate a release PR.
152+
153+
<a id="external-builds"></a>
154+
### Building for externally shared node builtins
155+
156+
If you are packaging `undici` for a distro, this might help if you would like to use
157+
an unbundled version instead of bundling one in `libnode.so`.
158+
159+
To enable this, pass `EXTERNAL_PATH=/path/to/global/node_modules/undici` to `build/wasm.js`.
160+
Pass this path with `loader.js` appended to `--shared-builtin-undici/undici-path` in Node.js's `configure.py`.
161+
If building on a non-Alpine Linux distribution, you may need to also set the `WASM_CC`, `WASM_CFLAGS`, `WASM_LDFLAGS` and `WASM_LDLIBS` environment variables before running `build/wasm.js`.
162+
163+
<a id="benchmarks"></a>
164+
### Benchmarks
165+
166+
```bash
167+
cd benchmarks && npm i && npm run bench
168+
```
169+
170+
The benchmarks will be available at `http://localhost:3042`.
171+
172+
<a id="documentation"></a>
173+
### Documentation
174+
175+
```bash
176+
cd docs && npm i && npm run serve
177+
```
178+
179+
The documentation will be available at `http://localhost:3000`.
180+
181+
<a id="developers-certificate-of-origin"></a>
182+
## Developer's Certificate of Origin 1.1
183+
184+
By making a contribution to this project, I certify that:
185+
186+
* (a) The contribution was created in whole or in part by me and I
187+
have the right to submit it under the open source license
188+
indicated in the file; or
189+
190+
* (b) The contribution is based upon previous work that, to the best
191+
of my knowledge, is covered under an appropriate open source
192+
license and I have the right under that license to submit that
193+
work with modifications, whether created in whole or in part
194+
by me, under the same open source license (unless I am
195+
permitted to submit under a different license), as indicated
196+
in the file; or
197+
198+
* (c) The contribution was provided directly to me by some other
199+
person who certified (a), (b) or (c) and I have not modified
200+
it.
201+
202+
* (d) I understand and agree that this project and the contribution
203+
are public and that a record of the contribution (including all
204+
personal information I submit with it, including my sign-off) is
205+
maintained indefinitely and may be redistributed consistent with
206+
this project or the open source license(s) involved.
207+
208+
<a id="moderation-policy"></a>
209+
### Moderation Policy
210+
211+
The [Node.js Moderation Policy] applies to this project.
212+
213+
[Node.js Moderation Policy]: https://github.com/nodejs/admin/blob/main/Moderation-Policy.md

0 commit comments

Comments
 (0)