Skip to content

Commit bb72b09

Browse files
Jack-KhuuJack-Khuu
authored
Multi Pin Bumps across PT/AO/tune/ET: pt dev20241213 (#1367)
* Bump PyTorch pin to 20241111 * bump to 1112 * Update install_requirements.sh * Update install_requirements.sh * Update checkpoint.py typo * Update install_requirements.sh * Update install_requirements.sh * Update install_requirements.sh * Bump pins, waiting for nvjit fix * Update install_requirements.sh * bump tune * fix tune major version * Bump AO pin to pick up import fix * misc * Update linux_job CI to v2 * Update install_requirements.sh PT pin to 1202 * Vision nightly is delayed * Bump Cuda version; drop PT version to one with vision nightly * Bump to 1205 vision nightly * Vision nightly 1205 needs 1204 torch(?) * Drop PT version to 1126 (friendly vision version), update devtoolset to 11 for almalinux * Test download toolchain instead of binutils * Test removing devtoolset * Remove dep on devtoolset 11 that doesnt' exist on the new machine * Bump ET pin * Test nightly with updated vision * Attempt to account for int4wo packing pt#139611 * Naive gguf int4wo attempt * Update install_requirements.sh to 1210 * Update install_requirements.sh to 20241213 Should fix the MacOS wheel regression * Update torchvision minor version to 22
1 parent 570aebc commit bb72b09

File tree

11 files changed

+74
-166
lines changed

11 files changed

+74
-166
lines changed

‎.github/workflows/more-tests.yml

Lines changed: 2 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -9,23 +9,17 @@ on:
99

1010
jobs:
1111
test-cuda:
12-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
12+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
1313
with:
1414
runner: linux.g5.4xlarge.nvidia.gpu
1515
gpu-arch-type: cuda
16-
gpu-arch-version: "12.1"
16+
gpu-arch-version: "12.4"
1717
timeout: 60
1818
script: |
1919
echo "::group::Print machine info"
2020
uname -a
2121
echo "::endgroup::"
2222
23-
echo "::group::Install newer objcopy that supports --set-section-alignment"
24-
yum install -y devtoolset-10-binutils
25-
export PATH=/opt/rh/devtoolset-10/root/usr/bin/:$PATH
26-
echo "::endgroup::"
27-
28-
2923
echo "::group::Download checkpoints"
3024
# Install requirements
3125
./install/install_requirements.sh cuda

‎.github/workflows/periodic.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,7 @@ jobs:
108108
set -eux
109109
PYTHONPATH="${PWD}" python .ci/scripts/gather_test_models.py --event "periodic" --backend "gpu"
110110
test-gpu:
111-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
111+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
112112
name: test-gpu (${{ matrix.platform }}, ${{ matrix.model_name }})
113113
needs: gather-models-gpu
114114
secrets: inherit
@@ -119,7 +119,7 @@ jobs:
119119
secrets-env: "HF_TOKEN_PERIODIC"
120120
runner: ${{ matrix.runner }}
121121
gpu-arch-type: cuda
122-
gpu-arch-version: "12.1"
122+
gpu-arch-version: "12.4"
123123
script: |
124124
echo "::group::Print machine info"
125125
nvidia-smi

‎.github/workflows/pull.yml

Lines changed: 11 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -215,7 +215,7 @@ jobs:
215215
set -eux
216216
PYTHONPATH="${PWD}" python .ci/scripts/gather_test_models.py --event "pull_request" --backend "gpu"
217217
test-gpu-compile:
218-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
218+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
219219
name: test-gpu-compile (${{ matrix.platform }}, ${{ matrix.model_name }})
220220
needs: gather-models-gpu
221221
strategy:
@@ -224,7 +224,7 @@ jobs:
224224
with:
225225
runner: linux.g5.4xlarge.nvidia.gpu
226226
gpu-arch-type: cuda
227-
gpu-arch-version: "12.1"
227+
gpu-arch-version: "12.4"
228228
script: |
229229
echo "::group::Print machine info"
230230
nvidia-smi
@@ -250,7 +250,7 @@ jobs:
250250
echo "::endgroup::"
251251
252252
test-gpu-aoti-bfloat16:
253-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
253+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
254254
name: test-gpu-aoti-bfloat16 (${{ matrix.platform }}, ${{ matrix.model_name }})
255255
needs: gather-models-gpu
256256
strategy:
@@ -259,18 +259,13 @@ jobs:
259259
with:
260260
runner: linux.g5.4xlarge.nvidia.gpu
261261
gpu-arch-type: cuda
262-
gpu-arch-version: "12.1"
262+
gpu-arch-version: "12.4"
263263
timeout: 60
264264
script: |
265265
echo "::group::Print machine info"
266266
nvidia-smi
267267
echo "::endgroup::"
268268
269-
echo "::group::Install newer objcopy that supports --set-section-alignment"
270-
yum install -y devtoolset-10-binutils
271-
export PATH=/opt/rh/devtoolset-10/root/usr/bin/:$PATH
272-
echo "::endgroup::"
273-
274269
echo "::group::Install required packages"
275270
./install/install_requirements.sh cuda
276271
pip3 list
@@ -291,7 +286,7 @@ jobs:
291286
echo "::endgroup::"
292287
293288
test-gpu-aoti-float32:
294-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
289+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
295290
name: test-gpu-aoti-float32 (${{ matrix.platform }}, ${{ matrix.model_name }})
296291
needs: gather-models-gpu
297292
strategy:
@@ -300,17 +295,12 @@ jobs:
300295
with:
301296
runner: linux.g5.4xlarge.nvidia.gpu
302297
gpu-arch-type: cuda
303-
gpu-arch-version: "12.1"
298+
gpu-arch-version: "12.4"
304299
script: |
305300
echo "::group::Print machine info"
306301
nvidia-smi
307302
echo "::endgroup::"
308303
309-
echo "::group::Install newer objcopy that supports --set-section-alignment"
310-
yum install -y devtoolset-10-binutils
311-
export PATH=/opt/rh/devtoolset-10/root/usr/bin/:$PATH
312-
echo "::endgroup::"
313-
314304
echo "::group::Install required packages"
315305
./install/install_requirements.sh cuda
316306
pip list
@@ -337,7 +327,7 @@ jobs:
337327
echo "::endgroup::"
338328
339329
test-gpu-aoti-float16:
340-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
330+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
341331
name: test-gpu-aoti-float16 (${{ matrix.platform }}, ${{ matrix.model_name }})
342332
needs: gather-models-gpu
343333
strategy:
@@ -346,17 +336,12 @@ jobs:
346336
with:
347337
runner: linux.g5.4xlarge.nvidia.gpu
348338
gpu-arch-type: cuda
349-
gpu-arch-version: "12.1"
339+
gpu-arch-version: "12.4"
350340
script: |
351341
echo "::group::Print machine info"
352342
nvidia-smi
353343
echo "::endgroup::"
354344
355-
echo "::group::Install newer objcopy that supports --set-section-alignment"
356-
yum install -y devtoolset-10-binutils
357-
export PATH=/opt/rh/devtoolset-10/root/usr/bin/:$PATH
358-
echo "::endgroup::"
359-
360345
echo "::group::Install required packages"
361346
./install/install_requirements.sh cuda
362347
pip list
@@ -384,7 +369,7 @@ jobs:
384369
echo "::endgroup::"
385370
386371
test-gpu-eval-sanity-check:
387-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
372+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
388373
name: test-gpu-eval-sanity-check (${{ matrix.platform }}, ${{ matrix.model_name }})
389374
needs: gather-models-gpu
390375
strategy:
@@ -393,17 +378,12 @@ jobs:
393378
with:
394379
runner: linux.g5.4xlarge.nvidia.gpu
395380
gpu-arch-type: cuda
396-
gpu-arch-version: "12.1"
381+
gpu-arch-version: "12.4"
397382
script: |
398383
echo "::group::Print machine info"
399384
nvidia-smi
400385
echo "::endgroup::"
401386
402-
echo "::group::Install newer objcopy that supports --set-section-alignment"
403-
yum install -y devtoolset-10-binutils
404-
export PATH=/opt/rh/devtoolset-10/root/usr/bin/:$PATH
405-
echo "::endgroup::"
406-
407387
echo "::group::Install required packages"
408388
./install/install_requirements.sh cuda
409389
pip3 list
@@ -1031,7 +1011,7 @@ jobs:
10311011
echo "Tests complete."
10321012
10331013
test-build-runner-et-android:
1034-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
1014+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
10351015
with:
10361016
runner: linux.4xlarge
10371017
script: |

‎.github/workflows/run-readme-periodic.yml

Lines changed: 6 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -10,24 +10,19 @@ on:
1010

1111
jobs:
1212
test-readme:
13-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
13+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
1414
secrets: inherit
1515
with:
1616
runner: linux.g5.4xlarge.nvidia.gpu
1717
secrets-env: "HF_TOKEN_PERIODIC"
1818
gpu-arch-type: cuda
19-
gpu-arch-version: "12.1"
19+
gpu-arch-version: "12.4"
2020
timeout: 60
2121
script: |
2222
echo "::group::Print machine info"
2323
uname -a
2424
echo "::endgroup::"
2525
26-
echo "::group::Install newer objcopy that supports --set-section-alignment"
27-
yum install -y devtoolset-10-binutils
28-
export PATH=/opt/rh/devtoolset-10/root/usr/bin/:$PATH
29-
echo "::endgroup::"
30-
3126
echo "::group::Create script to run README"
3227
python3 torchchat/utils/scripts/updown.py --create-sections --file README.md > ./run-readme.sh
3328
# for good measure, if something happened to updown processor,
@@ -44,23 +39,18 @@ jobs:
4439
4540
4641
test-quantization-any:
47-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
42+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
4843
with:
4944
runner: linux.g5.4xlarge.nvidia.gpu
5045
secrets: inherit
5146
gpu-arch-type: cuda
52-
gpu-arch-version: "12.1"
47+
gpu-arch-version: "12.4"
5348
timeout: 60
5449
script: |
5550
echo "::group::Print machine info"
5651
uname -a
5752
echo "::endgroup::"
5853
59-
echo "::group::Install newer objcopy that supports --set-section-alignment"
60-
yum install -y devtoolset-10-binutils
61-
export PATH=/opt/rh/devtoolset-10/root/usr/bin/:$PATH
62-
echo "::endgroup::"
63-
6454
echo "::group::Create script to run quantization"
6555
python3 torchchat/utils/scripts/updown.py --create-sections --file docs/quantization.md > ./run-quantization.sh
6656
# for good measure, if something happened to updown processor,
@@ -76,24 +66,19 @@ jobs:
7666
echo "::endgroup::"
7767
7868
test-gguf-any:
79-
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
69+
uses: pytorch/test-infra/.github/workflows/linux_job_v2.yml@main
8070
secrets: inherit
8171
with:
8272
runner: linux.g5.4xlarge.nvidia.gpu
8373
secrets-env: "HF_TOKEN_PERIODIC"
8474
gpu-arch-type: cuda
85-
gpu-arch-version: "12.1"
75+
gpu-arch-version: "12.4"
8676
timeout: 60
8777
script: |
8878
echo "::group::Print machine info"
8979
uname -a
9080
echo "::endgroup::"
9181
92-
echo "::group::Install newer objcopy that supports --set-section-alignment"
93-
yum install -y devtoolset-10-binutils
94-
export PATH=/opt/rh/devtoolset-10/root/usr/bin/:$PATH
95-
echo "::endgroup::"
96-
9782
echo "::group::Create script to run gguf"
9883
python3 torchchat/utils/scripts/updown.py --file docs/GGUF.md > ./run-gguf.sh
9984
# for good measure, if something happened to updown processor,

0 commit comments

Comments
 (0)