feat: add medium e2e CI job for each PR #551
Conversation
| {"Key": "GitHubPR", "Value": "${{ github.event.number }}"} | ||
| ] | ||
| e2e-large-test: |
There was a problem hiding this comment.
these are saying large runner, but I think you're starting a medium runner.
There was a problem hiding this comment.
this is our large runner -- l40sx4
There was a problem hiding this comment.
its a medium job because it does less, but the runner is large, let me adjust some of these labels
JamesKunstle
left a comment
JamesKunstle
left a comment
There was a problem hiding this comment.
I'm not against merging this as a useful test because it's much faster than the large e2e that we use in the upstream ilab repo. However, I think it'd be nicer if we called this an integration-e2e rather than an sdk test. We can isolate the SDK testing slightly differently s.t. we're testing against useful upstream data rather than tightly coupling our testing vs. the SDG output.
| - name: Update instructlab-training library | ||
| run: | | ||
| export CUDA_HOME="/usr/local/cuda" |
There was a problem hiding this comment.
please use ./scripts/install-ilab-with-cuda.sh from ilab repo. It should give you the necessary environment.
There was a problem hiding this comment.
hmmm, I don't want to install ilab from main though, this is meant to be a step away from having dependencies on ilab main and eventually we don't want any ilab dependency at all. So I'd instead like to maintain our own CUDA installation per repo perhaps? or maybe we should split the action into a install-cuda and install-ilab?
| . venv/bin/activate | ||
| ls scripts | ||
| ls ./ | ||
| ./scripts/test-sdk.sh |
There was a problem hiding this comment.
Except this line, is there any significant difference between the current e2e job and this sdk one? We may want to consolidate it under reusable action, see #563 (I don't insist you adopt it since it's not merged yet, but please put a comment so that we revisit.)
| @@ -0,0 +1,157 @@ | |||
| chat: | |||
There was a problem hiding this comment.
can we trim this file to what was actually modified from the defaults / needed for sdk tests?
There was a problem hiding this comment.
I removed chat, but generate, serve, eval, train are needed I believe
| pull_request_target: | ||
| branches: | ||
| - "main" | ||
| schedule: |
There was a problem hiding this comment.
do we need the schedule? (it was not mentioned in the PR description)
There was a problem hiding this comment.
I can remove schedule
This CI job runs the "SDK" of training in conjunction with `ilab data generate`. The goal here is to test convergence on each PR in a timely manner Signed-off-by: Charlie Doern <[email protected]>
|
@booxter looking at |
This CI job runs the "SDK" of training in conjunction with
ilab data generate. The goal here is to test convergence on each PR in a timely manner