ci: cache dependencies in worker pipelines

[wangbill (YunchuWang)]

Description

• Cache pip installed packages in major pipelines which get invalidated if setup.py changes or date is different.
• Test run pipelines - on average saved over 1min pipeline runtime per run

---

PR information

• The title of the PR is clear and informative.
• There are a small number of commits, each of which has an informative message. This means that previously merged commits do not appear in the history of the PR. For information on cleaning up the commits in your pull request, see this page.
• If applicable, the PR references the bug/issue that it fixes in the description.
• New Unit tests were added for the changes made and CI is passing.

Quality of Code and Contribution Guidelines

• I have read the contribution guidelines.

[Codecov Comments Bot (codecov-commenter)]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.89%. Comparing base (0913d9c) to head (8c73d1a).
Report is 101 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1467      +/-   ##
==========================================
- Coverage   85.23%   84.89%   -0.35%     
==========================================
  Files          35       35              
  Lines        1998     1986      -12     
  Branches      378      293      -85     
==========================================
- Hits         1703     1686      -17     
- Misses        219      224       +5     
  Partials       76       76              

Flag	Coverage Δ
unittests	84.89% <ø> (-0.30%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.