[RELEASE] Release version 2.16.0 #4771
Description
Release OpenSearch and OpenSearch Dashboards 2.16.0
I noticed that a manifest was automatically created in manifests/2.16.0. Please follow the following checklist to make a release.
How to use this issue
This Release Issue
This issue captures the state of the OpenSearch release, its assignee (Release Manager) is responsible for driving the release. Please contact them or @mention them on this issue for help. There are linked issues on components of the release where individual components can be tracked. For more information check the the Release Process OpenSearch Guide.
Please refer to the following link for the release version dates: Release Schedule and Maintenance Policy.
Entrance Criteria
| Criteria | Status | Description | Comments |
|---|---|---|---|
| Documentation draft PRs are up and in tech review for all component changes | 🟢 | ||
| Sanity testing is done for all components | 🟢 | ||
| Code coverage has not decreased (all new code has tests) | 🟢 | ||
| Release notes are ready and available for all components | 🟢 | ||
| Roadmap is up-to-date (information is available to create release highlights) | 🟢 | 2.16.0 Roadmap | |
| Release ticket is cut, and there's a forum post announcing the start of the window | 🟢 | ||
| Any necessary security reviews are complete | 🟢 |
OpenSearch 2.16.0 exit criteria status:
| Criteria | Status | Description | Comments |
|---|---|---|---|
| Performance tests are run, results are posted to the release ticket and there no unexpected regressions | 🟢 | ||
| No unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days | 🟢 | ||
| Documentation has been fully reviewed and signed off by the documentation community. | 🟢 | ||
| All integration tests are passing | 🟢 | Current Status: #4771 (comment) as of 2024/08/07 | |
| Release blog is ready | 🟢 | Related PR's opensearch-project/project-website#3140 |
OpenSearch-Dashboards 2.16.0 exit criteria status:
| Criteria | Status | Description | Comments |
|---|---|---|---|
| Documentation has been fully reviewed and signed off by the documentation community | 🟢 | ||
| No unpatched vulnerabilities of medium or higher severity that have been publicly known for more than 60 days | 🟢 | scan_docker_image.txt | |
| All integration tests are passing | 🟢 | Current Status: #4771 (comment) of 2024/08/07 | |
| Release blog is ready | 🟢 | Related PR's opensearch-project/project-website#3140 |
Preparation
- Release manager assigned.
- Update release page on the website with release manager and release issue details. Related PR: [Release 2.16.0] Update the website with 2.16.0 release information project-website#3073
- Existence of label in each component repo. For more information check the release-label section.
- Increase the build frequency. Related PR [Release 2.16.0] Increase the build frequency and onboard the test manifests. #4853
- Release Issue.
- New plugins:
* Query Insights plugin: [On-boarding]: Query Insights plugin #4826
Campaigns
Release Branch and Version Increment - Starts 2024/07/19
- Core Release Branch.
- Core Version Increment.
* [Forwardport] Bump 2.x to 2.16.0 (#14216) OpenSearch#14227
* Bump 2.x to 2.16.0 OpenSearch#14216
* Bump 2.x to 2.16.0 OpenSearch-Dashboards#7012 - Components Release Branch.
- Components Version Increment.
* OpenSearch Plugins
* OpenSearch Dashboards Plugins
Feature Freeze - Ends 2024/07/22
- OpenSearch / OpenSearch-Dashboards core and components teams finalize their features.
Code Complete - Ends 2024/07/22 Ends 2024/07/23
- Mark this as done once the Code Complete is reviewed.
- Create/Verify pull requests to add each component to relase input manifests/2.16.0/opensearch-2.16.0.yml and manifests/2.16.0/opensearch-dashboards-2.16.0.yml.
Release Candidate Creation and Testing - Starts 2024/07/23 Starts 2024/07/24 after 4PM PDT
- Generate Release Candidate.
- RC1 Generate on 2024/07/24: [RELEASE] Release version 2.16.0 #4771 (comment)
- RC2 Generate on 2024/07/25: [RELEASE] Release version 2.16.0 #4771 (comment)
- RC3 2024/07/26: [RELEASE] Release version 2.16.0 #4771 (comment)
- RC4 2024/07/30: [RELEASE] Release version 2.16.0 #4771 (comment)
- RC5 2024/07/30 Started at 5PM PDT: [RELEASE] Release version 2.16.0 #4771 (comment).
- RC6 2024/08/1 Started at 7:30PM PDT [RELEASE] Release version 2.16.0 #4771 (comment).
- RC7 2024/08/2 Started at 8:15 PM PST [RELEASE] Release version 2.16.0 #4771 (comment).
- RC8 2024/08/5 Started at 12:30 PM PST [RELEASE] Release version 2.16.0 #4771 (comment).
- RC9 Started at 2024/08/06 5:00 PM PST [RELEASE] Release version 2.16.0 #4771 (comment)
- Integ Test TAR.
- Integ Test RPM.
- Docker Build and Scan.
- Backwards Compatibility Tests.
- Windows Integration Test.
- Broadcast and Communication.
- Release Candidate Lock.
Performance testing validation - Starts 2024/07/23
- Post the benchmark-tests
- Longevity tests do not show any issues.
Pre Release - 2024/08/05 2024/08/07
- Release Labeled Issues.
- Go or No-Go.
- Promote Repos. Part of the central promotion workflow, Related build link
- Promote artifacts. Part of the central promotion workflow, Related build link
- Release Notes. Related PR for consolidated release notes [Release 2.16.0] Consolidated release notes #4917
Release - 2024/08/05 2024/08/07
- Maven Promotion. https://build.ci.opensearch.org/job/publish-to-maven/37/console
- Docker Promotion.
- Release Validation.
- Collaboration with the Project Management Team.
Release Checklist.
Release Checklist
Pre-Release activities
Part of the central promotion workflow, Related build link
- Promote Repos.
-
- OS
-
- OSD
-
- Promote Artifacts.
-
- Windows
-
- Linux Debian
-
- Linux RPM
-
- Linux TAR
-
- Consolidated Release Notes.
Release activities
- Docker Promotion.
- Release Validation part 1.
-
- OpenSearch and OpenSearch Dashboard Validation.
-
- Validate the native plugin installation.
-
- Merge consolidated release notes PR. Related PR: [Release 2.16.0] Consolidated release notes #4917
- Website and Documentation Changes. Related PR: [Release] 2.16 Release artifact volley project-website#3140
-
- Merge staging website PR. ([Release] 2.16 Release artifact volley project-website#3140)
-
- Promote the website changes to prod. ([prod] 2.16 release project-website#3149 (review), Add 2.16 to version selector on main documentation-website#7941)
-
- Add website alert.
-
- Release Validation part 2. (Part of the central promotion workflow, Related build link)
-
- Validate the artifact download URL's and signatures.
-
- Release Validation part 3. (Part of the central promotion workflow, Related build link)
-
- Trigger the validation build (Search for
Completed validation for <>in the logs).
- Trigger the validation build (Search for
-
- Maven Promotion. (Part of the central promotion workflow, Related build link)
- Publish blog posts.
- Advertise on Social Media.
- Post on public slack and Github Release issue.
Post-Release activities
- Release Tags.
OpenSearch: https://build.ci.opensearch.org/job/distribution-release-tag-creation/115/console
OpenSearch Dashboards: https://build.ci.opensearch.org/job/distribution-release-tag-creation/116/console - Input Manifest Update.
Related Workflow: https://build.ci.opensearch.org/job/release-manifest-commit-lock/49/
Related PR: [2.16.0] Manifest Commit Lock with action UPDATE_TO_TAGS #4928 - Decrease the Build Frequency.
- OpenSearch Build Release notes. [Release 2.16.0] Consolidated release notes #4917
- Retrospective Issue. [Retrospective] Release Version 2.16.0 #4847
- Helm and Ansible Playbook release.
Helm: [Release 2.16.0] Helm charts update helm-charts#569
Ansible: [Release 2.16.0] Update the OS and OSD version to 2.16.0 ansible-playbook#165 - Upcoming Release Preparation.
Post Release
- Release Tags.
OpenSearch: https://build.ci.opensearch.org/job/distribution-release-tag-creation/115/console
OpenSearch Dashboards: https://build.ci.opensearch.org/job/distribution-release-tag-creation/116/console - Input Manifest Update.
Related Workflow: https://build.ci.opensearch.org/job/release-manifest-commit-lock/49/
Related PR: [2.16.0] Manifest Commit Lock with action UPDATE_TO_TAGS #4928 - OpenSearch Build Release notes. [Release 2.16.0] Consolidated release notes #4917
- Decrease the Build Frequency.
- Retrospective Issue. [Retrospective] Release Version 2.16.0 #4847
- Helm and Ansible Playbook release.
Helm: [Release 2.16.0] Helm charts update helm-charts#569
Ansible: [Release 2.16.0] Update the OS and OSD version to 2.16.0 ansible-playbook#165 - Upcoming Release Preparation.
Components
OpenSearch
OpenSearch Dashboards
Documentation
| Component | Issues tagged with 2.16.0 |
|---|---|
| https://github.com/opensearch-project/documentation-website/ |   |
Legend
| Symbol | Meaning |
|---|---|
| 🟢 | On track with overall release |
| 🟡 | Missed last milestone |
| 🔴 | Missed multiple milestones |
Related Issues and PR's
- Remove dependency on opensearch build repo libs from custom build.sh security-dashboards-plugin#2033
- Switch 2.16.0 manifest to use al2 docker images #4858
- [Deprecation] Properly deprecate CentOS7 as CI build image/Supported OS and switch to Almalinux8 #4379
- Adding OS matrix changelog and remove CentOS7 as it is approaching EOL documentation-website#7287
- [BUG] Incremental build copy the entire distribution folder from S3 without cleaning up component zips that requires rebuild opensearch-build-libraries#455
- [BUG] Maven failed publishPluginZipPublicationToZipStagingRepository alerting#1599
- [3.0.0][Bug] Jarhell when installing the plugin zip on opensearch core security-analytics#1165
- add task resource tracking service to cluster service OpenSearch#14681
- sync bug fixes from core to the plugin repo query-insights#13
- Move query categorization changes to plugin query-insights#16
- Fix build failure due to patch conflict k-NN#1851
- [Backport 2.x] Ioc upload integ tests and fix update security-analytics#1169
- [BUG] Maven failed publishPluginZipPublicationToZipStagingRepository alerting#1599
- [3.0.0][Bug] Jarhell when installing the plugin zip on opensearch core security-analytics#1165
- remove redundant dependencies alerting#1571
- [Backport 2.x] Fixing build script to only publish alerting zip alerting#1609 (comment)
- [AUTO] Increment version to 2.16.0-SNAPSHOT performance-analyzer#669
- [AUTO] Increment version to 2.16.0.0 index-management-dashboards-plugin#1089
- [AUTO] Increment version to 2.16.0-SNAPSHOT performance-analyzer-rca#559
- [AUTO] Increment version to 2.16.0.0 dashboards-notifications#224 (comment)
- Upgrading org.bouncycastle:bcprov-jdk15to18 to 1.78 performance-analyzer#686
- Add SortResponseProcessor to Search Pipelines OpenSearch#14785
- Add SplitResponseProcessor to Search Pipelines OpenSearch#14800
- Add support for Lucene inbuilt Scalar Quantizer k-NN#1848
- Create listener to refresh search thread resource usage OpenSearch#14832
- [Remote Routing Table] Implement write and read flow for shard diff file. OpenSearch#14684
- Writable warm relocation recovery OpenSearch#14670
- add initial MLInferenceSearchResponseProcessor ml-commons#2688
- [RFC] ML Inference Processors ml-commons#2173
- Add initial search request inference processor ml-commons#2616
- [Remote Routing Table] Implement write and read flow for shard diff file. OpenSearch#14684
- [Backport 2.16][Backport 2.x] Refactor remote-routing-table service inline with remote state interfaces (#14668) OpenSearch#14886
- [Discover-next] add query enhancements plugin as a core plugin OpenSearch-Dashboards#7212
- [Discover-next] toggle enhancements OpenSearch-Dashboards#7314
- [Discover-next] remove plugin config toggle for data enhancements OpenSearch-Dashboards#7291
- [Discover-Next] Adding datasources support for dataframes OpenSearch-Dashboards#7106
- [WIP] Dataset navigator OpenSearch-Dashboards#7348
- New Query Enhancement UI OpenSearch-Dashboards#7309
- Fix add filter for numeric scripted field OpenSearch-Dashboards#7022
- [Backport 2.x] [Discover-next] (QueryEditorExtensions) change isEnabled to an observable OpenSearch-Dashboards#7193
- [Backport 2.x] [Discover-next] add datasource container OpenSearch-Dashboards#7164
- Move TopNavLinks to new Nav Bar Discover OpenSearch-Dashboards#7326
- [WIP] Async Queries Followup OpenSearch-Dashboards#6966
- Track progress updates in async querying interceptors OpenSearch-Dashboards#7337
- Add Scripted Field Support in Vis Builder OpenSearch-Dashboards#6974
- [VisBuilder][BUG] Flat render structure in Metric and Table Vis OpenSearch-Dashboards#6674
- [VisBuilder] Add Capability to generate dynamic vega OpenSearch-Dashboards#7288
- [Look&Feel] apply missing pattern guidance to Alerting experience alerting-dashboards-plugin#1004
- Make reroute iteration time-bound for large shard allocations OpenSearch#14848
- Update nextUrl validation to incorporate serverBasePath security-dashboards-plugin#2048
- Make reroute iteration time-bound for large shard allocations OpenSearch#14848
- [Remote Routing Table] Implement write and read flow for shard diff file. OpenSearch#14684
- [Backport 2.x] Star tree mapping changes OpenSearch#14655
- Updating Cluster Stats Optimisation Versions to 2.16 OpenSearch#14914
- [Backport 2.x] Use Lucene provided Persian stem OpenSearch#14894
- [Backport 2.16] Optimising hashmap.get call inside AwarenessAllocationDecider OpenSearch#14920
- Fix IngestServiceTests.testBulkRequestExecutionWithFailures OpenSearch#14918
- [Backport 2.16] Bump braces from 3.0.2 to 3.0.3 dashboards-query-workbench#361
- [Backport 2.16] Fix IngestServiceTests.testBulkRequestExecutionWithFailures OpenSearch#14927
- [Backport 2.16] Optmising AwarenessAllocationDecider for hashmap.get call (#14761) OpenSearch#14925
- [Backport 2.16] Create listener to refresh search thread resource usage (#14832) OpenSearch#14923
- [Backport 2.16] Implement write and read flow for shard diff file. OpenSearch#14922
- Optimising hashmap.get call inside AwarenessAllocationDecider OpenSearch#14761
- [Backport 2.16] Optimising hashmap.get call inside AwarenessAllocationDecider OpenSearch#14920
- [Backport 2.x] Optimising hashmap.get call inside AwarenessAllocationDecider OpenSearch#14919
- [Backport 2.x] Implement write and read flow for shard diff file. OpenSearch#14915
- [MDS] Adds datasource filter for version decoupling security-dashboards-plugin#2051
- Add initial search request inference processor ml-commons#2616
- [Backport 2.16] [MDS] Adds datasource filter for version decoupling (#2051) security-dashboards-plugin#2053
- [Backport 2.16] Make reroute iteration time-bound for large shard allocations OpenSearch#14954
- [Backport 2.16] Create new IndexInput for multi part upload OpenSearch#14937