ci: add next-gen integration tests #1661

ekexium · 2025-06-12T14:30:40Z

Blocked by the fix of the bug where start_ts = conflicting commi_ts.

Signed-off-by: ekexium <eke@fastmail.com>

cfzjywxk · 2025-06-16T12:23:36Z

@ekexium
There seems to be CI failures

ekexium · 2025-06-16T13:11:05Z

@cfzjywxk The failure is a next-gen TiKV bug. This PR will be merged after the bug is fixed. We can ignore it and finish the review first.

ti-chi-bot · 2025-06-17T13:10:59Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cfzjywxk, zyguan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [cfzjywxk,zyguan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ti-chi-bot · 2025-06-17T13:11:01Z

[LGTM Timeline notifier]

Timeline:

2025-06-17 10:42:26.184105509 +0000 UTC m=+181998.907284476: ☑️ agreed by zyguan.
2025-06-17 13:10:59.106462794 +0000 UTC m=+190911.829641776: ☑️ agreed by cfzjywxk.

* *: bump pd client (tikv#1575) Signed-off-by: Ryan Leung <rleungx@gmail.com> * OWNERS: Auto Sync OWNERS files from community membership (tikv#1576) Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> * sync etcd endpoints immediately after initializing the client (tikv#1573) Signed-off-by: Vlad Dmitriev <vldmit@gmail.com> * p-dml: resolve locks concurrently (tikv#1584) close tikv#1577 Signed-off-by: you06 <you1474600@gmail.com> * memdb: prevent iterator invalidation (tikv#1563) ref pingcap/tidb#59153 Signed-off-by: ekexium <eke@fastmail.com> * locate: refactor RegionRequestSender.SendReqCtx (tikv#1565) Signed-off-by: zyguan <zhongyangguan@gmail.com> * locate: fix the default settings of circuit breaker (tikv#1593) ref tikv/pd#8678 Signed-off-by: Ryan Leung <rleungx@gmail.com> * util: define and implement core interfaces for async api (tikv#1591) ref tikv#1586 Signed-off-by: zyguan <zhongyangguan@gmail.com> * Add a retry when getting ts from PD for validating read ts (tikv#1600) Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * pdclient: Add caller info to pd client (tikv#1516) ref tikv/pd#8593 Signed-off-by: okJiang <819421878@qq.com> * locate: fix TestTiKVClientReadTimeout (tikv#1601) Signed-off-by: zyguan <zhongyangguan@gmail.com> * *: update pd client (tikv#1605) Signed-off-by: Ryan Leung <rleungx@gmail.com> * Validate ts only for stale read (tikv#1607) ref pingcap/tidb#59402 Signed-off-by: ekexium <eke@fastmail.com> * execdetails: export scheduler write details (tikv#1606) Signed-off-by: Neil Shen <overvenus@gmail.com> * Update pd client (tikv#1615) Signed-off-by: disksing <i@disksing.com> * ci: allow use label to skip integration tests (tikv#1616) Signed-off-by: disksing <i@disksing.com> * remove useless metric tidb_tikvclient_cop_duration_seconds_bucket (tikv#1602) Signed-off-by: XuHuaiyu <391585975@qq.com> * client: implement SendRequestAsync for RPCClient (tikv#1604) ref tikv#1586 Signed-off-by: zyguan <zhongyangguan@gmail.com> * execdetails: export grpc process and wait time to time details (tikv#1614) Signed-off-by: Neil Shen <overvenus@gmail.com> Co-authored-by: Bisheng Huang <hbisheng@gmail.com> * Refine pessimistic lock related metrics and stats (tikv#1620) Signed-off-by: yibin87 <huyibin@pingcap.com> * metrics: adjust bucket count to reduce metrics data (tikv#1609) Signed-off-by: Lynn <zimu_xia@126.com> * update tidb for integration tests (tikv#1621) Signed-off-by: disksing <i@disksing.com> * support redact key in logs (tikv#1612) ref pingcap/tidb#59279 Signed-off-by: tangenta <tangenta@126.com> Co-authored-by: you06 <you1474600@gmail.com> * update integration_test/go.mod (tikv#1624) Signed-off-by: tangenta <tangenta@126.com> * memdb: introduce snapshot interface (tikv#1623) Signed-off-by: you06 <you1474600@gmail.com> Co-authored-by: ekexium <eke@fastmail.com> * pd: enable OutputMustContainAllKeyRange (tikv#1632) Signed-off-by: lhy1024 <admin@liudos.us> * Fix backoff lose info when forked (tikv#1627) ref pingcap/tidb#60271 Signed-off-by: yibin87 <huyibin@pingcap.com> * tikv: disable health-feedback in next-gen (tikv#1635) Signed-off-by: zyguan <zhongyangguan@gmail.com> * enable ts validation for normal read (tikv#1619) Signed-off-by: ekexium <eke@fastmail.com> * Add txn write conflict metrics (tikv#1551) close tikv#1550 Signed-off-by: sujuntao <juntao.su@foxmail.com> Co-authored-by: sujuntao <juntao.su@foxmail.com> * apicodec: fix a typo when encoding request for CmdMvccGetByKey (tikv#1638) Signed-off-by: tiancaiamao <tiancaiamao@gmail.com> Co-authored-by: cfzjywxk <lsswxrxr@163.com> * *: update kvproto version (tikv#1636) ref tikv#1631 Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * txn: provide more information in commit RPC / log mvcc debug info when commit failed for `TxnLockNotFound` (tikv#1640) ref tikv#1631 Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * txn: handle undetermined error in client go (tikv#1642) close tikv#1641 Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * txn: fix the implemention of undetermined error (tikv#1644) close tikv#1641 Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> * locate: implement SendReqAsync for RegionRequestSender (tikv#1618) ref tikv#1586 Signed-off-by: zyguan <zhongyangguan@gmail.com> * update pd client for resource group and keyspace (tikv#1645) Signed-off-by: lhy1024 <admin@liudos.us> * tests: bump tidb to fix integration tests (tikv#1650) Signed-off-by: zyguan <zhongyangguan@gmail.com> * Fix some metrics that miss const labels (tikv#1652) Signed-off-by: yibin87 <huyibin@pingcap.com> * Replace etcd safe point with txn safe point for read safety check (tikv#1634) Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> * Fix stale read metrics (tikv#1649) close tikv#1648 Signed-off-by: you06 <you1474600@gmail.com> * *: support async batch get (tikv#1646) ref tikv#1586 Signed-off-by: zyguan <zhongyangguan@gmail.com> * Update kvproto dependancy and set keyspace name for rpc context (tikv#1667) close tikv#1668 Signed-off-by: yibin87 <huyibin@pingcap.com> * ci: add next-gen integration tests (tikv#1661) Signed-off-by: ekexium <eke@fastmail.com> * snapshot: set `ReplicaRead` to false when `ReplicaReadType` fallbacks to `ReplicaReadLeader` (tikv#1663) ref pingcap/tidb#61745 Signed-off-by: you06 <you1474600@gmail.com> * resource_control: support collecting cross AZ traffic in ru consumption (tikv#1669) Signed-off-by: glorv <glorvs@163.com> * txnkv: prevent some actions from being interrupted by kill (tikv#1665) fix pingcap/tidb#61454 Signed-off-by: zyguan <zhongyangguan@gmail.com> * region_cache: add ForceRefreshAllStores function (tikv#1686) Signed-off-by: guo-shaoge <shaoge1994@163.com> * upgrade gRPC to allow consumption by peerdb * Bump gprc version to 1.73.0 Remove references to `grpc.NewSharedBufferPool()`, it was removed from the grpc package and causes build to fail - it's always on. (https://pkg.go.dev/google.golang.org/grpc/experimental#WithBufferPool) Signed-off-by: Tiago Scolari <git@tscolari.me> * go mod tidy --------- Signed-off-by: Ryan Leung <rleungx@gmail.com> Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Signed-off-by: Vlad Dmitriev <vldmit@gmail.com> Signed-off-by: you06 <you1474600@gmail.com> Signed-off-by: ekexium <eke@fastmail.com> Signed-off-by: zyguan <zhongyangguan@gmail.com> Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com> Signed-off-by: okJiang <819421878@qq.com> Signed-off-by: Neil Shen <overvenus@gmail.com> Signed-off-by: disksing <i@disksing.com> Signed-off-by: XuHuaiyu <391585975@qq.com> Signed-off-by: yibin87 <huyibin@pingcap.com> Signed-off-by: Lynn <zimu_xia@126.com> Signed-off-by: tangenta <tangenta@126.com> Signed-off-by: lhy1024 <admin@liudos.us> Signed-off-by: Chao Wang <cclcwangchao@hotmail.com> Signed-off-by: glorv <glorvs@163.com> Signed-off-by: guo-shaoge <shaoge1994@163.com> Signed-off-by: Tiago Scolari <git@tscolari.me> Co-authored-by: Ryan Leung <rleungx@gmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Co-authored-by: Vlad Dmitriev <vldmit@gmail.com> Co-authored-by: you06 <you1474600@gmail.com> Co-authored-by: ekexium <eke@fastmail.com> Co-authored-by: zyguan <zhongyangguan@gmail.com> Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com> Co-authored-by: okJiang <819421878@qq.com> Co-authored-by: Neil Shen <overvenus@gmail.com> Co-authored-by: disksing <i@disksing.com> Co-authored-by: HuaiyuXu <xuhuaiyu@pingcap.com> Co-authored-by: Bisheng Huang <hbisheng@gmail.com> Co-authored-by: yibin <huyibin@pingcap.com> Co-authored-by: Lynn <zimu_xia@126.com> Co-authored-by: tangenta <tangenta@126.com> Co-authored-by: lhy1024 <lhylhy1024@gmail.com> Co-authored-by: JT <43174723+ImSjt@users.noreply.github.com> Co-authored-by: sujuntao <juntao.su@foxmail.com> Co-authored-by: tiancaiamao <tiancaiamao@gmail.com> Co-authored-by: cfzjywxk <lsswxrxr@163.com> Co-authored-by: 王超 <cclcwangchao@hotmail.com> Co-authored-by: glorv <glorvs@163.com> Co-authored-by: guo-shaoge <shaoge1994@163.com> Co-authored-by: Kevin Biju <kevin@peerdb.io> Co-authored-by: Tiago Scolari <tiago@diagrid.io>

ti-chi-bot bot added dco-signoff: yes Indicates the PR's author has signed the dco. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 12, 2025

ekexium force-pushed the next-gen-ci branch 7 times, most recently from f45e927 to ecc2d74 Compare June 13, 2025 07:56

ci: add next-gen integration tests

14ddb6c

Signed-off-by: ekexium <eke@fastmail.com>

ekexium force-pushed the next-gen-ci branch from ecc2d74 to 14ddb6c Compare June 15, 2025 13:28

ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 15, 2025

ekexium added 4 commits June 16, 2025 15:10

fix snapshot_test.go

afcdf23

Signed-off-by: ekexium <eke@fastmail.com>

fix snapshot_fail_test.go

6144bf9

Signed-off-by: ekexium <eke@fastmail.com>

fix lock_test.go

fe4f632

Signed-off-by: ekexium <eke@fastmail.com>

fix scan_test.go

bb36f69

Signed-off-by: ekexium <eke@fastmail.com>

ti-chi-bot bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jun 16, 2025

test: remove online cfg change of in-memory as it won't take effect

61c8012

Signed-off-by: ekexium <eke@fastmail.com>

ekexium force-pushed the next-gen-ci branch from 963c451 to 61c8012 Compare June 16, 2025 09:10

ci: use gotestsum for concise output

8bf1954

Signed-off-by: ekexium <eke@fastmail.com>

ekexium force-pushed the next-gen-ci branch from 4e8b15f to 8bf1954 Compare June 16, 2025 09:28

Merge branch 'master' into next-gen-ci

fb85845

ekexium requested review from cfzjywxk and zyguan June 16, 2025 09:37

zyguan approved these changes Jun 17, 2025

View reviewed changes

ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Jun 17, 2025

ti-chi-bot bot added the approved label Jun 17, 2025

cfzjywxk approved these changes Jun 17, 2025

View reviewed changes

ti-chi-bot bot added lgtm and removed needs-1-more-lgtm Indicates a PR needs 1 more LGTM. labels Jun 17, 2025

ti-chi-bot bot merged commit 806e189 into tikv:master Jun 17, 2025
11 of 12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: add next-gen integration tests #1661

ci: add next-gen integration tests #1661

Uh oh!

ekexium commented Jun 12, 2025 •

edited

Loading

Uh oh!

cfzjywxk commented Jun 16, 2025

Uh oh!

ekexium commented Jun 16, 2025

Uh oh!

ti-chi-bot bot commented Jun 17, 2025

Uh oh!

ti-chi-bot bot commented Jun 17, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ci: add next-gen integration tests #1661

ci: add next-gen integration tests #1661

Uh oh!

Conversation

ekexium commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

cfzjywxk commented Jun 16, 2025

Uh oh!

ekexium commented Jun 16, 2025

Uh oh!

ti-chi-bot bot commented Jun 17, 2025

Uh oh!

ti-chi-bot bot commented Jun 17, 2025

[LGTM Timeline notifier]

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ekexium commented Jun 12, 2025 •

edited

Loading