feat: add query rewriting for IN clauses and bulk INSERT operations

[Copilot]

Standard SQL patterns with multiple parameters (IN clauses, bulk INSERT VALUES) are inefficient in YDB. Each parameter adds overhead, and the query optimizer can't leverage list-based operations.

Implementation

• New bind layer: RewriteQueryArgs transforms queries at blockYQL stage

  • IN clauses: IN ($p1, $p2, $p3) → IN $argsList0 with single list parameter
  • Bulk inserts: VALUES ($p1, $p2), ($p3, $p4) → SELECT FROM AS_TABLE($valuesList0) with struct list parameter
  • Also handles UPSERT and REPLACE operations

• Public API: WithRewriteQueryArgs() connector option enables transformations

• Regex-based parser: Simple pattern matching for common cases. Known limitations with nested parentheses and qualified table names documented in code.

Usage

connector, err := ydb.Connector(nativeDriver,
    ydb.WithRewriteQueryArgs(),
    ydb.WithNumericArgs(),  // composes with other bindings
)

// IN clause - 4 parameters become 1 list parameter
rows, _ := db.QueryContext(ctx, 
    "SELECT * FROM t WHERE id IN ($1, $2, $3, $4)",
    1, 2, 3, 4,
)

// Bulk insert - 6 parameters become 1 struct list parameter  
db.ExecContext(ctx,
    "INSERT INTO t (id, val) VALUES ($1, $2), ($3, $4), ($5, $6)",
    1, "a", 2, "b", 3, "c",
)

Notes

• Single-parameter IN clauses and single-row INSERT statements are not transformed
• Multiple transformations in one query each get separate list parameters ($argsList0, $argsList1, etc.)
• Limitations with edge cases (nested parens, qualified tables) acceptable for YDB use patterns

Original prompt

---

This section details on the original issue you should resolve

<issue_title>feat: auto-fix query and args with List of values in database/sql driver for YDB</issue_title>
<issue_description>Require behaviour

1. If query contains IN clause

// var db *sql.DB
rows, err := db.QueryContext(ctx, "SELECT * FROM t WHERE id IN ($p1, $p2, $p3, $p4)", 
  sql.Named("p1", 1),
  sql.Named("p2", 2),
  sql.Named("p3", 3),
  sql.Named("p4", 4),
)

then final query which send to YDB must be

// var s query.Session
r, err := s.Query(ctx, "SELECT * FROM t WHERE id IN $argsList", query.WithParameters(
  ydb.ParamsBuilder().Param("$argsList").BeginList().AddItems(
	types.Uint64Value(1),
	types.Uint64Value(2),
	types.Uint64Value(3),
	types.Uint64Value(4),
  ).EndList().Build(),
))

Few IN clauses in source query must be replaced with few argsList params

2. 1. If query contains INSERT INTO VALUES (), (), ()

// var db *sql.DB
rows, err := db.QueryContext(ctx, "INSERT INTO t (id, value) VALUES ($p1, $p2), ($p3, $p4), ($p5, $p6)", 
  sql.Named("p1", 1),
  sql.Named("p2", "1"),
  sql.Named("p3", 2),
  sql.Named("p4", "2"),
  sql.Named("p5", 3),
  sql.Named("p6", "3"),
)

then final query which send to YDB must be

// var s query.Session
r, err := s.Query(ctx, "INSERT INTO t (id, value) SELECT id, value FROM AS_TABLE($valuesList)", query.WithParameters(
	ydb.ParamsBuilder().Param("$valuesList").BeginList().AddItems(
		types.StructValue(
			types.StructFieldValue("id", types.Uint64Value(1)),
			types.StructFieldValue("value", types.TextValue("1")),
		),
		types.StructValue(
			types.StructFieldValue("id", types.Uint64Value(2)),
			types.StructFieldValue("value", types.TextValue("2")),
		),
		types.StructValue(
			types.StructFieldValue("id", types.Uint64Value(3)),
			types.StructFieldValue("value", types.TextValue("3")),
		),
	).EndList().Build(),
))

INSERT, UPSERT, REPLACE must works equal when VALUES rows great then 1

---

Behaviour for replace query and args must enabled only with specific ydb.Connector option for database/sql driver

Potential place for replace queries - internal/xsql (Bind api)

Some ways for implement this feature:

1. regexp
2. using yql-parser from https://github.com/ydb-platform/yql-parsers/tree/main/go, detect IN and other clauses, replace
3. is there something else (need to find out how the feature was implemented with query substitution and arguments in other databases)?

Need to discover all ways before implement - pros and cons, plus and minuses, risks and performance</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes feat: auto-fix query and args with List of values in database/sql driver for YDB #2021

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.