Data inconsistency: @unique constraints are bypassed when mutations are sent to non-owning groups

[eileenaaa]

Hi Dgraph team,

We have identified a significant data integrity issue in Dgraph multi-group environments. The @unique constraint is only enforced if a mutation is sent to an Alpha node within the group that owns the predicate. If the mutation is routed to an Alpha node in a different group, the unique check is bypassed entirely, leading to duplicate data.

Scenario to Reproduce

1. Setup: A Dgraph cluster with multiple groups (e.g., Group 1, 2, and 3).
2. Schema: Define a predicate Xid with a unique constraint:

curl -X POST 'http://localhost:8080/alter' -d '{"schema": "Xid: string @index(hash) @unique @upsert ."}'

Assume Xid is assigned to Group 1.
3. Inconsistent Behavior:

• Group 1 Alpha: Sending a mutation with duplicate Xid values is rejected (Correct).
• Group 2/3 Alpha: Sending the same mutation succeeds, allowing duplicates (Bug).

curl --location 'http://localhost:8080/mutate?commitNow=true' \
--header 'Content-Type: application/rdf' \
--data '
{
  set {
    _:node1 <Xid> "shared_xid_001" .
    _:node2 <Xid> "shared_xid_001" .
 
  }
}'

Root Cause Analysis

The issue stems from the addQueryIfUnique function in dgraph/edgraph/server.go. It relies on schema.State().Get() to verify if a predicate has the @unique directive.

In a sharded multi-group architecture, an Alpha node's local schema.State only contains detailed metadata for predicates managed by its own group. For "remote" predicates, Get() returns ok == false, causing the logic to skip the unique check generation:

// Current logic in dgraph/edgraph/server.go:
predSchema, ok := schema.State().Get(ctx, x.NamespaceAttr(namespace, pred.Predicate))
if !ok || !predSchema.Unique {
    continue // Bypass happens here for non-local predicates
}

Proposed Solution: On-Demand Cross-Group Schema Verification

To ensure consistency without sacrificing performance, I propose an on-demand fetching mechanism combined with a per-request local cache:

1. Cross-Group Fetch: If a predicate is not found in the local schema.State, the node should call worker.GetSchemaOverNetwork to fetch the schema from the group that owns the predicate.
2. Per-Request Cache: To prevent "RPC amplification", I introduced a repaired map[string]bool within the scope of addQueryIfUnique. This ensures only one network call per unique predicate per mutation request.
3. State Isolation: The fetched schema is used only for the current mutation logic and is not persisted to the local global schema.State. This ensures that if the schema is altered again, nodes do not hold onto stale cached data.

Draft Implementation Logic

// Inside addQueryIfUnique:
repaired := make(map[string]bool)

// ... inside the loop ...
if !ok {
	unique, predChecked := repaired[fullPred]
	if !predChecked {
		schReq := &pb.SchemaRequest{Predicates: []string{fullPred}}
		remoteNodes, err := worker.GetSchemaOverNetwork(ctx, schReq)
		if err == nil && len(remoteNodes) > 0 {
			predSchema.Unique = remoteNodes[0].Unique
			ok = true
		}
		repaired[fullPred] = predSchema.Unique
	} else {
		predSchema.Unique = unique
		ok = true
	}
}

if !ok || !predSchema.Unique {
    continue
}

Detailed changes can be found here:
eileenaaa@c66c398

[matthewmcneely]

@eileenaaa I think this should be created as an Issue. Can you do that so that it can be properly assigned? Thanks.

[eileenaaa]

Thanks for the suggestion. I've just created the issue: #9565.
I've included the detailed root cause analysis and a proposed fix there. Please let me know if you need anything else.