Debug logs affect query result

[ienkovich]

The problem can be reproduced on the H2O groupby 1e8 dataset. Here is a reproducer:

import pyhdk

pyhdk.initLogger(debug_logs=True)
hdk = pyhdk.init()
ht = hdk.import_csv("data/G1_1e8_1e2_0_0.csv")

tmp = ht.agg([], count_dist="count_dist(id6)").run()
print(tmp)

res = ht.agg(["id6"], v3="count").run()
print(res.shape)

The output is:

Schema:
  count_dist: count_dist(db_id=1677721601, table_id=1, column_id=2000 type=INT64)
Data:
1000000

(981529, 2)

We expect 1000000 rows in the second query and we actually get the correct result if remove the logger initialization line. The wrong result looks like it has data cut off at the tail.

[ienkovich]

Simpler reproducer:

import pyhdk

pyhdk.initLogger(debug_logs=True)
hdk = pyhdk.init()
ht = hdk.import_pydict({"a": [1, 1, 1, 2, 2, 2]}, fragment_size=3)
res = ht.agg(["a"], "count").run()
print(res.shape)
print(res)

Result:

(1, 2)
Schema:
  a: a(db_id=1677721601, table_id=1, column_id=2000 type=INT64)
  count: count(db_id=1677721601, table_id=1, column_id=2001 type=INT32[NN])
Data:
1|3
2|3

As we can see, the result itself is correct but it reports the wrong row count.

It happens because on DEBUG2 logging level we print the row count for each produced result before reduction. It sets the row count cache for each part and it is not invalidated after reduction.