docBatchSize warning is confusing

[rjrudin]

So we can address your issue, please include the following:

Version of MarkLogic Java Client API

5.4.0

Version of MarkLogic Server

N/A

Java version

N/A

OS and version

N/A

Input: Some code to illustrate the problem, preferably in a state that can be independently reproduced on our end

DHF develop branch is using Java Client 5.4.0 now. DHF uses a QueryBatcher to run a flow. It constructs a QueryBatcher based on an Iterator on a Java collection. Every time I run a flow now, I see the following logged at the warning level:

"docBatchSize is beyond maxDocBatchSize"

In my test, the Java collection used to build the iterator has 6 items in it. I get this warning no matter what I set the batch size to - 1, 3, 6, 30, 300.

The main issue is that because this is logged at the warn level, it always shows up when a user uses Gradle to run a flow. The message doesn't have any context, nor does the user know what either of those values are (particularly 'maxDocBatchSize'), so the user doesn't know what to do about it.

What that means is if DHF 5.5 is released and this warning shows up, the DHF team will be getting questions about why this is showing up and what users ought to do about it. And we don't know.

Actual output: What did you observe? What errors did you see? Can you attach the logs? (Java logs, MarkLogic logs)

Expected output: What specifically did you expect to happen?

Given that I don't know what to do with this message, my first though is - don't show it. Or log it at the debug level. But it likely needs to be improved to provide more context, and it also needs to be written in a more user-facing way - e.g. "The batchSize is greater than (some explanation of what maxDocBatchSize refers to); consider (doing something that likely makes more sense than what the user is currently doing)".

Alternatives: What else have you tried, actual/expected?

[rjrudin]

@ehennum @llinggit Could this be addressed in a 5.4.x release? I'm concerned about this going into the DHF 5.5.0 release and the avalanche of questions users will have about "Why am I getting 'docBatchSize is beyond maxDocBatchSize' warnings? What is wrong with DHF 5.5.0??? This wasn't happening before!!"

[llinggit]

Hi @rjrudin, I'll try to hide this error msg to a lower level first. At the same time, could you share with us which server version you are using, please? Thanks.

[rjrudin]

It was ML 10.0-6.x

[llinggit]

Thanks.

[georgeajit]

Can we remove the commented overloaded method - one with throttle factor in QueryBatcher class?

//public QueryBatcher withBatchSize(int docBatchSize, int docToUriBatchRatio, int threadThrottleFactor);

[georgeajit]

Ran a test with ratio parameter included. Did not see any warning messages.

[georgeajit]

Can be shipped once stale code is stripped out.

[llinggit]

There's no server-side change in this issue.

[rjrudin]

This can be closed due to the 5.5.0 release, right?