CXXCBC-345: Range scan improvements & resolve concurrency issues

[DemetrisChr]

Motivation

A number of issues were observed with the existing range scan implementation, such as items missing & the scan getting stuck indefinitely when more than one IO threads are used in the SDK. This change simplifies the implementation of range scan and resolves issues with concurrency and IO threads.

Changes

• Create a range_scan_load_balancer class to contain all of the logic around selecting which vbucket to scan.
• Instead of using one concurrent channel for each vbucket scan when sending items to the caller (i.e. having 1024 channels), only use a single channel shared by all vbucket scans. This significantly simplifies the logic of next_item(). Items are sent through this channel, as well as signals that a vbucket scan has been completed (either successfully or with a fatal error), which allows the orchestrator to keep track of whether there are any outstanding vbucket scans.
• Use asio::post when triggering KV operations within a KV callback (e.g. when doing a range_scan_continue from the callback of range_scan_create)
• Add unit tests for the load balancing logic
• Fix an issue with the "orchestrator prefix scan, get 10 items and cancel" test where it was fetching documents created in a different test.
• For sampling scans, if a seed is provided it is now used for shuffling the vbuckets in the load balancer to ensure that a given seed always yields the same sample when concurrency 1 is used. We cannot have the same guarantee for concurrency > 1, as the order the vbuckets are scanned depends on factors such as how long each scan takes and how many scans are taking place on each node at any given time.

Results

All tests pass, including FIT

[Matt-Woz]

LGTM but maybe want another pair of eyes to confirm - also verified with PHP and it seems to have solved the previous issues w/concurrency

[thejcfactor]

Apart from my 1 comment, FIT looks good w/ the Python and Node.js performers. I can consistently get all tests to pass (wasn't the case previously). I will look more closely at the changes tomorrow if @avsej cannot get to them first. 👍

[avsej]

Looks good in general, couple of style notes