What happened:
When benchmarking with the random data generator and num_workers > 0, all generated requests contain identical text content. This occurs even when multiple worker processes are used, as each worker generates the same sequence of "random" tokens. In the specific case where all input lengths are the same (e.g., std_dev: 0.0), every single request across all workers becomes a bitwise duplicate of the others.
What you expected to happen:
Each request should contain unique random text content. Using multiple workers should parallelize the generation of different random sequences, rather than duplicating the same sequence in parallel.
How to reproduce it (as minimally and precisely as possible):
Run a benchmark with the following data configuration and num_workers set to a value greater than 1:
data:
type: random
path: null
input_distribution:
min: 8192
max: 8192
mean: 8192.0
std_dev: 0.0
total_count: 601
type: normalObserve the logs or captured prompts to see that every request has the exact same content.
Anything else we need to know?:
The issue stems from how RandomDataGenerator initializes its random number generator (RNG) in inference_perf/datagen/random_datagen.py.
- RNG Initialization: The RNG is initialized in the constructor without a specific seed:
self.rng: np.random.Generator = np.random.default_rng().
- Multiprocessing Copy: When
num_workers > 0, the RandomDataGenerator object is pickled and copied to each worker process.
- Deterministic Generation: Since each worker starts with an identical RNG state snapshot, calls to
_generate_exact_length_text (triggered via load_lazy_data) produce the same token sequences across different processes.
This requires a fix to re-seed the RNG within each worker process after the fork/spawn occurs to ensure divergence.
Environment:
- inference-perf version: [Please provide version]
- config.yml: [Include the relevant snippet provided above]
- others: Multiprocessing enabled with
num_workers > 0.
What happened:
When benchmarking with the
randomdata generator andnum_workers > 0, all generated requests contain identical text content. This occurs even when multiple worker processes are used, as each worker generates the same sequence of "random" tokens. In the specific case where all input lengths are the same (e.g.,std_dev: 0.0), every single request across all workers becomes a bitwise duplicate of the others.What you expected to happen:
Each request should contain unique random text content. Using multiple workers should parallelize the generation of different random sequences, rather than duplicating the same sequence in parallel.
How to reproduce it (as minimally and precisely as possible):
Run a benchmark with the following data configuration and
num_workersset to a value greater than 1:Observe the logs or captured prompts to see that every request has the exact same content.
Anything else we need to know?:
The issue stems from how
RandomDataGeneratorinitializes its random number generator (RNG) ininference_perf/datagen/random_datagen.py.self.rng: np.random.Generator = np.random.default_rng().num_workers > 0, theRandomDataGeneratorobject is pickled and copied to each worker process._generate_exact_length_text(triggered viaload_lazy_data) produce the same token sequences across different processes.This requires a fix to re-seed the RNG within each worker process after the fork/spawn occurs to ensure divergence.
Environment:
num_workers > 0.