Currently caliper will aggressively check for non deterministic allocation 
behavior and fail the measurement when non determinism is detected.

This can be partially worked around by setting 
instrument.allocation.options.trackAllocations=false
which simply reduces the amount of data that caliper tracks and thus limits the 
kinds of non determinism that can be detected.

On the one hand a non deterministic benchmark is a bad thing since it makes 
your measurements suspect so users should try to eliminate it (i.e. make the 
benchmark perform the exact same amount of work on each rep).  However, not all 
non-determinism (of allocation behavior) can be eliminated.  For example:

In multithreaded benchmarks, lock contention will cause allocations and is 
inherently non deterministic.
Lazy initialization of data is a common pattern and its possible that not all 
such initialization can be triggered in warmup.
and others

Caliper should try to support such use cases.

One proposal would be for the AllocationWorker to be configured to work like 
the RuntimeWorker and instead of subtracting a baseline measurement to remove 
overheads just run a large number of reps to minimize them.  This could be 
configurable via something like
instrument.allocation.options.strategy