Historically the step was always 3.
I found, over time, that increasing the step greatly to something like 128 could greatly reduce memory consumption and performance in many cases.
In the situation where a large number of small objects are allocated then this number like 128 becomes highly abusive.
The simple low allocation step will only allocate a single unit on the very first allocation.
If the next allocation is on an array that has a size greater than one and less than four (via the tiny define), then the step size is set to four during allocation.
If the next allocation is on an array that has a size greater than four and less than eight (via the small define), then the step size is set to eight during allocation.
If the next allocation is on an array that has a size greater than eight and less than sixty-four (via the large define), then the step size is set to sixty-four during allocation.
In all cases, if the request step is less than the calculated step, then the requested step is used.
For example, if the requested step is twelve, then after eight is allocation, then the next generated step size is twelve rather than sixty-four.
Using some test files, shows the following reduction:
- Old: ~8GB of RAM -> New: ~200MB of RAM.
- Old: ~500MB of RAM -> New: ~20MB of RAM.
Update the unit tests accordingly and fix any problems exposed.