BigArray Update 2022-09-16

As of next week I will be starting on the BigArray Update for TerreSculptor 3.0.
The BigArray Update is expected to take at least two to four months to complete, so no updates will be released during this time as this is a change that requires a complete code update.

The BigArray Update changes how the memory array system in TerreSculptor manages heightmap files.
The current array system is limited by the C# API to an 8GiB Floating-Point array.
This means that the current maximum heightmap resolution is 46329 × 46329 using this array type (46329 is the the square root of 8GiB).
For the BigArray Update I am changing the entire array system so that it allows for a maximum resolution of 2 Billion × 2 Billion values.
Initially the software will use constants and tests for limiting the maximum heightmap size to 1 Million by 1 Million pixels.
A 1 Million by 1 Million pixel heightmap would require a computer with more than 4TiB of memory.

The BigArray system will be completely memory based, so the maximum heightmap resolution will be limited to the installed computer system memory.
I am staying with memory based arrays rather than file based mainly for performance reasons.
This is because the current performance of DDR4-2133 is 40 times faster than an SSD at SATA 500MB/s speed.
File based arrays would mean that something that requires only 1 minute to execute in memory could require as much as 40 minutes or more to execute disk based with page swapping on a fast SSD.
I am also not simply changing to a tiled array system using the limited C# arrays, like most other heightmap software that is currently out there (you know who you are), because that results in edge issues between tiles for many of the modifiers and filters.
So in order to work on larger and larger heightmaps, more computer system memory will be required.
Note that as you go larger in heightmap size, the time to operate on that array data increases, so faster computers with more cores will also be necessary.

Many of the modifiers in TerreSculptor require multiple temporary arrays to operate on the heightmap.
I will be updating more dialogs to show how many arrays they require for operation, so that better estimating of total memory footprint can be accomplished.
There will also be updates to the supported file formats to allow supporting this change to larger heightmaps.

I am also working on multi-threading as many of the modifiers and functions as possible, so that larger heightmap sizes operate at the highest performance possible.

The maximum heightmap resolution for common memory setups is as follows, note that these numbers do not include the operating system footprint, so the actual available maximum size will be a bit less.

16GiB = 17179869184 bytes
= 4294967296 floats
= 65536 x 65536 maximum resolution

32GiB = 34359738368 bytes
= 8589934592 floats
= 92681 x 92681 maximum resolution

64GiB = 68719476736 bytes
= 17179869184 floats
= 131072 x 131072 maximum resolution

128GiB = 137438953472 bytes
= 34359738368 floats
= 185363 x 185363 maximum resolution

256GiB = 274877906944 bytes
= 68719476736 floats
= 262144 x 262144 maximum resolution

512GiB = 549755813888 bytes
= 137438953472 floats
= 370727 x 370727 maximum resolution

1024GiB = 1099511627776 bytes
= 274877906944 floats
= 524288 x 524288 maximum resolution