Author: dgreen

The BigArray Update was completed in about one quarter of the time scheduled for its development, with the first beta release within about four weeks after starting on it.
TerreSculptor’s code base for heightmaps now supports resolutions up to 2 Billion x 2 Billion which is 18 Exabytes.
The build release for Version 3.0 is using a maximum resolution value of 262,144 x 262,144 which is a 256GB heightmap.

The next major update under development is the Multi-Thread Update.
This will update as many modifiers as possible to support multiple processor threads for faster execution times.
A four-core processor will typically get a boost of around 350% faster code execution.

Multi-threading as many of the modifiers as possible will help with faster execution times for larger heightmaps.
This will also be noticeable on the Terrain Stack where multiple modifiers can be executed sequentially.

Some of the modifiers use algorithms that cannot be easily multi-threaded, and those will remain as single-threaded methods.

The Hydraulic Erosion modifier should be getting a performance boost, as should the Rain Erosion modifier.

In addition to the Multi-Thread Update, the next few months will also include the following major Updates:
– File Format Update 1: TSmap file format support for BigArray sizes including GZip – completed as of 332 Beta.
– OpenGL Update 1: Changes to some of the OpenGL functions for the viewport world – completed as of 332 Beta.
– Filter Update 1: Add Filter menu and move filter type modifiers to menu – completed as of 332 Beta.

Once the final build is released for the completed Multi-Thread Update, then the following months into Q1 and Q2 of 2023 will be used for the following major updates:
– File Format Update 2 which includes support for LibTif.net including better TIF image and GeoTIFF support.
– Filter Update 2 with new Filters.
– Modifier Update 1 with multiple new Modifiers.
– OpenGL Update 2 with larger Viewport World setup.
– System Update 1 with additional system information for the motherboard.
– and hopefully Material Update 1 with the new Auto-Material.

My big list of future features is slowly getting completed, but still has more than four dozen major updates and thousands of minor updates.

-eof-

As of next week I will be starting on the BigArray Update for TerreSculptor 3.0.
The BigArray Update is expected to take at least two to four months to complete, so no updates will be released during this time as this is a change that requires a complete code update.

The BigArray Update changes how the memory array system in TerreSculptor manages heightmap files.
The current array system is limited by the C# API to an 8GiB Floating-Point array.
This means that the current maximum heightmap resolution is 46329 × 46329 using this array type (46329 is the the square root of 8GiB).
For the BigArray Update I am changing the entire array system so that it allows for a maximum resolution of 2 Billion × 2 Billion values.
Initially the software will use constants and tests for limiting the maximum heightmap size to 1 Million by 1 Million pixels.
A 1 Million by 1 Million pixel heightmap would require a computer with more than 4TiB of memory.

The BigArray system will be completely memory based, so the maximum heightmap resolution will be limited to the installed computer system memory.
I am staying with memory based arrays rather than file based mainly for performance reasons.
This is because the current performance of DDR4-2133 is 40 times faster than an SSD at SATA 500MB/s speed.
File based arrays would mean that something that requires only 1 minute to execute in memory could require as much as 40 minutes or more to execute disk based with page swapping on a fast SSD.
I am also not simply changing to a tiled array system using the limited C# arrays, like most other heightmap software that is currently out there (you know who you are), because that results in edge issues between tiles for many of the modifiers and filters.
So in order to work on larger and larger heightmaps, more computer system memory will be required.
Note that as you go larger in heightmap size, the time to operate on that array data increases, so faster computers with more cores will also be necessary.

Many of the modifiers in TerreSculptor require multiple temporary arrays to operate on the heightmap.
I will be updating more dialogs to show how many arrays they require for operation, so that better estimating of total memory footprint can be accomplished.
There will also be updates to the supported file formats to allow supporting this change to larger heightmaps.

I am also working on multi-threading as many of the modifiers and functions as possible, so that larger heightmap sizes operate at the highest performance possible.

The maximum heightmap resolution for common memory setups is as follows, note that these numbers do not include the operating system footprint, so the actual available maximum size will be a bit less.

16GiB = 17179869184 bytes
= 4294967296 floats
= 65536 x 65536 maximum resolution

32GiB = 34359738368 bytes
= 8589934592 floats
= 92681 x 92681 maximum resolution

64GiB = 68719476736 bytes
= 17179869184 floats
= 131072 x 131072 maximum resolution

128GiB = 137438953472 bytes
= 34359738368 floats
= 185363 x 185363 maximum resolution

256GiB = 274877906944 bytes
= 68719476736 floats
= 262144 x 262144 maximum resolution

512GiB = 549755813888 bytes
= 137438953472 floats
= 370727 x 370727 maximum resolution

1024GiB = 1099511627776 bytes
= 274877906944 floats
= 524288 x 524288 maximum resolution

-eof-

TerreSculptor 3.0 2022.233 for August 21st is now available.

The main features of this release are:
– The PNG Image importer supports UE5 Landscape Tile Export.
– The Image and ImagePlus folders are persisted between sessions.
– The Normalmap Creator supports saving to TGA file format.

For the future build releases over the next year, the main features under development will be adding multi-threading to the single-threaded Modifiers, and adding multi-lingual support to the dialogs.
Both of these features are estimated to take up to 2000 hours of work.
The multi-lingual support will be added to the dialogs for French, German, and Spanish, which might require some dialog control resizing.  This language support will be slowly added to each dialog, so for the next year or so the software will be in flux for which dialogs have been converted and which are still pending, so the software will be a mix of the current culture and the default English.

TGA file format is also being added to the Bitplane Creator and Splatmap Creator file save functions.

-eof-

Over the next six to twelve months, which are Q3/Q4 2022 and Q1/Q2 2023, the following items are planned for development in TerreSculptor 3.0.
These are all Build Updates which will be completely free for all licensed users of the 3.0 software.
To see the list of build updates for each current build release, see the Release Notes text file, available on the software’s Help menu.

1. The Multi-Thread Update.

Somewhere around 20% of the modifiers are currently multi-threaded. That leaves 80% as currently single-threaded.

The threading system in TerreSculptor supports virtually unlimited core-count and thread-count, although it is currently limited to 1024 maximum threads in easily-extendable code.
It will be a while before we start to see 512-Core HyperThread mainstream consumer processors and computers.

Many of the modifiers execute in less than one second even on large 16k+ heightmaps.
However, I would still prefer that every modifier possible be threaded, simply because now with the Terrain Stack, modifier performance can start to impact Stack build times.

With currently 98 Devices, and about 80% of them requiring multi-threading, even if it only required 10 hours per Device to update the code, that is still about 800 hours of development time.
So the multi-thread update will consume a large portion of the next few quarters.

Some of the Filter Functions will be difficult to multi-thread, so expect probably 5% of the Devices to not have multi-threading.
I have some tricks up my sleeve for working on threading such difficult Devices as the Rain Erosion, I still have to test my code hypothesis to see if it is functional.

2. Auto-Material.

I have been lightly researching the addition of an Auto-Material to the current Terrain Material set.

This Auto-Material would support two elevation values and two slope values.
This would allow for beach/snow for elevation, and shallow/steep slope for grass and rock variance.
The terrain coloring would of course be something such as sand for beach, grass for shallow-slope, rock for steep-slope, and snow.

The Auto-Material would provide similar terrain coloring to what the Splatmap system typically supports, but with an easier to use interface and no splatmap bitplane extraction required.

3. Biome Extraction.

One of the features that I have been working on for some time is an automated Biome Extraction and Splatmap Creation system.

Additional work will be performed on this feature over the next few quarters.

4. Digital Elevation Model File Header Parsing.

One of the features that I would like to see in TerreSculptor, is better handling of the Header Information for Digital Elevation Model files.
With the ability to automatically scale the terrain mesh to the file information.

I have still been working out exactly how I am going to manage this, without being obtrusive, and performing the sizing logically.
TerreSculptor is not GIS software, and is not in that market segment at all, but I would like to see more DEM support features over the next few quarters.

5. Pathspec Persistence.

You may have noticed (or not) that many of the drive and folder paths are persisted between sessions in TerreSculptor.
This is a feature that I have had since the early builds.
Nothing is more frustrating than having to constantly re-navigate back through drives and folders to re-load some image file.

A few additional Pathspec Persistence settings are required, such as for the Alpha Brush path for the Brush Modifier.
It gets irritating to have to constantly re-navigate back to that folder if it changes with another software operation that shares that current persisted path.

So the new Brush Pathspec Persistence code plus a few others will be implemented over the coming weeks.

6. Odds and Ends.

My documentation file that contains the list of future features contains well over 1200 entries of major features plus a few minor features and updates.
This is a compact list of a short description of each future feature that I want in the software.

At the top of this list is always the highest priority or most recent minor items that require the quickest attention for code development and updates.
This section of the list is usually at least 30 or 40 items that should be completed within the next quarter or two.

So these “odds and ends” will also be showing up in the next few builds over the next few quarters.

-eof-

TerreSculptor has been increased to Version 3.0 with this latest build release.
This is due to the large number of changes in the software:

– It is now retail, which was a tough decision.
– The new Terrain Stack, which is a major milestone feature.
– The new Brush Modifier, which is a powerful feature.

The Terrain Stack was in the original design documentation from the beginning, to provide an Autodesk 3DS Max or Blender 3D style of Modifier Stack, with more features such as Device Item Masks.  The Stack has now been completed to its first release iteration, with additional features planned for the future.

The Brush Modifier is a new concept and feature, employing Unreal Engine 4/5 style Landscape Alpha Brushes in a new Modifier.  This feature works well up to terrain sizes of 8192 and possibly even 16384, although the preview starts to get a little small by that resolution.  Future design considerations may include 3D Widgets for Location, Scale, and Rotation in the main viewport when a Brush is selected in the Terrain Stack — we’ll see how that goes.

I decided to take this new version to retail.  This choice was made because Patreon Support didn’t come anywhere near covering development expenses, and I gave it a few years to see if people would step up.
With more than 100,000+ downloads, there are only about 30 Patreon subscribers, with most paying $1 per month ($12 per year).  That is only 0.03% of the people who downloaded the software decided to help support it.  Or about 1 person out of every 3400.  And since going with Patreon I have brought in just under $3000 in total support income.  That doesn’t even cover one of my software development yearly subscriptions for the years that I was on Patreon.
Over the past few years I have gone into debt by almost $40,000 for development tools and development software, to provide the TerreSculptor software for free to the terrain design community.  I decided that I couldn’t continue on that trajectory, since I am a single individual making only average income in northern Canada.
So the choice was made to make TerreSculptor 3.0 now retail software.  However, a low price was chosen so that the software is still obtainable by people in countries with low income.  The price of $99 CAD or about $77 US was chosen, which puts TerreSculptor significantly less expensive than any of the competing terrain software, and even less than their Indie versions which have significant limitations.  You get the full TerreSculptor for the low price.

There are many new and powerful features still under development for the software.  The plan is to release new versions every 12 to 18 months for resale that will include major feature updates, such as the File Format Update and the BigArray Update.  And build updates every month or two that are free updates for those who have purchased a copy of the software.  The free updates will include such features as bug fixes, performance enhancements, multi-threading of devices, and other minor features and improvements.

For anyone who downloads the version 3.0 builds and tries to run them without a registration key, the software will run in Demo Mode for about three minutes.  TerreSculptor 2.0 is still freely available to anyone who wants to use it.

-eof-

Alpha Brushes are a feature that Unreal Engine 4/5 uses with the Landscape actor to interactively create terrain layouts from terrain shape brushes, such as various mountain shapes that can be stamped onto the terrain.
These Alpha Brushes can be positioned, scaled, and rotated, anywhere on the Landscape actor.
Alpha Brushes are often real-world terrain details.

https://docs.unrealengine.com/4.27/en-US/BuildingWorlds/Landscape/Editing/Brushes/

TerreSculptor now includes support for these same Alpha Brushes with the Brush Modifier.  The Brush Modifier allows for interactively stamping alpha brushes onto the current heightmap.  The Brush Modifier supports the blending of one alpha brush file, and can be used iteratively on the menus
When used in conjunction with the now-functional Terrain Stack, multiple Brush Modifiers can be placed onto the stack, iterated and edited, and a final complex brush layout can be achieved, such as complete mountain ranges, areas filled with various size craters, and special terrain features such as volcanoes, all with non-destructive editing.
The TerreSculptor Brush Modifier supports multiple Blend Modes and a fully interactive user interface.

The Brush Modifier should ship in the same build as the Terrain Stack later this year.

TerreSculptor Terrain Stack Update
This major Update should be publicly available in Q3/Q4 2022.

Q. What is the Terrain Stack in TerreSculptor?
A. The Terrain Stack is a highly interactive device operation layer system that allows for non-destructive workflow of terrain creation.

Each of the items on the Generate, Noisemap, Weightmap, Adjust, Modify, Transform, and Erosion menus are available as ‘Devices’ that can be added to the Terrain Stack, so that they can be edited and executed in any logical order, and re-edited and iterated.
This allows for a flexible terrain design system that can be edited over and over again, until the desired final terrain look is achieved.
An example Stack may include the following Items in descending order: Gradient Noisemap, Altitude, Altitude Center, Displace, Rain Erosion, Hydraulic Erosion, FloodLevel, Normalize, Datamap Save.
This Stack of Items can then be built and the device properties iterated over and over until the final desired terrain design is realized.
The Terrain Stack can be saved to a file, so that a library of common terrain design systems can be created.
The Terrain Stack can be thought of as similar to the Layer system in Photoshop, but more accurately to the Modifier Stack in Autodesk Max or Blender 3D.

Q. When is it available?
A. The initial Terrain Stack is already in beta testing with the Patreon subscribers.

There is still a lot of work to be completed on the Terrain Stack before it is ready for public access.
There will also be Feature Updates to the Terrain Stack over the next few years.
So while the Terrain Stack is currently functional, it won’t be released in a public build for a few months still.

Q. What is remaining to be completed?
A. Some of the current features that are pending on the Terrain Stack include: Variable Size Stack, Modifier Stack Pointers, and Unique Item GUIDs.

The Variable Size Stack allows for such Modifiers as Resample and Rotate Custom, where the dimensions of the Stack can be changed on a per-Device level.
The Modifier Stack Pointers will allow specific Modifiers such as Blend, Combine, and Shaper, to reference a Stack Item as a secondary Heightmap or Mask.  For example, the Blend Modifier blends two heightmaps together, and the Modifier Stack Pointers will allow it to use a Stack Item reference instead of having to reference an external file on disk.
The Unique Item GUIDs will allow the Modifier Stack Pointers to track changes to the Stack Item order and “fix up” pointer references automatically.

Q. What advanced features will it include?
A. The Terrain Stack will eventually include a per-Item Mask, and it will eventually be used for the 2D Terrain Paint and Brush system, and the future 3D Spline Road River and FaultLine systems.

The Item Masks will allow for such things as complex biomes, by allowing for masking out regions that the Modifiers operate on.  For example, Terraces can be applied to one region of the terrain, while FloodLevel is applied to another region.
The future Paint and Brush systems will allow for using paint tools to modify the terrain elevations, or terrain brushes to ‘stamp’ a terrain shape such as a mountain range onto the heightmap.
The future 3D spline modifiers will include a spline in the main viewport that can be manipulated to modify the heightmap.

Q. What else is coming soon?
A. Coding will also begin in Q3/Q4 on the ‘CLI’ Command Line Interface Update, the File Format Update, and the Project Update.

The CLI Update will allow for automation of TerreSculptor functions through the command prompt and through batch files.  Initial CLI functionality will be such things as heightmap file format conversion.
The File Format Update will be bringing EXR image file format and FBX mesh file format to TerreSculptor, in addition to supporting libpng and libtif to improve the total supported PNG and TIF file formats.
The Project Update will be adding the newer Modifiers and the Terrain Stack to the TerreSculptor Project file format, so that these items can be saved and loaded with a Project file.

I don’t get much time to make blog posts, most of my time is spent developing the TerreSculptor software and creating tutorial videos for the YouTube channel.  I work a full-time job in the tech sector for 30 to 40 hours each week, and then spend every waking hour outside of that writing code or working on tutorial videos.  I typically get in another 40+ hours of programming each week, and each video usually requires about 1 hour per finished-minute of work (typically 12 hours work to create a 10 minute video, not including the research time).

Here is what is upcoming for TerreSculptor in the near future:
– During Q3-2022 additional code changes are being made to accommodate the upcoming BigArray Update.  BigArray is a near-future update that will see the current 46k x 46k maximum heightmap limitation removed, and the new maximum dimensions will be 2 Billion x 2 Billion.
– Q3/Q4-2022 will see the initial Terrain Stack Update release.  The Terrain Stack allows for non-destructive editing of heightmaps using a layered stack system, where modifiers can be added and adjusted over and over.  The Stack will eventually also support a per-item Mask, so that Modifiers can be applied to a masked region of the terrain, allowing for multi-biome work.
– Q1/Q2/Q3-2023 will see the Multi-Thread Update where most devices in the software will be threaded for greater performance, which is a requirement once the BigArray update is released.  This will also benefit build times of the Terrain Stack.
– Q3/Q4-2023 will see the BigArray Update release.
– Somewhere in between all of this will be one of the File Format Updates, which will see the initial EXR image file format added, which is required for large heightmaps, since the Windows API for BMP,GIF,JPG,PNG,TIF is really limited.  Plus also some initial FBX file format code added.
– The future beyond that still has more than 1000 major features to be completed in the software, including multiple 3D Engine Updates that will add new features to the main editor viewport and other render functions.  These include terrain sculpting, terrain brushes, 3D splines, and 3D mesh scene objects.

To always get the latest information on what is happening with the TerreSculptor software, be sure to follow me on Twitter, as I usually make daily tweets, or check out the Facebook Group (link is on the software’s Help menu or the web site’s Media page).  And always consider becoming a Patreon subscriber.  For as little as $1 a month you can help make this software better.  I have continual subscriptions for the web site and development tools that require payment, so I appreciate any and all financial support.