Author: dgreen

A lot of work was completed on TerreSculptor in the past two months.

In addition to all of the work mentioned in the last blog post, the new BigPNG is also now complete.
This allows PNG file formats to support files up to the full 1 million x 1 million datamap size.
The TileEx Creator is also completed, which allows for tiling the full 1 million x 1 million datamap size.
Plus more Device multi-threading and a lot of other smaller updates.

I have been spending a lot of time in the past two weeks updating the original TerreSculptor Reference Manual for Version 3.0 of the software.
This PDF format manual is a 400 page book on everything about TerreSculptor.
This manual will become part of the regular install distribution files.
As new features are added to TerreSculptor, the manual will also be updated.
The latest version of the manual will also always be available on the Google Free Assets Drive (see your software Help menu for the link).

Starting in a few days will be the work on two new major features, the new Biome Menu item for Automated Splatmap creation, and the main viewport Terrain Auto-Material Material.

The new Biome Menu will be used for biome and splatmap extraction features.
I have been thinking about how I will develop the initial Automated Splatmap creator, and it will initially have at least six to eight RGB and RGBA/RGBAK Splatmap auto-creation tools.
The design will be a single dialog with multiple sliders that allow for specifying the settings for most of the more common Splatmap configurations.
Such as Beach Flat/Steep, Grass Flat/Steep, and Mountain Flat/Steep.
And adjusting the sliders will automatically create a full Splatmap based on the settings, that can be saved to disk.
This will automate the task that currently requires using multiple Weightmap Extractors saved to disk, and then assembling it all manually in the Splatmap Creator.
I am still considering how to perform such things as an automated transfer and viewing of the resulting Biome Splatmaps in the current Terrain Colormap Material.
This will also require creating a set of common coloring setups for the current Terrain Colormap Material system.

The new main viewport Terrain Auto-Material Material will allow for a higher quality rendered view of the terrain.
This will improve visualization of the terrain system.

Some of you may have also noticed the Mesh List Function Panel button and panel contents.
This is a future feature that will allow for placing a variety of primitives into the viewport scene, such as rough buildings and location placeholders.
This will also include such things as scattered trees for terrain size perspective.
The initial Mesh List functionality should be available in 2023.

I also still have some new Filter and Geology Devices in the works, which will be showing up randomly over the next few months.

-eof-

Development has been going quick on TerreSculptor 3.0, with a large number of new features in the past couple of months, and new build releases typically every 5 to 8 days.

As of January 9th the following new features have already been completed:
– The integration of LibTIF.Net and support for Large TIF and GeoTIF files up to 1 million x 1 million resolution.
– The new 2D Texture Plane view in the viewport for viewing masks and weightmaps more easily.
– All of the latest 11+ Devices have been added to the Terrain Stack, bringing the total number of Stack Devices up to 111 (still to be updated is the Composite Weightmap Extractor for 112 Devices).
– RAW Interleaved file format has been added, with support for up to 256 channels, and loading and saving into any channel number.
– All of the Gradient and Shape Generators received Previews for live editing of their parameters.
– Plus dozens of minor updates.

What will be happening during Q1 2023?
– More Devices will get multi-threading.
– More file formats: I am currently working on Large PNG support, Open EXR, FBX, PCX, and others.
– The new TileEx Creator. I decided to rename the Split Device to TileEx Creator. This version of TileEx will support tiling the full 1 million x 1 million heightmap size, with only support for grayscale formats.

What about Q2 through Q4 of 2023?
– The new Auto-Material Material which will color the viewport terrain with various gradients based on elevation and slope.
– The Biome menu will be available with the ability to create complex Splatmaps directly in one dialog.
– With the Biome features I would also like to implement automatic biome extraction.
– More Devices, including a bunch of new Geology Devices and more Filter Devices.
– Updates to the Convolution Filter Device with support for up to 9×9 size filters, providing new filters such as motion blur and others.
– The Quick Mask Creator.
– Additional work on the Mask Editor. This tool will eventually have complete painting and brushes and mask creation.
– I am considering adding another toolbar to the Terrain Stack so that viewing of the Datamap, Mask, ResultMaskA and ResultMaskB can be done in the viewport. Plus possible Mask Overlaid viewing.
– Plus whatever else I can fit in!

As a solo software developer, I typically work 12 to 16 hours a day, 7 days a week, on this project.  In order to bring you the new and updated features as rapidly as possible.

-eof-

The last month of 2022 saw a lot of updates to the TerreSculptor software.
The maximum heightmap size was increased to 1 Million x 1 Million, this will, however, require a computer with 4000GB of memory.
A good number of the Devices were Multi-threaded, with more still to complete.
There were seven new Devices released in December: Border, Dilate, Erode, Sinewave, Smear, Spin, and Stretch.
That brings the recent Q4 total up to ten new Devices including: Feedback, MinMax, and Translate, and the software total up to 112 Devices.
Note that Devices only get added to the main TSworld file format and the Terrain Stack after a few weeks of exhaustive testing, so the recent set of new Devices will be available on the Terrain Stack in Q1 2023.
Plus features added to the Terrain Stack, and interface updates such as being able to pan the 1:1 3D Preview mesh.
If you check out the Release Notes text file distributed with each new build, there is typically 50 or more new items and changes with every release, and build releases occur around once every week.

Starting in January of Q1 2023, there will be more Device Multi-threading.
Another six more Devices are planned for Q1 of 2023, but this time for the Geology area of Devices.  In addition to the new Geology Devices, there will be a few more Filter Devices.
Plus libtif file format integration to increase the maximum possible TIF file size to hopefully match the heightmap maximum of 1 Million.
Plus more GeoTIF file format compatibility with the libtif library.

High on the development list for completion is also the 2D Plane Texture View in the main viewport for viewing Masks and Heightmaps as a flat grayscale texture.
An Orthographic Camera view, plus updates to the existing Cameras.
Plus the Auto-Material Material system that allows for easy designing of complex terrain coloring by slope and elevation, which will also introduce some new color gradients specifically for this material.

Additional high priority features that should get some development time in 2023 includes: libpng to increase the PNG file format support and maximum size; EXR file format; FBX File Format; the Split Creator for exporting tiles up to the maximum heightmap size; DEM Scaling in the import/export pipeline; possibly integrating MapBox online DEM sources; adding a more complete Command Line Parser, adding Third-Party Plugin capabilities; the new Quick Mask Device, the Continent Creator, and more work on the Mask Editor; multi-lingual support for a few major languages; and much more.

Further future features include: additional Noisemap Generators; additional Extractors; additional Erosion Devices; more File Formats, Texture Generators; and much more.

-eof-

The BigArray Update was completed in about one quarter of the time scheduled for its development, with the first beta release within about four weeks after starting on it.
TerreSculptor’s code base for heightmaps now supports resolutions up to 2 Billion x 2 Billion which is 18 Exabytes.
The build release for Version 3.0 is using a maximum resolution value of 262,144 x 262,144 which is a 256GB heightmap.

The next major update under development is the Multi-Thread Update.
This will update as many modifiers as possible to support multiple processor threads for faster execution times.
A four-core processor will typically get a boost of around 350% faster code execution.

Multi-threading as many of the modifiers as possible will help with faster execution times for larger heightmaps.
This will also be noticeable on the Terrain Stack where multiple modifiers can be executed sequentially.

Some of the modifiers use algorithms that cannot be easily multi-threaded, and those will remain as single-threaded methods.

The Hydraulic Erosion modifier should be getting a performance boost, as should the Rain Erosion modifier.

In addition to the Multi-Thread Update, the next few months will also include the following major Updates:
– File Format Update 1: TSmap file format support for BigArray sizes including GZip – completed as of 332 Beta.
– OpenGL Update 1: Changes to some of the OpenGL functions for the viewport world – completed as of 332 Beta.
– Filter Update 1: Add Filter menu and move filter type modifiers to menu – completed as of 332 Beta.

Once the final build is released for the completed Multi-Thread Update, then the following months into Q1 and Q2 of 2023 will be used for the following major updates:
– File Format Update 2 which includes support for LibTif.net including better TIF image and GeoTIFF support.
– Filter Update 2 with new Filters.
– Modifier Update 1 with multiple new Modifiers.
– OpenGL Update 2 with larger Viewport World setup.
– System Update 1 with additional system information for the motherboard.
– and hopefully Material Update 1 with the new Auto-Material.

My big list of future features is slowly getting completed, but still has more than four dozen major updates and thousands of minor updates.

-eof-

As of next week I will be starting on the BigArray Update for TerreSculptor 3.0.
The BigArray Update is expected to take at least two to four months to complete, so no updates will be released during this time as this is a change that requires a complete code update.

The BigArray Update changes how the memory array system in TerreSculptor manages heightmap files.
The current array system is limited by the C# API to an 8GiB Floating-Point array.
This means that the current maximum heightmap resolution is 46329 × 46329 using this array type (46329 is the the square root of 8GiB).
For the BigArray Update I am changing the entire array system so that it allows for a maximum resolution of 2 Billion × 2 Billion values.
Initially the software will use constants and tests for limiting the maximum heightmap size to 1 Million by 1 Million pixels.
A 1 Million by 1 Million pixel heightmap would require a computer with more than 4TiB of memory.

The BigArray system will be completely memory based, so the maximum heightmap resolution will be limited to the installed computer system memory.
I am staying with memory based arrays rather than file based mainly for performance reasons.
This is because the current performance of DDR4-2133 is 40 times faster than an SSD at SATA 500MB/s speed.
File based arrays would mean that something that requires only 1 minute to execute in memory could require as much as 40 minutes or more to execute disk based with page swapping on a fast SSD.
I am also not simply changing to a tiled array system using the limited C# arrays, like most other heightmap software that is currently out there (you know who you are), because that results in edge issues between tiles for many of the modifiers and filters.
So in order to work on larger and larger heightmaps, more computer system memory will be required.
Note that as you go larger in heightmap size, the time to operate on that array data increases, so faster computers with more cores will also be necessary.

Many of the modifiers in TerreSculptor require multiple temporary arrays to operate on the heightmap.
I will be updating more dialogs to show how many arrays they require for operation, so that better estimating of total memory footprint can be accomplished.
There will also be updates to the supported file formats to allow supporting this change to larger heightmaps.

I am also working on multi-threading as many of the modifiers and functions as possible, so that larger heightmap sizes operate at the highest performance possible.

The maximum heightmap resolution for common memory setups is as follows, note that these numbers do not include the operating system footprint, so the actual available maximum size will be a bit less.

16GiB = 17179869184 bytes
= 4294967296 floats
= 65536 x 65536 maximum resolution

32GiB = 34359738368 bytes
= 8589934592 floats
= 92681 x 92681 maximum resolution

64GiB = 68719476736 bytes
= 17179869184 floats
= 131072 x 131072 maximum resolution

128GiB = 137438953472 bytes
= 34359738368 floats
= 185363 x 185363 maximum resolution

256GiB = 274877906944 bytes
= 68719476736 floats
= 262144 x 262144 maximum resolution

512GiB = 549755813888 bytes
= 137438953472 floats
= 370727 x 370727 maximum resolution

1024GiB = 1099511627776 bytes
= 274877906944 floats
= 524288 x 524288 maximum resolution

-eof-

TerreSculptor 3.0 2022.233 for August 21st is now available.

The main features of this release are:
– The PNG Image importer supports UE5 Landscape Tile Export.
– The Image and ImagePlus folders are persisted between sessions.
– The Normalmap Creator supports saving to TGA file format.

For the future build releases over the next year, the main features under development will be adding multi-threading to the single-threaded Modifiers, and adding multi-lingual support to the dialogs.
Both of these features are estimated to take up to 2000 hours of work.
The multi-lingual support will be added to the dialogs for French, German, and Spanish, which might require some dialog control resizing.  This language support will be slowly added to each dialog, so for the next year or so the software will be in flux for which dialogs have been converted and which are still pending, so the software will be a mix of the current culture and the default English.

TGA file format is also being added to the Bitplane Creator and Splatmap Creator file save functions.

-eof-

Over the next six to twelve months, which are Q3/Q4 2022 and Q1/Q2 2023, the following items are planned for development in TerreSculptor 3.0.
These are all Build Updates which will be completely free for all licensed users of the 3.0 software.
To see the list of build updates for each current build release, see the Release Notes text file, available on the software’s Help menu.

1. The Multi-Thread Update.

Somewhere around 20% of the modifiers are currently multi-threaded. That leaves 80% as currently single-threaded.

The threading system in TerreSculptor supports virtually unlimited core-count and thread-count, although it is currently limited to 1024 maximum threads in easily-extendable code.
It will be a while before we start to see 512-Core HyperThread mainstream consumer processors and computers.

Many of the modifiers execute in less than one second even on large 16k+ heightmaps.
However, I would still prefer that every modifier possible be threaded, simply because now with the Terrain Stack, modifier performance can start to impact Stack build times.

With currently 98 Devices, and about 80% of them requiring multi-threading, even if it only required 10 hours per Device to update the code, that is still about 800 hours of development time.
So the multi-thread update will consume a large portion of the next few quarters.

Some of the Filter Functions will be difficult to multi-thread, so expect probably 5% of the Devices to not have multi-threading.
I have some tricks up my sleeve for working on threading such difficult Devices as the Rain Erosion, I still have to test my code hypothesis to see if it is functional.

2. Auto-Material.

I have been lightly researching the addition of an Auto-Material to the current Terrain Material set.

This Auto-Material would support two elevation values and two slope values.
This would allow for beach/snow for elevation, and shallow/steep slope for grass and rock variance.
The terrain coloring would of course be something such as sand for beach, grass for shallow-slope, rock for steep-slope, and snow.

The Auto-Material would provide similar terrain coloring to what the Splatmap system typically supports, but with an easier to use interface and no splatmap bitplane extraction required.

3. Biome Extraction.

One of the features that I have been working on for some time is an automated Biome Extraction and Splatmap Creation system.

Additional work will be performed on this feature over the next few quarters.

4. Digital Elevation Model File Header Parsing.

One of the features that I would like to see in TerreSculptor, is better handling of the Header Information for Digital Elevation Model files.
With the ability to automatically scale the terrain mesh to the file information.

I have still been working out exactly how I am going to manage this, without being obtrusive, and performing the sizing logically.
TerreSculptor is not GIS software, and is not in that market segment at all, but I would like to see more DEM support features over the next few quarters.

5. Pathspec Persistence.

You may have noticed (or not) that many of the drive and folder paths are persisted between sessions in TerreSculptor.
This is a feature that I have had since the early builds.
Nothing is more frustrating than having to constantly re-navigate back through drives and folders to re-load some image file.

A few additional Pathspec Persistence settings are required, such as for the Alpha Brush path for the Brush Modifier.
It gets irritating to have to constantly re-navigate back to that folder if it changes with another software operation that shares that current persisted path.

So the new Brush Pathspec Persistence code plus a few others will be implemented over the coming weeks.

6. Odds and Ends.

My documentation file that contains the list of future features contains well over 1200 entries of major features plus a few minor features and updates.
This is a compact list of a short description of each future feature that I want in the software.

At the top of this list is always the highest priority or most recent minor items that require the quickest attention for code development and updates.
This section of the list is usually at least 30 or 40 items that should be completed within the next quarter or two.

So these “odds and ends” will also be showing up in the next few builds over the next few quarters.

-eof-

TerreSculptor has been increased to Version 3.0 with this latest build release.
This is due to the large number of changes in the software:

– It is now retail, which was a tough decision.
– The new Terrain Stack, which is a major milestone feature.
– The new Brush Modifier, which is a powerful feature.

The Terrain Stack was in the original design documentation from the beginning, to provide an Autodesk 3DS Max or Blender 3D style of Modifier Stack, with more features such as Device Item Masks.  The Stack has now been completed to its first release iteration, with additional features planned for the future.

The Brush Modifier is a new concept and feature, employing Unreal Engine 4/5 style Landscape Alpha Brushes in a new Modifier.  This feature works well up to terrain sizes of 8192 and possibly even 16384, although the preview starts to get a little small by that resolution.  Future design considerations may include 3D Widgets for Location, Scale, and Rotation in the main viewport when a Brush is selected in the Terrain Stack — we’ll see how that goes.

I decided to take this new version to retail.  This choice was made because Patreon Support didn’t come anywhere near covering development expenses, and I gave it a few years to see if people would step up.
With more than 100,000+ downloads, there are only about 30 Patreon subscribers, with most paying $1 per month ($12 per year).  That is only 0.03% of the people who downloaded the software decided to help support it.  Or about 1 person out of every 3400.  And since going with Patreon I have brought in just under $3000 in total support income.  That doesn’t even cover one of my software development yearly subscriptions for the years that I was on Patreon.
Over the past few years I have gone into debt by almost $40,000 for development tools and development software, to provide the TerreSculptor software for free to the terrain design community.  I decided that I couldn’t continue on that trajectory, since I am a single individual making only average income in northern Canada.
So the choice was made to make TerreSculptor 3.0 now retail software.  However, a low price was chosen so that the software is still obtainable by people in countries with low income.  The price of $99 CAD or about $77 US was chosen, which puts TerreSculptor significantly less expensive than any of the competing terrain software, and even less than their Indie versions which have significant limitations.  You get the full TerreSculptor for the low price.

There are many new and powerful features still under development for the software.  The plan is to release new versions every 12 to 18 months for resale that will include major feature updates, such as the File Format Update and the BigArray Update.  And build updates every month or two that are free updates for those who have purchased a copy of the software.  The free updates will include such features as bug fixes, performance enhancements, multi-threading of devices, and other minor features and improvements.

For anyone who downloads the version 3.0 builds and tries to run them without a registration key, the software will run in Demo Mode for about three minutes.  TerreSculptor 2.0 is still freely available to anyone who wants to use it.

-eof-

Alpha Brushes are a feature that Unreal Engine 4/5 uses with the Landscape actor to interactively create terrain layouts from terrain shape brushes, such as various mountain shapes that can be stamped onto the terrain.
These Alpha Brushes can be positioned, scaled, and rotated, anywhere on the Landscape actor.
Alpha Brushes are often real-world terrain details.

https://docs.unrealengine.com/4.27/en-US/BuildingWorlds/Landscape/Editing/Brushes/

TerreSculptor now includes support for these same Alpha Brushes with the Brush Modifier.  The Brush Modifier allows for interactively stamping alpha brushes onto the current heightmap.  The Brush Modifier supports the blending of one alpha brush file, and can be used iteratively on the menus
When used in conjunction with the now-functional Terrain Stack, multiple Brush Modifiers can be placed onto the stack, iterated and edited, and a final complex brush layout can be achieved, such as complete mountain ranges, areas filled with various size craters, and special terrain features such as volcanoes, all with non-destructive editing.
The TerreSculptor Brush Modifier supports multiple Blend Modes and a fully interactive user interface.

The Brush Modifier should ship in the same build as the Terrain Stack later this year.

TerreSculptor Terrain Stack Update
This major Update should be publicly available in Q3/Q4 2022.

Q. What is the Terrain Stack in TerreSculptor?
A. The Terrain Stack is a highly interactive device operation layer system that allows for non-destructive workflow of terrain creation.

Each of the items on the Generate, Noisemap, Weightmap, Adjust, Modify, Transform, and Erosion menus are available as ‘Devices’ that can be added to the Terrain Stack, so that they can be edited and executed in any logical order, and re-edited and iterated.
This allows for a flexible terrain design system that can be edited over and over again, until the desired final terrain look is achieved.
An example Stack may include the following Items in descending order: Gradient Noisemap, Altitude, Altitude Center, Displace, Rain Erosion, Hydraulic Erosion, FloodLevel, Normalize, Datamap Save.
This Stack of Items can then be built and the device properties iterated over and over until the final desired terrain design is realized.
The Terrain Stack can be saved to a file, so that a library of common terrain design systems can be created.
The Terrain Stack can be thought of as similar to the Layer system in Photoshop, but more accurately to the Modifier Stack in Autodesk Max or Blender 3D.

Q. When is it available?
A. The initial Terrain Stack is already in beta testing with the Patreon subscribers.

There is still a lot of work to be completed on the Terrain Stack before it is ready for public access.
There will also be Feature Updates to the Terrain Stack over the next few years.
So while the Terrain Stack is currently functional, it won’t be released in a public build for a few months still.

Q. What is remaining to be completed?
A. Some of the current features that are pending on the Terrain Stack include: Variable Size Stack, Modifier Stack Pointers, and Unique Item GUIDs.

The Variable Size Stack allows for such Modifiers as Resample and Rotate Custom, where the dimensions of the Stack can be changed on a per-Device level.
The Modifier Stack Pointers will allow specific Modifiers such as Blend, Combine, and Shaper, to reference a Stack Item as a secondary Heightmap or Mask.  For example, the Blend Modifier blends two heightmaps together, and the Modifier Stack Pointers will allow it to use a Stack Item reference instead of having to reference an external file on disk.
The Unique Item GUIDs will allow the Modifier Stack Pointers to track changes to the Stack Item order and “fix up” pointer references automatically.

Q. What advanced features will it include?
A. The Terrain Stack will eventually include a per-Item Mask, and it will eventually be used for the 2D Terrain Paint and Brush system, and the future 3D Spline Road River and FaultLine systems.

The Item Masks will allow for such things as complex biomes, by allowing for masking out regions that the Modifiers operate on.  For example, Terraces can be applied to one region of the terrain, while FloodLevel is applied to another region.
The future Paint and Brush systems will allow for using paint tools to modify the terrain elevations, or terrain brushes to ‘stamp’ a terrain shape such as a mountain range onto the heightmap.
The future 3D spline modifiers will include a spline in the main viewport that can be manipulated to modify the heightmap.

Q. What else is coming soon?
A. Coding will also begin in Q3/Q4 on the ‘CLI’ Command Line Interface Update, the File Format Update, and the Project Update.

The CLI Update will allow for automation of TerreSculptor functions through the command prompt and through batch files.  Initial CLI functionality will be such things as heightmap file format conversion.
The File Format Update will be bringing EXR image file format and FBX mesh file format to TerreSculptor, in addition to supporting libpng and libtif to improve the total supported PNG and TIF file formats.
The Project Update will be adding the newer Modifiers and the Terrain Stack to the TerreSculptor Project file format, so that these items can be saved and loaded with a Project file.