Ecology Simulation List

December 18th, 2024

I created S4L5_Ecology using C++. S4L5 was useful for learning how to use C++. It showed me the good parts and those which are less useful. C++ worked OK but it did not scale well. I could not get to 1000 x 1000 cells.

How it began: Ecology.pdf

S4L5 Lesson: S4L5_ecology.pdf

S4L5 executable: eecology.exe

Then I wrote ecoNode.c using C. I recreated my simulation using a few techniques I have developed. Buffering helps organize and accelerate the app. Simplifying indexing made writing the code much easier. I can work with a 2D E(i,j) ecology cell or a 1D PL(i) plant node. Once I got the addressing framework developed and debugged I was able to increase my problem space first to 3000 x 3000 and then to 5000 x 5000 where it is set now. The new tools let me breed many more organisms. 2 million plants is not uncommon, nor are 500 thousand herbivores, or 200 thousand carnivores. I have had stable ecosystems working beyond 250 thousand cycles.

Since ecoNode is a command line application, I needed a way to monitor the population levels of the ecosystem. I have been using gnuplot for many years; I enjoy its simplicity and ease of use. I generate many data files to monitor my code. I use gnuplot to read those files and create 2D graphs. It naturally plots with linear x, y coordinates. If I type "set logscale" I get a log-log chart of my data. Or type "set logscale y" to get a semi-log chart. Each chart has its uses by showing some characteristics better than others. I tried 'set logscale x' and found another way to look at changes over time. gnuplot is very useful to have on your computer. My normal procedure is to open two terminal windows pointed at the same file location. One of them is running my application generating a data file. The other terminal window is running gnuplot which reads that data file and generates a 2D chart.

Find gnuplot here: gnuplot

A plot of the ecosystem's population data.

A log log plot of the ecosystem's population data.

A semi-log plot of the ecosystem's population data.

I implemented many values as floating point where they had been integers in S4L5. Tracking the energy flow through the environment is easier. That allowed me to determine how much energy was being lost to metabolism, or through senescent organisms. Once I knew those values I could replenish exactly that much back to the nutrient layer.

ecoNode: Ecosystem_simulation.pdf

ecoNode source code: ecoNode.c

ecoNode Linux executable: en

ecoNode Windows executable: en.exe

I reused code from a graphics template I've built. I modified the main loop of ecoNode.c so it would run as a thread in a Windows application. I wanted to retain the speed of the ecological engine, so I created another thread to display the ecosystem. I did not want to slow down either one by synchronizing them. So there are glitches. However, they are few at first, then almost non-existent once the simulation has matured. Since they work in a producer consumer symbiosis they can get out of sync. Especially when population counts drop during the first two hundred cycles. Once, and if, the ecosystem recovers equilibrium, cycles require over a second to complete. Modifying the bitmap takes about one second. When the producer creates frames faster than the consumer can display them you will see the cycle count jump erratically.

The background bitmap is 5000 x 5000 pixels square. 25,000,000 pixels take a while to change. My foreground display is a window onto that background bitmap. You can move the display around by scrolling the mouse, using the scrollbars, or by using the arrow keys on the keyboard. The display does not scroll smoothly; there are blanks. This is also due to the size of the background bitmap. I need to work on this; but it recovers quickly. The data display is a small, separate window which is blitted on top of the ecosystem window. I fill that window with black and draw the text on top of it. Then I blit it using the SRCINVERT raster operation. Notice how the color of the letters change as the underlying image changes. This is especially noticeable when a herd of herbivores pass underneath the numbers. It appears they are eating the legend :)

The colors designate various things. The gradient from dusky orange to white is the amount of energy in the nutrient layer. Brighter points have more energy than the dusky orange ones. Notice what happens when plants pass through an area as they go through their life cycle. Energy is lost via metabolism and through the death of senescent organisms. This amount of energy is evenly spread back to the nutrient layer. Over the cycles, some areas get so energetic they pass beyond my color palette. They indicate as black dots. When plants grow over them they breed rapidly. However, the herbivores always seem to find them before they're done feeding on the available nutrient energy.

Plants are green, herbivores are blue, and carnivores are red. When they overlap they produce shades of violet. The color palette for the nutrient layer starts at blue, goes to red, then yellow, and finally white. I chose the lowest color I could and still have enough contrast to see the carnivores. The topmost color is white, which is set at 6000.0 energy units at present. When the energy level exceeds 6000.0, the pixel turns black. However, this does not happen too often. If it does change the upper range of the color palette in colorTransform().

I am going to continue work on the user interface, and work on sampling and marking tools. I will probably add a tool to create 'synthetically' created sample files. That will let the user modify the individual genes of each chromosome of an organism. Then clone one thousand of them and add them to a stable ecosystem. Use the GM chromosome to follow your new organisms as they live and breed. Lots of other fun things to do. Like expressing more genes to give the herbivores and carnivores more behavior. I have a sensor net idea which would help the carnivores find herbivores more easily. Either by scent or hearing or sight. I'm not sure which; the ideas are working themselves out.

Plants, herbivores, and carnivores have many traits which are genetically determined. Some of the 36 tweakable parameters are an offset to a gene value. Instead of looking at genes solely as integers, I implement them as floating point values, or as factors between 0.0 and 1.0. For instance: if my gene is four bits long, its raw value can range from { 0 .. 15 }. Using zero is not good, it can lead to problems. So I add one to get the range { 1 .. 16 }. If I divide the raw value plus one, by 16, I get a range of { 1/16 .. 16/16 } or { 0.0625 .. 1.0 }. That is good for something like the absorption rate. For instance, a plant can gain 5/16 of the nutrients available in the cell. If I use a factor here, instead of a set amount, I avoid negative numbers. 5/16 of very little is still very little, but not below zero. When I saw the first images from ecoDraw I was reminded of paintings by Joan Miro. The palette was not chosen to that end, I think :)

Some of the user tweakable parameters are offsets from the gene value. Say your gene is 3 bits long, which ranges from { 0 ... 7 }, instead of using an offset of 1, use 3 to get a range of { 3 ... 10 }. Use casting to make these values floats instead of ints. Remember: 10 bits gives you 1024 steps, 8 bits gives you 256, and 3 bits gives you 8 steps. Build your genes to fit your problem settings.

A rudimentary gene map: geneCruft.c

ecoDraw.exe was compiled under a Windows 7 environment. It will run fine on any Windows machine of that era or newer. The application requires 1.3 GBytes of memory. It also employs three threads of execution. Two threads run at ~100% duty cycle, the third thread is for the UI and requires little processing time. Fast, multi-core machines work best. One of mine is creating one frame every two to three seconds. Your mileage will vary. One thousand cycles an hour is good.

ecoDraw: ecoDraw.pdf

ecoDraw source code: ecoDraw.c

ecoDraw executable: ecoDraw.exe

I rewrote ecoDraw.exe so it can start without any other files. It is more boring, and the ecosystem thus created will likely crash. If you have sample9GM.dat you are using organisms bred over many cycles. If you have the nut.dat file you are using a mature nutrient layer which displays a wide variation of energy levels. This creates an ecosystem from data sampled during long simulation runs. Without it the system is populated by organisms with random sets of chromosomes. A few boom, bust cycles follow. If you are lucky, the carnivores will not die out and your simulation will stabilize.

I normally work from a command line in a terminal window under a Linux OS. Any debugging or logging operations go directly to this terminal window. In Windows parlance this is a command window. If you launch the application from the Windows file system it will create a display window then open a terminal window. I use the data flashing by to monitor the energy condition of the ecosystem. If it is too annoying I can change the compiler flags to turn it off. I just want you to know a command window will appear.

If you are running a Macintosh, or Linux, operating system the Wine translator works fine with my code. I use Virtualbox to compile in a Windows 7 environment. Then I run the executable under Wine so each operation has been tested many times. I have not tried running my code on a Mac. I would like to hear how it works for you.

Find wine here: wine

When you start the application you are presented with a series of six dialog boxes, with three choices each. I recommend you hit ENTER for each of them, to use the default settings, until you get a feel for the program. During any run you can hit the 'h' key for a help list of your options. I implemented a keyboard UI instead of using menus. They are simpler and take less screen space. While the simulation is running you can change a few settings. Call up your chosen dialog box and the simulation will pause. Modify a setting and hit OK, or CANCEL. The simulation will resume running with your change(s) intact controlling the ecosystem's destiny.

There are a number of non-linear relationships, so the parameters may be finicky. Often the effects of a change will not be visible for many cycles. Monitor the system health by using gnuplot on 'dat.dt'. Chart both the population levels and maximum energy levels for a good idea of what's going on. If you see a steady decline in population over a few thousand cycles there may not be enough energy in the system. But, too much energy is also a problem. Change the parameters carefully, and with forethought to achieve good results. Sometimes the settings are counterintuitive :)

Incrementing settings one click at a time is tedious. Use the tab key to choose the up, or down, arrow you want. Then if you hold down the ENTER key you can change the value much more rapidly. Tab to the next arrow key and repeat. Laziness, Impatience, and Hubris are three great virtues :)

Currently Tweakable parameters:

int Ppop = 500000, Hpop = 230000, Cpop = 50000;

float PRenergy = 12.5, HRenergy = 65.0, CRenergy = 35.5; // Reproduction energy level hard wire

int PNoffset = 3, HNoffset = 19, CNoffset = 39; // Senescence - EN::G1 + offset

float PSoffset = 19.1, HSoffset = 56.1, CSoffset = 8.0; // Satiety - EN::G3 + offset

float PMoffset = 7.5, HMoffset = 4.2, CMoffset = 5.3; // Metabolism - EN::G5 + offset

I added the GM chromosome to each organism. Previously, only plants had it. GM contains 31 bits of information. I use it as a marker to trace inheritance. The GM chromosome is passed directly by the mother organism without any crossover. I have been using it to see which genes compete better than others. Or a random population compared to a mature population.

When I had the genetic marker chromosome in place, I needed tools to analyze my sample files. The first thing I did was count how many plants had a genetic marker as a percentage of the whole population. Then I generalized extracting genes from chromosomes. I created a data structure, gmap, which holds the mask and the shift amount, for each gene in a given chromosome. This has to be mapped by chromosome by organism since they are not all the same.

I wrote readSamples.c to display chromosomes and extract genes. Once you extract the raw data you need to transform it into integers or floating point numbers. Each type can have offsets and divisors. I wrote the function compareGenes() to examine the competition between a plant's poison level and the ability of a herbivore to resist poison. Or the ability of a herbivore to hide from a keen sighted carnivore.

Every 50 cycles I randomly sample the ecosystem for 1000 plants, 1000 herbivores, and 1000 carnivores. I store their genes, energy levels, and ages for analysis. I use readSamples() to read the organism data in sample9GM.dat. I want to automate this because I am seeing some interesting changes. For instance, the simulation selects only carnivores with excellent sight. It also breeds for herbivores which the ability to resist poison at the highest levels. Conversely, the plants breed higher poison levels, but they do so very slowly. Similarly, the herbivores breed for better camouflage rather slowly.

A random sample of plants, herbivores, and carnivores is collected and stored every 50 cycles. I wanted to sample interesting groupings from the simulation while it was running. So I created a spot sampling method. Display buffering prevents me from drawing the characteristic rubber band selection scheme. But I can capture two clicks in order. If the user did not sweep from upper left to lower right I flip the coordinates in the code. The two click points select a rectangle from the E(i,j) ecosystem. The function scans the rectangular area counting, and logging organisms. I have not written the tools to read and analyze the data collected. I am using it for spot population counts. You can open spot.dat in an editor and update it each time you pick a new rectangle. I need to expand readSamples.c with its own user interface. Then it can compare genes more easily.

You can store the nutrient layer at any time by hitting the N key. This stores nut.dat which is a map of the current nutrient layer. I let one of my laptops run ecoDraw.exe for long periods of time. They can go to sleep so I have runs over a week long. I have been saving nut.dat from them. However, I rewrote ecoDraw today so you don't need nut.dat, or sample9GM.dat, to get started. If the application doesn't find either, or both of them, it places random organisms on a uniform nutrient layer. If you can get the system to run until it becomes stable save the nutrient layer and a sample9GM.dat file for the future.

January 9th, 2025

I have expanded readSamples.c to create longer reports. Instead of reading the data randomly sampled every 100 cycles, I wrote a new routine. Click on a point bounding an interesting growth feature. Then click diagonally opposite from the first point. That creates a spot sample file of all the plants, herbivores, and carnivores inside the rectangle you chose. Now that you have 'spot.dat' you can analyze it by typing ./rs (if you have compiled readSamples.c < g++ -o rs readSamples.c > ).

The report begins with the present cycle count. Each cycle is not really a time unit. It is not a generation and it is not a day. It is one cycle through the ecosystem loop: move, metabolize, eat, reproduce, die. One organism can live as little as one cycle, or as long as ninety cycles. The former organism was probably eaten while the latter is close to senescent. It is good to choose a large number of all organisms. This gives a better spot sample. I wanted to track the traits of 'herds' of herbivores or carnivores. Or track the ages of a meadow of plants. Or compare the energy levels of the organisms to the energy level of the nutrient layer.

Plants, herbivores, and carnivores have traits which compete with one another. For instance: plants have a poison strength gene, while herbivores have a gene which allows them to eat some level of poison without harm. Herbivores also have a camouflage effectiveness gene which competes with the carnivores' eyesight gene.

The next two genes are chosen as a null test. Plants have a gene for spine presence and herbivores have a gene which allows them to eat some level of spines. I did not express this limit in the code, so they are not in competition. However, while the carnivores select for the very best eyesight, and herbivore's resistance to poison grows better, plants don't select for more poison at the same rate. Neither do herbivore select for only the best camouflage. These two genes improve around the same rate as do the un-competitive genes. Hmmm...

I wondered if there was any bias in direction choice so I created a tool to test this. The tool is now incorporated into readSamples.c. It produces this report:

Each herbivore and carnivore has a DR chromosome. Each chromosome has nine genes mapped onto it. Eight genes, of 3 bits each, represent eight points of the compass from E to NE to N to ... to SE. I sum the value of each gene then go back and divide each raw value by the total. This gives me a normalized direction probability vector. A perfectly balanced animal would have 0.125 in each bin. It is interesting to see how separate 'herds' have different direction biases.

Lastly there are lists of average values from plants, herbivores, and carnivores in your sample.

I am using this report to modify the gene expression coefficients. Presently I have no idea why plants have an average senescent age of 17.98 but rarely get more than five cycles old. I also don't understand why the metabolism of plants is higher than either herbivores or carnivores. I need someone more familiar with the energy flows than me. Spot sampling has been instructive. I have removed, and then reinstated senescence for all organisms. Without it the population levels grew too fast and too large. There are over forty parameters which can be changed to modify the system's health. Hopefully someone will find this interesting enough to continue my quest.

I created a better help window. Press the 'h' key to get:

The dialog lists some of the parameters which can be changed during a simulation. You can only modify the population levels at the start. Otherwise all other parameters can be modified. Press the correct key to pause the simulation while you make changes to a parameter or two. Once you hit OK or Cancel the simulation resumes where it left off. The help dialog box does not pause the application. If you press 'n' you log the current nutrient layer to nut.dat. This file is useful to start the ecosystem with a varied nutrient layer. When you place new organisms on top of this some do very well, while others perish from lack of nutrients. A uniform layer of nutrients shows another growth pattern which favors different genetics. The spot sample reports are quite useful to adjust the framework.

January 28th, 2025

I rewrote ecoDraw.c so it loads 'nut.dat' and 'sample9GM.dat' if they are available. If they are not in the current folder ecoDraw.c generates two random sets of data. Then it creates a population of plants, herbivores, and carnivores using samples from both data sets. Currently it creates one half from one data set and the other half from the second data set. I plan to let the user inject new organisms during the simulation. The genetic marker chromosome GM is on each species. I have one data set marked as 3855 = 0000 1111 0000 1111 and the other set as 61680 = 1111 0000 1111 0000. I chose the numbers to contrast their binary representations. You can mark them however you wish. More tools to write :)

Previously I used readSamples.c to get a report. Now I have implemented that code inside ecoDraw.c. Find an interesting group of herbivores and carnivores on the display. Click above, and to the left of the center of interest. Click again diagonally across the feature. After the second click, a report dialog box appears. Currently there are six report options in a row of radio buttons. The choices include: energy, direction, compare, absorption, spine war, and markers.

Energy displays the average plant metabolism, average absorption coefficient, average satiety level, average senescent age, and minimum, average, and maximum age of plants.

Direction displays the normalized direction probabilty vectors of herbivores and then carnivores. These are not histograms, but coordinate values of the probability vector. Next their average speeds.

Compare displays the minimum and maximum energy values for plants, herbivores, and carnivores. Then it compares a few competing genes. Poison level versus poison resistance. Then camouflage effectiveness versus visual acuity. Lastly spine probability versus ability to eat spines. I chose these genes to see how they populations changed over time. I found herbivores could gradually eat higher levels of poison. But I also found carnivores with excellent eyesight dominated quickly. I used the spine vs ability to eat spines as a control group. Since I had not implemented the choice in the eating algorithm there should be no tendencies in either gene. This choice is mirrored in spine war. Here you see the percentages change over time. I created histograms in the spine war report.

Absorption introduces histograms. I determine the end points of the values of each trait. Divide the result by ten and use it to create the bins of histogram. I calculated the value of each trait, counting how many fit into each of the bins. Here I have the plant energy absorption coefficient histogram. This shows how much energy each can absorb from the nutrient layer during each cycle. Next, the herbivore energy absorption coeffient histogram. This displays the same trait for herbivores. Lastly the carnivores histogram. Use the reports to track certain traits over time.

Spine war displays the camouflage effectiveness, visual acuity, poison strength, poison resistance, spine probability, and spine eating histograms. Click on Compare to check the similarities.

Markers displays how many plants came from the PS or the PB buffers. Restart ecoDraw.exe with 'sample9GM.dat' in the folder. Make sure it is from a long simulation run so the chromosomes have been highly selected. Then rerun the application. After a few hundred cycles take a spot sample and compare their genetic markers. The 'mature' genetics should far outnumber the new, randomly constructed organisms.

A random selection from the population is stored every 50 cycles. It is called 'sample9GM.dat'. If you press N you save the 'nut.dat' file, a copy of the nutrient layer. Start ecoDraw.exe with one or both of these files present. Random populations on a mature nutrient layer is a good choice. Or use the sample file for mature genetics on a uniform nutrient layer. Or, if you have both 'nut.dat' and 'sample9GM.dat', you are using a mature nutrient layer with a highly selected genetic population of organisms. It is up to you, and how you wish to experiment. When you are done reading reports click the dialog's only button, or hit the enter key to exit. Notice this dialog does not block the execution of the simulation. You can read each report as they ecosystem continues without pause. Keep track of the gene competition by taking spot samples and reading the reports. Have fun!