In 2002, Microsoft developed a fun little developer game called Terrarium that demonstrated the code portability of the .NET Framework. Back then, C# and .NET were just newcomers and Microsoft had a tough sell; after all, it was dethroning VBA and Visual Basic as well as trying to steal away Java developers. The game allowed developers to build tiny programmable creatures that would be born, eat, sleep, reproduce, and die in a terrarium. It also featured the ability of allowing tiny programs to infect other connected terrariums. In fact, Microsoft has a contest that pitted these tiny creature programs against each other and the winning developer was the one that had the most successful creature. While we won't go so far as to build a connected terrarium infrastructure, we will do our best to replicate a number of the cooler features that made the original game so entertaining (as a developer, anyway).
The focus of this chapter will be to develop a similar terrarium game which allows our agents to live and die, but with a few twists. While neural nets and other ML technologies were available at the time, virtually all the terrarium creatures were based on A* and heuristic methods that were created by developers. Now, we have the ability to let the little terrarium creatures program themselves and develop their own winning behaviors, possibly in ways we have never even thought about. This is going to be an exciting chapter and here is what we will cover:
- What Terrarium was/is
- Building the Agent ecosystem
- Basic Terrarium – Plants and Herbivores
- Carnivore: the hunter
- Next steps
- Exercises
In this chapter, we have a lot of ground to cover and we will need to go over some advanced Unity concepts quickly. If you find yourself in trouble, download the book's source code and look at the completed example.
If the concept of growing your own little programmable creatures in a terrarium doesn't excite you, then you probably picked this book up by accident. Please give it back to its rightful owner, thanks.
![]()
Terrarium was a concept that was probably ahead of its time. While the game did allow developers to transfer some form of state between parents and offspring, it was a feature that was never used in successful agents. Instead, developers just optimized their code for a particular set of fixed strategies, which in many cases turned out to be a lot of code. The following is an example screenshot of a Terrarium client running in ecosystem mode:
Original Microsoft Terrarium
Now, the bottom line is that we are not going to be able to replicate the entire Terrarium connected ecosystem concept in one chapter. The elegance that was the Terrarium network ecosystem infrastructure could possibly take several chapters to explain. Instead, our goal is going to build a multi-agent multi-brain ecosystem that is designed to teach you more about building an ML-Agents environment and Deep Reinforcement Learning. Perhaps someone like Microsoft or Unity will take the connected ecosystem concept further – who knows.
The Terrarium game rules were fairly simple and one of the basic rules was that your code had to run in x amount of time without failure or the creature died. There were other aspects that allowed you to apply points into a creature's abilities like size, speed, toughness, and so on that we will focus on. Developers had 100 points that they could apply to various categories for creatures of type carnivore and herbivore, as shown in the following sample tables:
| Attributes | Herbivore | Carnivore | Description |
| Max Energy | 20 | 10 | Determines the amount of energy the creature can store for actions. Carnivores consume energy by eating other creatures. Herbivores eat plants. |
| Eating Speed | 2 | 6 | Determines how fast the creature can consume energy from another creature. |
| Attack Damage | 0 | 14 | Amount of damage another creature causes in an attack. |
| Defend Damage | 10 | 10 | Determines how effective a creature defends from an attack. A creature with a higher defense will inflict damage back on the attacker. |
| Max Speed | 8 | 10 | Determines how fast a creature moves. |
| Camouflage | 25 | 5 | How difficult it is to see the creature. |
| Eyesight | 5 | 20 | How far away a creature can detect other creatures. |
| Mature Size | 30 | 20 | We are adding Mature Size to the original parameters in order to hopefully make the game more balanced. |
| Growth Rate | 5 | 5 | This is another parameter that has been added to manage creature growth a little cleaner. The original game handled creature growth a bit like a black box. |
On top of this, all creatures have a Mature Size attribute that can vary from 24 to 48, which corresponded to the icon's image size in the original game. We will map all of these attributes to various game mechanics when we build the example, but for us to perform reinforcement learning, we are going to need to apply some form of reward structure as well.
Previously, all of our rewards have been goal orientated; get to the goal and get a reward. With Terrarium however, the goal was and still is to make our agents live as long as possible and reproduce. Therefore, we want to introduce a reward system that also mimics this. We will do this by giving rewards based on the following events:
| Event | Reward | Description |
| Die of old age | +1 | Our goal will be for the creature agents to survive as long as possible. |
| Reproduce | +1 | We want to encourage reproduction; more agents means more learning and food for others. |
| Eat | +energy eaten/100 | We give this micro reward in order to encourage creatures to eat. Without it, they may realize they don't need to do this and just continually die of starvation. Besides, everyone enjoys e... |
