October 2020

Since I originally made this page, I have gone through four or so iterations on Galapagos and it's changed a lot because actually following the principles I set out is really difficult!

What happened is that as I drew closer and closer to a working runtime, I started on tools and frontends and started to discover all these unresolved issues. How many settings should be presented? Well, if the intended format is simple enough, none. But if it's like an emulator, as many as possible. What about saving data? How do I archive it in a way that's "sustainable"? If you're debugging a project you want extra settings. And so on and so forth.

What I gradually realized I had run into - and as March rolled around and lockdowns hit, started to contemplate more deeply and gave a name to - is what I call the "terrarium problem". The terrarium problem is this: the larger your programming ecosystem is, the more of it you have to carry around, and the harder it is to reproduce. If you try to cover everything in your terrarium, you get a modern operating system like Windows. If you try to cover a minimal set like PICO-8, you still encounter edge cases over the smallest things like how standard libraries behave or how joystick input is polled. And most of those features are commodities - they don't really influence the creator or the audience because they're taken for granted, but they have to exist. If you(as the designer of the system) try to kick the can down the road and implement solutions with high generality, you haven't really solved anything - the user still builds their own terrarium, and each time you do that, the "point" of using the system as a functioning whole diminishes.

This was a particularly big problem with the idea of using WebAssembly. When you look at it from afar, it seems like a VM that runs compiled languages is just the thing. But the closer you get to building with WASM, the more you become entangled with the specific mechanisms by which WASM is built - which assumes you are, in fact, targeting a web browser. It's not just code and memory. The standard library has to bootstrap off of something to get access to I/O, and so all this other stuff gets baked in when you build your code. I concluded that it wouldn't be sustainable at all, in fact!

So I looked into ways of realigning the project to reduce the size of the terrarium. This led me through a cycle of build and compilation systems, further changes to backends, further changes to the programming model to embrace source code first, another version of compliation systems using a "binder" organizational metaphor, and finally, the one I'm feeling settled on right now:

A common binary data format, a set of libraries compatible with existing game engines (which I will write reference code for), and some tools for working with those things.

This is less integrated than I originally imagined, but what I determined is that a sustainable system that addresses the terrarium problem has to coexist with change and chaos, and be able to reorganize. And this does that. It solves the terrarium problem along a few different axes:

• The binary data format helps clarity in a way that most common formats today don't - it's defined to have unambiguous division types - common primitive and container types that can be completely described just as divisions of bytes(various sizes of numbers, strings, arrays, key-value maps), which can then be extended with intent types defined by users and libraries. So it degrades gracefully when a tool doesn't understand everything - you can still extract the data in terms of how the bytes are organized, even if you don't know the meanings. And it doesn't try to guess at things like textual formats do - you won't encounter the string "yes" parsing to "true", or assumptions of a time being linked to a time zone.
• I can build small editors and small libraries on top of that format, with correspondingly small interfaces - simpler to make and to reimplement.
• If you write to the data format, you can mix and match with the engine you want to use - the boundaries are permeable and it is robust to change and chaos around it.
• It lets me write more specific code, which is better in terms of something that gives you leverage as a user and in terms of indulging myself.
• It allows the commodities to remain commodities: If the primary content of the experience you're making is conveyed in Galapagos formats, the experience is easier to maintain and to remake, because more of it exists in a legibly documented form.
• It means that Galapagos coexists with rather than replacing other tools. You don't have to deal with my bad version of something another tool does much better at.
• If you want a complete terrarium, it's always been possible by writing for an existing platform, old or new, and then emulating enough to run the binary. It's in the build process that things always get hairy: the exact steps to turn source code into something that runs are often extremely ecosystem-dependent. The more you can rely on a simple build strategy leveraging common formats, the better.

Having said that, what's the timeline look like on this new take on Galapagos? Simply put, it's become more of an open-ended background project, after spending a long time being the top thing on my mind. I am taking a break to work on pinball simulation but want to return to Galapagos periodically, first to solidify the binary format, and then to work on specific tools. Maybe things will start to appear by the end of the year at this rate.

And as I do that I'll probably get around to updating this page, so, stay tuned.