Microblog

I am obsessed with Tomorrow Corporation Tech Demo for a while now and this weekend I had two cool insights that made me think it was possible to prototype this. The brief overview of features is this I would say:

• hot loading assets
• hot loading code on keystroke
• breakpoint, step forward AND backwards (aka time travel debugging)
• profiling with fast instrumentation
• find slow frames and functions and jump there (back in time)
• save timestamps into objects and jump to those stored timestamps
• seek backwards to any point during the demo
• portable sessions, saved every time

Some of the features are not that fancy or are simply a result of having a very fast custom compiler (hot-loading, compile in instrumentation for profiling). The real central piece of this engine is, I think a very efficient replay system. If you have a replay system that has little overhead and stores efficient sessions, you can record every time by default. Jumping to slow frames, saving timestamps and seeking backwards is essentially just a little feature if you have the replay system. This weekend I realized two important things. First, you don’t need to save every state of the program after every line of script code to be able to jump everywhere. Because the game is likely running at interactive framerates, you really just have to snapshot the state of the game every frame and in (probably) less than 16ms you can jump to every line of code, because it takes less than 16ms to calculate the whole frame. Second, stepping backwards is then also just a consequence of that, because you can simply restore the start of the frame and then simulate again, but stop one line earlier than last time. Usually when implementing time travel debugging it’s difficult to know when to do full snapshots, but in a game you have frames. So this is essentially easy mode! So what you really need is an amazing replay system.

You also need a scripting language that lets you debug it and control until where you want to execute. I thought I should try tinycc for this! And it was a massive fucking pain without any results. I time-boxed this experiment to this weekend and 80% of the time I was messing with tinycc. The most recent release on the web page, does not build on modern GCCs/libc anymore (no __malloc_hook). The current version on the dev branch has a different API and somehow I struggled a lot with making it find the runtime library, which imho should just be statically linked into libtcc. Then I wasted a solid 5 hours trying to get debug symbols out of it, which would be emitted if you built an ELF file, but not if you JIT compile to memory. I do not like the code. Anyway, I gave up. I looked into a ton of other things as well. QBE (can’t embed), cproc (need QBE), chibicc, lacc, fcc and pcc. I am flabberghasted that there is no little language you can embed from C(++) that JIT compiles and provides debug info. At least I can’t find it. And therefore this experiment is indefinitely postponed. MIR might be something worth trying once I actually bite the bullet and build my own scripting language for this, but that is neither today nor soon, because that would not be a weekend project.

Yesterday I attended a gamedev meetup and it got me really excited about making games again - not just engines and libraries. I decided I should take a break from the engine dev and make a game instead.. but with what engine? I hate them all. The solution I came up with was to make the game first and then go from the MIDDLE OUT (not the best fit, but I want to call it that now).

I’ll start with an executable that runs a Lua script, hardcode the window title, resolution - everything I can. Just add womf.newFrame() that returns false as soon as the window gets closed and clears the screen. Load a single shader from s hard-coded path and add womf.draw("modelfile.obj", {uniforms, ...}). The goal will be to build as little as possible of what could be called an engine and later on replace parts with nicer engine components.

I have been messing around with eBPF at work and I have become a huge fan of it. When I talked about this with my colleague at work, he mentioned a talk he gave at MariaDB ServerFest Berlin about eBPF very recently. I have seen a few resources on eBPF before but never quite grasped what kind of superpower it can be, before I used it myself. If I had seen Max’s real-life case study I would have understood much earlier how useful eBPF can be. So if you are developing or debugging on Linux and haven’t hopped on the hype train, this might sell you on it.

As a serial “new engine” developer I also have very strong opinions about what kind of game engine I would like to use or, more accurately, can tolerate using. Racenis has some ideas that I very much agree with. Especially the first three sections. I have a couple design docs scattered about that outline what I actually want from a game engine and I think I’ll turn it into a living blog post soon.

I think building things is bridging the gap between grand ideas of things you wish existed and reality. So a builder is constantly thinking of cool stuff they would love to see become real. Or rather someone that thinks like that tends to become a builder of some sort. And even though you spend so much time thinking about a cooler future, sometimes you get blown away by something someone else has done because you didn’t even consider that someone could think this big. The thing itself might be difficult, but not much more than a ton of other difficult things people do all the time and nonetheless people need to see the thing before they realize that it’s possible to do something like that. This video really blew the door wide open in terms of what I think engines and game tooling might be able to do. If I ever have a couple of years of free time and enough money I’d build exactly what Tomorrow Corporation showed in this video myself.

This is quite long, but very good. I think it’s part my “N+1 arc” mentioned in the post before. It introduces a really cool concept of tying the abstract concept of a lifetime to something concrete, which is allocation arenas.

Though he is very smart, I find Casey has a narrow view of software, because he seems unaware that many (most) software is not compute-bound, like games and adjacent software often are and he also shows a lack of understanding of the realities of larger development teams. He then tends to attribute apparent flaws in software today to people simply being bad or dumb, when in fact their practices are a result of a development culture that has good reason to exist, but not necessarily in (small) games. I agree it’s usually a mistake to misuse those common practices in compute-driven applications like games (e.g. TDD, heavy OOP, etc.), but people have good reason to do it this way, he just doesn’t really understand those reasons. Anyways my point in adding this disclaimer is that I understand perfectly if you do not want to watch this video or have to try not rolling your eyes as has as you can, if you do watch it, but it is worth seeing. I think this video and some other videos of his sort of started a new era for me. When I started programming I did it the “non-OOP” way, but in the most horrible spaghetti chaos kind of way. Later on I did learn about encapsulation, layers of abstraction and the value in making software robust, but I carried this into my personal, small (one-man) projects and it kind of ruined them. Everything was full of abstractions and allocations and crazy ownership graphs and just a lot of code that doesn’t actually do anything in the sense of modifying memory in a useful way. In large teams or old code bases the extra abstractions keep different programmers aligned, newbies in check and avoid subtle regressions, but a lot of this is likely unnecessary or counterproductive in some kinds of projects (compute-bound, small teams, etc.) or even certain pieces of code. Just when I started watching some videos of Casey I have been reading a lot of code by people that do have the “N+1 thinking” and I knew it was much better than mine, but I didn’t really understand why. The “N+1 thinking” is about structuring your code never around single objects, but always around collections of objects. Smart-Pointers, RAII, ZII? Becoming an N+2 programmer describes it in more detail, but the video is relatively long and somewhat noisy. Overall Casey opened my eyes on what exactly I should change and what I need to learn. I think I am still a little too OOP-brained, but I am currently on my “N+1 arc” because of him.

I have had “Website: Link Page!” on my todo for possibly years and this blog post inspired me to build this site. I post very sparingly on this website, because I don’t like thinking I have anything to say that anyone else has to hear, though I do feel like that sometimes. This also kept me from creating such a link page. Simon Willison’s post changed my mind about it - especially this paragraph:

The point of that article was to emphasize that blogging doesn’t have to be about unique insights. The value is in writing frequently and having something to show for it over time—worthwhile even if you don’t attract much of an audience (or any audience at all).

I really don’t really need a reason for someone to listen, when I am just screaming into the void anyways. Right around this time I also read What is microblogging? and I thought I’d go for something more general like that instead.