4 min read

New Year, New App Version

New Year, New App Version

Well hello. And happy New Year!

It’s been a while. Over four months. So what’s been happening? Let me catch you up on everything over here at Curious Duck Digital Laboratory.

When the last newsletter came out, I had created a prototype web interface for the underlying simulation engine. It was a hacked together mess, but it was good enough for demos. (You may even have played with it.)

Since then, I went on a journey to create a “real” app, something with a more game-like interface that was carefully test-driven rather than just thrown together. That took much longer than I expected. Just sorting out the details of the tech stack and test frameworks took weeks. Along the way, I fell down the rather deep rabbit hole of modern JavaScript, particularly for Rails apps. (I tweeted a long and slightly ranty thread about what I learned.) The investment was worth it though. When I came out the other side of the rabbit hole, I had not just a reasonably well-factored app but also a setup for test-driving the JavaScript that powers the interactivity in the GUI.

Better yet, there’s a preview you can play with! Note that this is my staging site and is likely to be volatile. Also, don’t bother to create an account since I will be blowing away all the data regularly. You don’t need one; you can play without a login.

The app is still far from done, but at least you can see the direction it’s headed. I have been sending the link around so a few folks have already played with it. If you want to provide feedback or talk about the simulation, there’s a Discord. It’s pretty quiet over there at the moment, but drop me a note if you would like an invitation.

When you run through the scenario, the character Nora gives you feedback after each level. Initially Nora’s dialog was the same no matter what choices you made. Matt Wynne rightly called me out on that, noting that “It’s pretty hard to get Nora mad at me.” So I decided to make her dialog more realistic.

Turns out realistic dialog is way harder than it sounds. I had to develop a way to manage the ever increasing number of possible dialog phrases and choose which phrases the character would say based on the player’s actions. I found a GDC talk by Elan Ruskin on AI-driven Dynamic Dialog through Fuzzy Pattern Matching especially inspiring.

I still don’t have her tuned quite right, so you may find her feedback unhelpful or sometimes even unfair. James Thomas found a case where his estimate was accurate within a day and Nora told him she didn't think he was trying to estimate accurately. Nora went from not expressing disappointment no matter what the player did to holding the player to impossible standards. OOPS!

Despite the flaws, the new dialog system is a big improvement over the first version. Many thanks to both Matt and James for their incredibly helpful feedback.

Finally, I decided to rework the way that bugs spawned. That turned out to be more of an adventure than I imagined.

Let me back up and explain a little more about bug spawning.

In the first hacky prototype, bugs were delivered every Monday morning at 9am. Same number of bugs, same day every week, same time. All surprises happened on a neatly pre-defined schedule. Easy to code, but not at all realistic. The resulting burndown chart looked much too regular.

The first iteration of the real app had a much improved bug spawning algorithm. Instead of scheduling bugs to show up at 9am on Mondays, I defined a number of bugs to occur each week. The simulation then randomly sprinkled those bugs throughout the week. When the simulation launched it still knew exactly how many bugs there would be and when they would show up, but the burndown chart looked much more irregular, so it felt more realistic.

I affectionately refer to both of these bug spawning strategies as “All Surprises Will Be Scheduled in Advance.” Although the new app felt more realistic than the hacky prototype, neither approach allowed the simulation to adapt to changing conditions during game play. That’s a problem. Ultimately the whole point of the simulation is that the player can affect the outcome. You make choices and experience consequences.

Even in a scenario as simple as the Estimate challenge, if you choose an overly optimistic estimate, the developers should notice that they’re behind. When they inevitably rush to make the date, they produce lower quality work. Lower quality work has more bugs. Scheduling all bug spawning in advance means the simulation cannot adjust the rate at which bugs spawn.

So I tried again. In a newer version (not yet released on the preview site), bugs can occur whenever a developer delivers work. The probability of a bug occurring is controlled by the reliability of the work. The player can make choices that raise or lower reliability. So deadline pressure makes developers rush, they deliver less reliable work, and more bugs spawn. The more bugs, the more work there is, and the more pressure on the developers. More pressure, more rushing. More rushing, lower reliability. The cycle continues.

If unchecked, it’s a dynamic that can spiral out of control. But my goal was not to create an unwinnable scenario; it was to invite players to provide the business stakeholders accurate information without overcommitting their team. So if a player invests in quality by slowing the pace of feature delivery, reliability goes up and fewer bugs spawn.

It was a lovely theory, and it mapped neatly to the real world where time pressure leads to teams cutting corners which in turn leads to production incidents and outages.

Alas, things did not go as I expected.

However I’ll leave that story, and the lesson I learned from it, for the next newsletter. Now that the holidays are behind us I expect to be back to a more frequent publishing schedule.

In the meantime, I wish you the happiest possible 2022!

Stay Curious,

Elisabeth