[Blog] - Debugging the "Doubling Money Bots" Breaking 2009scape Tests

Various development-related discussions
Post Reply
User avatar
Red Bracket
Retired Staff
Posts: 317
Joined: Thu Aug 11, 2022 7:18 am
Location: Varrock West Bank
Has thanked: 282 times
Been thanked: 74 times
Contact:

[Blog] - Debugging the "Doubling Money Bots" Breaking 2009scape Tests

Post by Red Bracket »

Thought I would make a casual blog post about how I debugged the failing tests in my open MR on 2009scape's server.

It's worth noting, yes I'm retired from 2009scape's development, no I'm not coming back. I'm just dropping by to create a fun piece of immersion that has no benefit to the game, since that's my idea of having fun on 2009scape.

That said, things have changed a lot since I was running the project here. Testing used to be the process of "yeeting it onto the live server" and "seeing if players stopped complaining".

Image

After resolving some disagreements on the bots, including reluctant changes, the MR was meant to be tested last week, only to find out 2009scape is at the quality level it has automated testing now. Looks like the server is better than when I ran it from my grandparents house with 5Mbps upload back in 2019.

Looking at the failing tests, and comparing them to the old tests that failed (since on my computer, tests are failing even on the master branch), I was immediately confused since many of the tests that were failing actually worked when I ran the server locally:

Image

Image

So, it looks like it was time to debug. Unfortunately, I'm mostly burnt out from my real world job, so digging through code isn't something I want to do. Instead, I invite you to meet my friend: git bisect.

git bisect is simple - You choose a commit where the tests are failing (i.e the tip of my MR), and type git bisect bad. Then, you checkout the commit where the tests are working, and type git bisect bad. git will automatically give you the middle commit, and let you decide if it's working there or not, and you can mark that commit as either "good" or "bad".
[before MR] --- [c1] --- [c2] --- [c3] --- [c4] --- [c5] --- [c6] --- [c7] --- [c8] --- [c9] --- [c10] --- [c11] --- [c12]
bisect good                                               good or bad?                                             bisect bad
If the current commit is good, the new commit git will give you is further to the right. If the commit is bad, it'll give you a commit further left. The idea is you can very quickly find which commit broke the tests.

Since the tests take a minute to run, I'm not going to watch that. So instead of doing a simple rm -rf Server/target ; ./run -t, I added a notify-send to let me know when the tests were done.

Image

Oh look, the tests failed. That means we run git bisect bad:

Image

After running this a few times, git finally told me where I went wrong:

Image
It's worth noting git can actually automate this if you tell it what to look for in a failing test

Reading the test results closely also tells us what went wrong:

Code: Select all

Caused by: java.lang.NullPointerException: data.getBoolean("world.e…doubling_money_scammers") must not be null
	at rs09.game.system.config.ServerConfigParser.parseGameSettings(ServerConfigParser.kt:78)
	at rs09.game.system.config.ServerConfigParser.parseFromFile(ServerConfigParser.kt:41)
	at rs09.game.system.config.ServerConfigParser.parse(ServerConfigParser.kt:30)
	at TestUtils.preTestSetup(TestUtils.kt:58)
	at HouseManagerTests.<clinit>(HouseManagerTests.kt:13)
	... 63 more
It looks like the config we added to enable doubling money bots is missing from the test config (Server/src/test/resources/test.conf). What a silly mistake! Well, it's time to let Ryan know the MR is ready again.
Post Reply