We’ve been talking a lot lately about the new features and engine improvements coming to ROBLOX. But behind the development is a relatively unseen – and extensive – process that ensures the changes don’t break the game for players: Testing.

ROBLOX undergoes constant, rigorous, automated tests. Every time a developer “checks in” a new line of code, our testing system creates a trial version of ROBLOX and runs it through a battery of automated tests. This not only ensures the new code works, but also that it didn’t inadvertently introduce new bugs somewhere else.

Testing: All day, all night

We run full test suites on ROBLOX dozens of times each day. Each test suite consists of hundreds of tests – and this number is growing daily – for multiple platforms. Do the math and that means we run tens of thousands of tests per day, with each test covering hundreds of thousands of lines of code.

Each colored block represents activity by ROBLOX’s various testing systems.

As you can see, our PC and Mac testing machines are active throughout the day. 

Testing: Any type, anywhere

Our testing framework consists of three varieties of tests: manual, unit and automated. Considering the size of ROBLOX, each one is necessary for us to release regular, quality updates.

Manual tests

Manual testing, in the most basic sense, consists of jumping into a ROBLOX game and seeing whether a specific behavior works. ROBLOX is too complex for us to test it in this way, so we’ve built up a library of testing scripts, written with very specific steps, that we send to a dedicated testing house. Using their extensive resources, they run the tests – usually overnight – and return detailed diagnostics the next day.

For example, we wrote a script that tests whether rockets function as they should. If a code change causes them to, say, fall to the ground, the test fails and we know there’s a regression that needs fixing. We have a lot of these scripts, covering a spectrum of ROBLOX behaviors.

Unit tests

Unit testing refers to ROBLOX running tests on itself. If ROBLOX were a car, its unit tests would be checking its own oil level, transmission temperature, etc., and reporting problems – like being low on oil. We have a bank of machines that almost always have ROBLOX running tests on itself because the tests fire up each time a developer checks in a line of code.

To write unit tests, we use the boost::test framework. Here’s an example of a test that makes sure our raycasting code works:

Automated tests

Automated testing determines whether a specific behavior happens as it should. The tests actually open instances of ROBLOX and run Lua scripts – the same language ROBLOX players use to create games – to determine whether the behavior happened as desired. For instance, an automated test might open ROBLOX and run for 30 seconds to ensure a ball continues bouncing until 30 seconds have passed (as opposed to bouncing away, falling through the floor or some other buggy behavior).

Testing: You can, too

To make writing automated tests easy, we extended the boost::test framework into our Lua environment. If you’re an advanced ROBLOX Studio user, you might have noticed a new TestService feature. This lets you write sophisticated tests right inside a game. All scripts inside the TestService are executed when a test is run, but not during normal gameplay.

Here’s a script we run to make sure the BindableEvent feature is working:

Let’s say there was a bug in BindableEvent. In that case, when we run the test we might see the following output:

What good does all this testing do you? Continuously testing ROBLOX is allowing us develop new functionality more quickly, and with less risk of causing regressions in the process. This ultimately means a better gaming experience and more consistent releases for our users.