Production Period: 2022-2024
Platform: Browser
Game Engine: Radiance (proprietary)
Roles: QA/Test Manager & User Researcher
Responsibilities (QA):
- Ember Sword’s end-to-end quality
- Requirement analysis from design documents and meetings
- Creation of a test repository, writing more than 1000 test cases in a callable system (Xray/Jira)
- Functional testing of both game and engine in builds and editor
- Release and environment management
- Managing both an internal team of 12 student testers, and external FQA teams during times of high through-put
Ember Sword was a browser based classless MMORPG based in the world of Thanabus*. Through the ground breaking, in-house technology, players could load in within 15 seconds and explore the vast continents of Thanabus.
*due to a lack of funding, the game was taken offline after an initially successful early access launch in December 2024.
What I Learned
It is impossible to list everything I learned in these nearly three years. To list a few thing: I participated in mid-scale game development processes from design documentation, over analysis, to implementation; gained a deep knowledge of Jira, JQL, and its automation functions; built on and improved my ability to talk to artists, designers, and programmers in their own languages.
Within QA, I built our whole quality processes and measures from ground up. I aimed to enable teams as much as possible to own quality themselves – this saves spending more on QA resources and prevents issues early/before they arise. For example, I held workshops about QA best practices, bug reporting, bug life cycles, and applicable ISTQB standards. Furthermore, I built a complete test repository with reusable, collable test cases in Xray. I constantly aimed for improvement and made it a ritual to progress learning paths in the Jira and Xray academies, and obtained an ISTQB certification.
However, the biggest values were the personal things I learned at Bright Star Studios. How to come to a consensus in bigger teams – and if not that, then how to move on as a team anyway. What information every level of a company needs to make the best decisions they can. That my biggest contribution to any team is my ability to speak up when I think something is wrong.
Executing tests from exploration, over confirmation, to regression. We ran change and confirmation tests on our editor and game at least on a weekly basis and tested developing feature as required on-goingly.
Complemeting manual testing with automation. To free up resources in our small team, I taught myself Playwright (TypeScript) to automate repetitive smoke tests which ran after every deployment.
Creating and maintaining a feature based test repository in Xray. Calling tests was a corner stone of our repository, enabling quick expansion of features and adjustments to changes via datasets.
Liaising with production and leads regarding issues, delays, and risks. Aside from participating in standups and all agile rituals, I owned our bug-triage meetings, where I highlighted critical issues and we evaluated bugs at hand of Cost to Fix, Impact, and Contagion [ref].
Managing twelve student testers and two external FQA vendors. To increase capacities I got a bit inventive and started a work-experience collaboration with Sunderland University and College. The students were primarily involved in exploratory and regression testing.
Supporting functional testing with quantitative data. Amongst other things (like tracking levels, items, etc.) I generated heatmaps of player locations and overlayed them with our map to identify world, navigation, or respawn issues.
Other tasks included designing, executing, and reporting 5 user research studies; branch management in git; evaluating, managing, and deploying new builds to live (and testing) environments; serving as a Jira admin for up to 80 users; and minor bug fixes.
Responsibilities:
(Release Management)
- Evaluating build stability before release
- Deploying builds via Jenkins
- Ensuring that the correct builds are on all internal (test) and public environments
- Liaising with all teams what to include in releases, including evaluation of commit risks
Responsibilities:
(Production)
- Admin for two Jira project with up to 50 users, managing issue types, labels, groups, dashboards, etc.
- Owning sprint start and end, ensuring all stakeholders are ready and informed
- Leading standups and retrospectives when no producer was available
Responsibilities:
(User Research)
- Designing user studies
- Pooling and segmenting playtesters
- Running tests using observations, interviews, and focus groups
- Analysis of data via a “grounded theory light” approach, utilizing mindmapping tools to speed up the process
- Reporting results to game design and higher level management both in presentation and writing
Credits:
- too many to mention here