Inven Global

“Actual user testing does not make emotional choices or decisions for the development team. It simply conveys objective facts about what users find difficult. Based on that feedback, the development team must decide the direction of the solution themselves and fix the issues.”

These were the words of Shift Up Chief Technology Officer (CTO) Dong-ki Lee during the fourth session of the global track at the Console Game Developers Conference 2026, held on the 9th at SKY31 Convention in Lotte World Tower, Seoul. Taking the stage under the topic 'Polishing Time: A Record of Late-Stage Development for Stellar Blade’s Refinement,' he shared how the team identified and resolved a total of 549 issues over roughly two years with 68 testers leading up to release.

Throughout his presentation, CTO Lee emphasized the development team's attitude toward data over the data collection itself. While user research reveals where players get stuck and what feels inconvenient through stark numbers and facts, how to fix those issues remains entirely up to the development team. Recalling the intensive iteration loops that drove boss fight satisfaction from a low 4.85 points in initial tests to a soaring 8.6 points, he outlined practical decision-making principles that overcame the limitations of single-player console games.

Lee opened the talk by addressing a long-held misconception among developers regarding the timing of user testing. Mobile or online games can undergo large-scale closed beta tests (CBT) or open beta tests (OBT) and continue refining the game based on live feedback. In contrast, single-player console games—where players learn the entire story and progression upon completing the game once—are structurally unable to conduct repeated testing with the same players over an extended period.

“Because testing opportunities are hard to come by, developers tend to push test dates back, hoping players can experience the game fully from start to finish. However, as long as the development team clearly understands the core value of their game and can ask players relevant questions, conducting the first test as soon as the core gameplay loop works will yield sufficiently meaningful results.”

The core values Shift Up strove to preserve in 'Stellar Blade' were striking visuals, immersive combat, and adventure within a post-apocalyptic world. In particular, combat aimed for a fusion of two styles—neither a typical Soulslike nor pure combo action like 'Devil May Cry' or 'Bayonetta.' It needed to combine the tension of parrying and dodging enemy attacks with the stylish execution of diverse combo actions.

To verify whether this vision functioned in practice, Shift Up conducted its first test in March 2022, about two years before official release. At the time, the build contained only a basic framework: core combat mechanics, mid-game level play in the Levoire region, three bosses, and basic combat progression elements. CTO Lee collaborated with Sony's User Research team and Keywords' Player Research team to seek answers to three primary questions:

First, where do players experience inconvenience or get stuck while playing our game? Second, does the perceived difficulty match what the development team intended? Third, are players truly having fun with the game?

CTO Lee stressed, “I believe the core of leveraging user research lies in deciding whom to ask, when to ask, what to ask, and how the development team responds to those answers.” Shift Up combined four research methods timed appropriately: Usability Evaluation (where expert researchers pinpoint build issues), Usability Testing (observing actual player experience), Accessibility Evaluation (checking for physical or visual difficulties), and Appeal Testing (verifying difficulty and fun). Playtest volume grew from roughly 15 hours early on to 28–32 hours in the final stages of development.

Starting tests early provided the runway needed to make drastic changes to the game's foundation. A key decision stemming from the early 2022 tests was the complete removal of the 'death penalty.' The game originally featured a Soulslike system where accumulated skill points were lost upon death.

Interestingly, testers never actually requested the removal of the death penalty itself. CTO Lee explained, “The feedback from user testing didn't point out that the penalty system itself was flawed. In fact, the actual feedback noted that items meant to offset the death penalty suffered from poor usability.”

However, rather than fixing the item UI, the development team opted to drop the entire system. Internal debates had already questioned whether the penalty fit Stellar Blade's action design, and when combined with item usability issues, the team boldly removed it two years prior to launch. It was a decision reached by interpreting the core issue beyond the symptoms reported by players.

A similar process reshaped the acquisition rules for combat resources, the Beta Gauge and Burst Gauge. During tests in May 2022, players remarked, “Filling the gauge feels great because the payoff is high, but I'm not sure exactly how I'm filling it.” While internal rules governed gauge buildup based on attacks and defense, the connection between actions and rewards was not clear enough for players to grasp intuitively.

CTO Lee determined that since the gratification of the reward worked, only the acquisition conditions needed clear separation. The team split the acquisition pathways: successful attacks, blocks, and parries built the Beta Gauge, while successful dodge-based actions filled the Burst Gauge. Complemented by improved tutorials and UI, the structure was corrected to let players intentionally generate resources according to their playstyle.

The area of accessibility—which can be somewhat unfamiliar to domestic Korean studios—was also refined by actively incorporating data from external research teams. Based on 85 cases and guidelines provided by Sony's research team, Shift Up implemented 28 options in total, including color vision filters, high-contrast settings for key objects, and shape-based visual cues alongside color. Their effectiveness was verified through re-testing.

In the latter half of the talk, CTO Lee illustrated how the team revamped features based on failure data, citing concrete examples. The most dramatic transformation occurred with the final boss, the Elder Native. During its first test in May 2023, satisfaction for this boss fight registered a low 4.85 points.

“Analyzing the root cause revealed that the combat mechanics themselves weren't unfun. Instead, there was a structural flaw where players suffered from a severe lack of ammo needed to counter the boss's instant-death gimmick, preventing them from clearing the fight altogether regardless of the boss's remaining health.”

To stop the Elder Native's instant-death move, players must shoot and destroy airborne cores within a time limit. The issue was that ammo spent in previous attempts was not restored upon dying and retrying. For example, if a player started with 30 rounds, used 25, and died, the next attempt began with only 5 rounds remaining. Overcoming the mechanic required 10 rounds, yet knocking the boss into a staggered state yielded only around 8 rounds. Even players who fully understood the mechanics inevitably ran out of resources and died.

The team added an ammo supply machine at the rest spot right before the boss fight and significantly increased ammo drops when staggering the boss during combat. Identical fixes were applied to other boss encounters sharing the instant-death mechanic. Once unfair bottlenecks were removed, satisfaction for the fight surged to 8.6 points in the final October 2023 test, comfortably exceeding the internal threshold of 8.0 points. The overall combat average score also reached 8.25 points, instilling confidence ahead of release.

Data also helped bridge perceived difficulty gaps across target audiences. Results from the October 2022 test showed that while Soulslike fans familiar with FromSoftware games expressed satisfaction, general action RPG players found the game excessively difficult.

To address this, Shift Up created 'Story Mode,' going beyond simple stat reductions to enemy health and attack power. They introduced an 'Action Assist' feature that visually prompts response timings for specific attacks and assists player input, easing the burden of execution. Furthermore, players could lower the difficulty to Story Mode whenever stuck and return to Normal Mode at any time, mitigating the psychological stress associated with changing difficulty.

The tug-of-war between developer philosophy and player data was most evident during VFX and UI polishing. When refining the 'Blink' and 'Repulse' mechanics used to counter special boss attacks, the team established a firm principle.

“Our principle was to give clear response cues to players without displaying direct UI icons on screen. We wanted players to read attack signals naturally within the game's visuals without breaking visual immersion.”

However, early test feedback flooded in stating players had no idea when to use Blink or Repulse. Rather than compromising with an easy UI solution, the team chose to relentlessly overhaul the VFX. Initially, a flash was added only to the boss's blade; when that proved hard to notice, an effect was added above the boss's head. When feedback indicated the signal was still too faint, the timing was adjusted. When overlapping signals caused confusion, the exposure time and sequence of each cue were meticulously separated. After four to five major VFX overhauls, the team achieved both visual immersion and clarity.

Conversely, there were exceptions where developers boldly broke their prized 'no-UI rule.' The instant-death mechanic in the Raven Beast boss fight was one such case.

“While reviewing playtest failure data, we saw a log where a tester died 80 times to a boss's instant-death mechanic without figuring out the solution, ultimately asking the research facilitator directly how to beat it. They passed only after hearing the explanation. It was a moment where data honestly demonstrated that presentation obvious to developers who spend all day creating the game could be completely unreadable to a first-time player.”

Faced with progress-blocking frustration, player readability took precedence over presentation principles. The team decided to display explicit UI signals—a red circle and an exclamation mark—specifically for that instant-death mechanic. It was a flexible judgment call, balancing artistic presentation against player convenience depending on data severity.

CTO Lee shared not only success stories but also honest regrets over issues that couldn't be fully resolved due to late testing. A prime example was the platforming section in the Altess Levoire dungeon. Players had to step on platforms matching shapes on the floor, but complaints erupted over precise control difficulties caused by 'movement inertia,' which made the character slide about half a step when stopping or jumping.

The problem was that testing for this section occurred only in May 2023. With release approaching, overhauling the character movement and animation systems across the entire game carried too high a risk. Ultimately, the team opted for temporary workarounds: enlarging platforms, adding ledges, and exceptionally removing movement inertia only within that specific dungeon area. Consequently, player feedback regarding clumsy platforming controls persisted in outdoor field areas.

“Looking back while preparing this talk, many of the issues troubling us late in development were actually mentioned as early as our first usability evaluation in March 2022. Catching issues early is the only way to fix fundamental structures rather than applying band-aids, and to secure time to re-verify the fixes.”

He also offered practical advice to domestic Korean studios developing physical console games: back-calculate the launch timeline. Considering physical lead times for disc mastering, manufacturing, global distribution, and marketing material production, actual development completion for boxed titles must occur at least 4 to 6 months before launch. By the time a demo build releases a month or two-to-three weeks prior to launch, internal work on the 'Day-1 build' must already be largely complete. Back-calculating this timeline to schedule user research and polishing earlier is essential to prevent schedule blowups.

Closing his session, CTO Lee reminded developers—whose senses can dull from playing the same game every day—of the vital importance of external perspectives.

“Developers who spend all day working on their game often become desensitized, feeling the game is too easy, or conversely, feeling anxious about whether it's good enough. At those moments, I hope you question your own senses and trust the objective data provided by user research. Our ability to polish Stellar Blade over two years leading up to the Switch version release, resolving 549 issues along the way, was possible because we rapidly iterated through test-fix-retest loops without stopping and truly listened to player voices.”