Welcome to the Dolphin Progress Report for December 2020 and January 2021! Things ended up running a little behind for this report due to some technical details that we needed to hammer out for a few of these entries. We on the blog team are familiar with the emulator, however there are a lot of technical details that are simply beyond our expertise. Going from things like the AArch64 JIT to GUI changes to IOS updates to game patches that go into low-level hardware behavior is enough to make anyone's head spin! More often than not, we rely on core developers and the authors of a specific change to help us understand what a pull request does so that we can express its purpose accurately here on the blog.
With Progress Reports coming at a mostly bimonthly schedule at this point, this means that sometimes authors have moved onto different things or aren't available to talk. As a blog about emulation, getting these details correct about the various changes and how the emulator works is one of our highest priorities. So, with that out of the way, we hope you enjoy this belated Dolphin Progress Report!
5.0-13152 - Add Fallback Region Option by flagrama¶
Handling game regions is something that users shouldn't have to think about most of the time. After all, games have to report their region to the console, so the task should be simple for the emulator, right? It may come as a surprise that Dolphin has had a lot of complications with properly detecting game regions and even has fallbacks for when it cannot detect a region. For many years, Dolphin used the GameID of a title to detect the region. It's actually very simple! Look at The Legend of Zelda: The Wind Waker. Dolphin used to detect the NTSC, PAL and NTSC-J by the 4th letter in its GameID: GZLE01, GZLP01, and GZLJ01. While this system worked for many years, exceptions started popping up and causing issues. For instance, Michael Jackson: The Experience - Walmart Edition uses an annoying X in the GameID for NTSC and a comparable version uses Y in NTSC-J! As more and more exceptions piled up, we were forced to find a better solution.
Turns out that all disc releases for the GameCube and Wii games have a region code on the disc. Additionally, all Wii titles, including discs, channels, WiiWare, and Virtual Console include a region code in their TMD. By moving to a more accurate system, Dolphin's days of misidentifying regions is a thing of the past. That isn't to say it's impossible for complications to come up.
Some Wii Channels, such as the Mii Channel, don't have a unique region code. That's because certain Wii Channels are actually multi-regional! The same channel is designed to work on any region Wii, which means that it uses a multi-region code. In this case, Dolphin can't rely on a region code. There's also the case of homebrew, which doesn't have a set way of expressing a region. For these cases and any other exceptions, Dolphin had instituted a simple backup. It would just check for what region of the Wii Menu was installed to the currently selected NAND and use that. When there was no Wii Menu installed to the current Wii NAND, it would fallback to PAL for Wii titles and NTSC for GameCube titles. Why? The answer was right in the code!
// TODO: Right now GC homebrew boots in NTSC and Wii homebrew in PAL. // This is intentional so that Wii homebrew can boot in both 50Hz and 60Hz, without forcing all GC homebrew to 50Hz. // In the future, it probably makes sense to add a Region setting for homebrew somewhere in the emulator config.
We prominently mentioned the Legend of Zelda: Ocarina of Time randomizers when talking about Dolphin's JIT behaviors and making the games more playable. While their GameID seems to imply it's for NTSC consoles, they actually use the same region free flag that multi-region channels use. This is likely to make things easier when running the WADs on real consoles. This would trigger Dolphin's fallback code, sometimes causing the randomizers to run at the incorrect framerate when no Wii Menu was installed or the PAL Wii Menu was installed with PAL60 disabled.
In order to reduce confusion and have more sane control over cases like these, flagrama has added a region fallback option to the configuration menu. This allows users to customize the region that Dolphin chooses when no region can be detected for a particular game, homebrew, or channel. We've also changed the default from PAL to NTSC in order to simplify things for a majority of our users.
5.0-13386 - Use Storage Access Framework Scoped Storage for the Gamelist on Android by JosJuice¶
This change brings support for the Storage Access Framework and Scoped Storage to Dolphin's gamelist. The idea behind Storage Access Framework and Scoped Storage is solid - basically, the user has to grant an application access to the user's files. The Storage Access Framework has existed since Android 5, however it's been optional and we've chosen not to use it due to its various shortcomings. Unfortunately, come November, Dolphin will be forced to use the Storage Access Framework and that means that there are going to be some changes.
If things were as they should be, we'd be silently moving over to the Storage Access Framework API and users wouldn't even be reading about this. Unfortunately, it isn't that simple. On top of being hard to implement, especially for a C++ application like Dolphin, the Storage Access Framework API is extremely slow. Its shortcomings aren't just apparent to us, either. Slow performance in file access is extremely frustrating, but that is at least limited to first opening a file. Gamelist loading times have been increased by more than tenfold in our testing, but this does not actually affect emulation performance.
Eventually we will have to bring everything in Dolphin Android over to Storage Access Framework API, and there are situations where its limitations matter. Some features look like they're going to have to be dropped altogether or modified to be a part of Dolphin's core files. This includes things like customizing paths for the Wii NAND. This is unfortunate because many of our users like to take advantage of multiple NANDs due to the Wii's strict space limitations that emulation cannot easily bypass. Currently, we're planning on using a single preset Wii NAND directory to bypass the need for Scoped Storage. While this means no performance issues, it does mean Android users will be locked to using a single Wii NAND. It also will have to use a device's internal storage. No more taking advantage of that pesky external SD card! ...not that many phones come with those today anyway.
For our Android TV users, the news is even better. While the file picker works on some devices, the folder picker that most apps do not use is completely busted across everything. Dolphin uses the folder picker in order to select game directories. Because it's completely broken, Android TV devices running Android 11 cannot use Dolphin's gamelist functionality whatsoever. For older versions of Android on Android TV devices, Dolphin will continue to use the old version of the folder picker. For devices like the Shield TV that still run a maximum of Android 9, Dolphin's gamelist will continue to work normally unless they update to Android 11. At that point, it would be out of our hands.
Since Dolphin is currently targeting the Android 10 SDK, we have been able to slowly implement support for Storage Access Framework without being forced to use it in problematic areas. That loophole disappears come November 2021, meaning that things will change for Android 11 users at that time. However, in order to continue to updating Dolphin on Android, we are forced to conform to the Storage Access Framework API. While these losses are frustrating, we're thankful that the core emulation experience won't need to be compromised. On the bright side, using Storage Access Framework bypasses a bug affecting devices running Android 11 that was causing the gamelist not to work for directories on the SD card.
Edit: When this article was originally posted, the only Android 11 devices reporting the SD card issue were Samsung devices. As such, we surmised that the bug was with Samsung devices. As more devices have released or upgraded to Android 11 since then, we now know that it was an issue with Android 11 in general.
5.0-13440 - Make Wii Save Import Behave More like SD Import by AdmiralCurtiss and 5.0-13562 - Android: Add Wii Save Import Functionality by JosJuice¶
Dolphin's Wii Save Import feature saw some nice improvements this month. AdmiralCurtiss has simplified Dolphin's Wii Save import to work more like it does on the Wii when importing from SD card. This should remove some of the seemingly random failures when importing Wii saves. Do note that before you import a Wii save file, you still must have run the game at least once. This is just due to how the Wii filesystem works.
Adding to these improvements, JosJuice has added the ability to import Wii save files into the Android GUI! This means it should pretty easy to transfer Wii saves over to your mobile device to play on the go. Exporting them back to PC is another story, but that'll be added at a later date.
JIT Improvements by JosJuice, Sintendo, MerryMage, and smurf3tte¶
We've had so many JIT improvements over the past two months that it's actually crazy. While most of them are very small, we feel it's important to highlight everyone's effort and showcase a few fixes that we thought stuck out.
JosJuice has been working feverishly on the AArch64 JIT used on ARM devices, such as most phones and tablets. Most of these changes are incredibly small optimizations and additions that slightly improve performance. There are some bigger fixes on the way still, but a few of them have already managed to land. 5.0-13309 adds missing support for updating the Performance Monitor to Interpreters and the AArch64 JIT. Though JosJuice was merely trying to make performance testing easier, it turned out at least one game relies on the cycle count updating. As such, Harry Potter and the Prisoner of Azkaban now functions across all of Dolphin's CPU backends.
Sintendo and MerryMage have also submitted multiple small performance optimizations across both JITs. They may not be much alone, but they add up and give small boosts in various situations.
5.0-13553 by smurf3tte found a bug in an optimization for the
dcbst instructions that caused an address mismatch in games that rely on Dolphin's MMU Speedhack or full MMU emulation. This fixes a crash in Happy Feet for Nintendo GameCube. The Wii version doesn't rely on MMU features and thus wasn't affected by the bug.
5.0-13447 - IOS: WD and NCD Fixes by Leoetlino¶
Nintendo DS Connectivity is present in several games across the Wii library. Some games let you connect to a particular DS game in order to get bonus features while others simply let you connect the DS as a controller for another player. Regardless of what the DS is used for, Dolphin pretty much had no support nor testing for the IOS modules that handles broadcasting and connecting to the Nintendo DS.
Enter Ubisoft and the Wii version of Driver: San Francisco. Despite sharing the name of the 360, PS3, and PC counterparts, the Wii version is its own unique game especially crafted for Nintendo's console! This prequel to the original Playstation games features motion controlled vehicle combat and a draw distance that would make the original games blush. For all of its flaws, the game itself isn't really bad; it's a cute little throwback side game in the series with a neat comic book art style. But for emulator developers there's a huge red flag. Remember how we said it was a Ubisoft game? Much like how seeing Factor 5 at startup means that the game is going to push the hardware as hard as possible, Ubisoft developed games have their own reputation. Namely, their games usually do something that causes them to be a pain in the ass to emulate. Driver: San Francisco is no exception to this rule.
This game has been troublesome in Dolphin for quite some time. Driver: San Francisco is another Ubisoft game with MetaFortress anti-piracy. It seems like they actually learned from the days of The Smurfs: Dance Party as it no longer proudly proclaims "MetaFortress RESPONSE!" and is sneakier about crashing the game long after it has detected something amiss. Funny enough, Driver: San Francisco's MetaFortress proved wholly ineffective against Dolphin as it uses the exact same trigger as The Adventures of Tintin: The Game.
This version of MetaFortress uses the same tricks as the older versions with some new checks added on top. What actually tipped us off that it was anti-piracy was that The Adventures of Tintin's crash matched complaints online about the game crashing on console. Putting two and two together, we were able to determine that the cause of the crash was not a specific Dolphin bug but MetaFortress rearing its ugly head once again. As booting the game from the Wii Menu was a work-around, all we had to do was figure out what differences there were between a gamelist boot and a Wii Menu boot. Being that Dolphin had both options at the ready, it was fairly trivial to fix without ever digging into what MetaFortress was doing.
With MetaFortress fixed, we had hoped that Driver: San Francisco would start working. However, nothing changed at all and the game remained broken. It turned out that Dolphin was breaking before MetaFortress even had a chance to shut things down. This meant we had to go on yet another wonderful debugging experience featuring a Ubisoft game. Would it join the ranks of Red Steel, Far Cry: Vengeance, Your Shape, Just Dance, The Smurfs: Dance Party, and many more? Or would it prove to be even more stubborn, like Rayman Arena which is a Ubisoft GameCube title that is still broken for unknown reasons?
In a strange way, Driver: San Francisco was a breath of fresh air. It wasn't doing anything particularly wrong at a glance. In fact, it clearly was logging what was going on through the loading process and that it was getting stuck in an obscure IOS module. This was enough of a push to bring in Ubisoft game expert and IOS expert Leoetlino to look at the issue. What he was discovered was rather interesting. When you start up the main campaign, Driver: San Francisco uses the obscure WD and NCD IOS modules to broadcast a file called "NTRJ41(00)'DRIVER5". We immediately recognized this as a Nintendo DS GameID and these modules are used to broadcast to the Nintendo DS. But there seemed to be almost no mention of DS connectivity in the game or by Ubisoft, so we booted up the Wii to check ourselves.
And so the problem revealed itself. Unlike every other game that broadcasts to DS, Driver: San Francisco doesn't ask or advertise the feature. It just starts broadcasting at the start of every mission and even if there's an error, it ignores it and keeps trying to broadcast no matter what. It seems that Ubisoft had cooked up a far more effective anti-emulator trap that MetaFortress could never live up to: their coding standards. For Dolphin, these modules were low priority because there was almost no value to emulating them right now. However, because of how Driver: San Francisco is coded, Dolphin needed at least remedial support of DS broadcasting in order to run the game.
Through some light reverse engineering, Leoetlino mapped out the behaviors of the NCD and WD IOS modules responsible for DS broadcasting. He also tested DS broadcasting in games like Tales of Graces, Castlevania Judgment, Pokemon Battle Revolution, and the hidden DS broadcast features in the Mii Channel!
After a few weeks of work and testing and tweaking, things began to work. Games were able to attempt to broadcast and shutdown broadcasting without crashing. After all of these years, Driver: San Francisco is finally running correctly!
Does this mean that Dolphin supports DS <-> Wii communication? Absolutely not. The foundation is now in place and most games no longer crash when starting DS broadcasts. However, Dolphin does not broadcast any Wi-Fi data, and even if it did there's currently no DS emulator that can even listen for it. In order to get complete DS <-> Wii support in these titles, there's still a very long road ahead and a lot more work to be done. Even this remedial state, we're still running into hangs in certain scenarios, particularly in Tales of Graces. If you're familiar with the original version of Tales of Graces, this shouldn't surprise you at all as it was actually recalled in Japan.
5.0-13215 - Enable Certain Compatibility Game Patches by Default by JosJuice¶
As Dolphin has gotten more mature, we've learned more about our favorite (and not so favorite) games than we ever could have imagined. At this point, we've discovered dozens of games with bugs and behaviors that Dolphin will likely never emulate. These games are saved by various low level hardware quirks, such as CPU cache shenanigans, that would present such a performance penalty to Dolphin that no one would want to use it even if someone went through the trouble of emulating it.
In order to improve the user experience, we have changed our patch structuring to now enable certain game patches by default in cases where there is zero reason why a user would want them disabled. These games include...
- MVP Baseball 2004 and 2005
- Wallace and Gromit in Project Zoo
- Casper's Scare School: Spooky Sports Day
- Monster High: Ghoul Spirit
- Resident Evil 2 and 3
...Wait... what were those last two? Could that mean...
5.0-13452 - Support Conditional Patches and Add Patches for Resident Evil 2/3 Audio Bugs by smurf3tte¶
These two games are among the most requested to be fixed in comment sections and on the forums. Resident Evil 2 and Resident Evil 3: Nemesis are no stranger to Dolphin developers. For years now, the music in Resident Evil 2 and 3 has presented quite the mystery. An old hack, now removed, seemed to be the only answer to strange issues causing the intermittent music.
The old savior of these games was the Instant ARAM DMA hack, a heuristic that made Direct Memory Access timings to the Audio RAM instant and thus completely bypassed Dolphin's emulation of a GameCube's Audio RAM bus throughput. Since our timings were horrific back then, this little trick would fix a lot of games that had timing related trouble in the early years of Dolphin. It was a dirty, dirty hack - instant timings are extremely inaccurate and this feature caused crashes in some games - but Dolphin's timings were so inaccurate back then that, when combined with a heuristic to only turn it on when needed, it helped more than it hurt.
As Dolphin has matured, timings have improved across the board. Invisibly, the ARAM DMA hack was rendered less and less useful, and even began to become harmful due to false detections. Eventually Dolphin reached a point where every game that originally needed it worked without it! Except for Resident Evil 2 and 3. This put us into an unenviable situation. With decent timing emulation the Instant ARMA DMA hack became a dead weight, with the heuristic erroneously activating and needlessly exposing users to problems. In hopes of solving what was actually wrong with Resident Evil 2 and 3 we removed the hack despite it breaking these two games. We hoped that this would lead to someone solving what was wrong and the games would soon be fixed.
That didn't exactly happen, at least very quickly. Finally, we have a clear picture of what went wrong and why the Instant ARAM DMA hack worked. It turns out the games are erroneously zeroing buffers before the game's audio samples can be fully copied to ARAM. The partial transfer of the audio samples resulted in the characteristic on then off again music that users experienced. This is also why the Instant ARAM DMA hack worked - it made it so that everything was copied before the buffers were cleared. But since this is a game bug, why didn't console run into this problem? For that, we have to look at what happens after it erroneously clears the buffer. The troublesome
memset() call that wipes the buffers are followed by a call to
DVDRead() which issues instructions to invalidate the data cache (
dcbi instructions). This cancels the
memset and the buffers never actually get cleared despite the mistake! Because Dolphin lacks data cache emulation, the
memset() actually reaches RAM and wipes out the buffers.
To fully emulate this behavior, Dolphin would have to implement data cache emulation. Not only would this be a gargantuan task, it would also slow Dolphin's emulation performance considerably. However, these games aren't our first joust with problems like this. In the past, we've implemented patches to fix game bugs like these and bypass the need for a data cache entirely. In fact, this bug is almost exactly the same as Casper's Scare School: Spooky Sports Day's issue we patched years ago. Unfortunately, Resident Evil 2 and 3 couldn't be solved so easily. In testing, the method worked - nop'ing out the erroneous
memset() does resolve the issue, no data cache emulation required. However, Resident Evil 2 and 3 both use multiple executables, so a patch that worked on one scenario in Resident Evil 2 would cause another to crash! This would force users to constantly juggle between which patch to have enabled and hope that the loader executable itself didn't crash due to the patch! What we needed wasn't just a patch, but a smarter patching system.
Progress Report newcomer smurf3tte went above and beyond to fix this issue and extended Dolphin's game patching system to support conditional patches right in this change! With all of this together, the buffer clearing game bugs in every executable are corrected and Resident Evil 2 and Resident Evil 3: Nemesis's audio issues are a thing of the past.
5.0-13426 - DSP: Fix write masks on AUDIO_*/AR_* MMIO registers by smurf3tte¶
smurf3tte brings us some audio fixes this time around with a change to audio address masking. When investigating audio clipping issues in Nickelodeon Teenage Mutant Ninja Turtles (not to be confused with TMNT, TMNT or Teenage Mutant Ninja Turtles), smurf3tte found some new behaviors for masking in various DSP registers.
The game starts an
AUDIO_DMA_START_LO with an unaligned address. Because Dolphin was not masking off the low 5 bits as it was supposed to do, all future audio DMAs were misaligned. Further reads are not masked, so the error was not corrected, meaning all future DMAs were then offset. By adding in the correct masking behavior to
AUDIO_DMA_START_LO, the issue was quickly resolved, resulting in clearer audio with less clipping. Some of the clipping actually happens on console, which Dolphin faithfully recreates.
As a side note, this also fixes broken audio in another very popular Wii game: The Bachelor. The audio clipping behavior was nearly identical to Nickelodeon Teenage Mutant Ninja Turtles so it wasn't much of a surprise that this change actually fixed both.
5.0-13509 - Fix Wii Remote Disconnect Crash on Windows by Billiard¶
There are tons of benefits to connecting real Wii Remotes to Dolphin's InputCommon, such as emulated infrared, customizable inputs and more! Unfortunately, if the Wii Remote turned off while connected to Dolphin, the emulator would crash. This was due to a bit of messiness when connecting things on Windows. Billiard went through and cleaned things up so that Wii Remotes can power down and disconnect at any time without causing issues.
5.0-13240 - Wii Remote Motion Simulation - Allow 360 Degree Tilt by Billiard¶
Sticking to the topic of Wii Remotes, let's talk a bit about Dolphin's motion simulation. While many of our users have moved onto using Motion Passthrough, Billiard is still hard at work ironing out some of the issues in Motion Simulation. In the case of Another Code R, it was reported that Motion Simulation was unusable for some puzzles due to oversights in how it handles 360 degree tilt motions. The problem was actually very apparent - Dolphin simply wouldn't let the Wii Remote tilt more than 180 degrees. Thus if the player tried to do a tilt that went further than 180 degrees, Dolphin would simply rotate it the other way around to get to the destination angle.
Billiard corrected the wrapping behavior on tilt simulation in so that the motion will now take the shortest path to the destination angle. This now allows motion simulation tilts to spin a full 360 degrees if necessary.
Warning: Technical Information Ahead.¶
5.0-13242 - Final Fantasy: Crystal Chronicles Mogmail Crash Fix by smurf3tte¶
Yet another game debugged by smurf3tte. While this one was strongly suspected to be another dcache issue, it actually turned out to be something entirely different. It was a game bug... but one that Dolphin could actually emulate after understanding when the bug was happening. Essentially, after completing the Goblin Wall level, you'd get Moogle Mail and a cutscene. At the end of all of this, Dolphin would freak out with a ton of invalid reads/writes or the game would outright crash if MMU was enabled. This bug has been around for years and has been pretty hard to track down because it seems to appear and disappear with no rhyme or reason.
After years of confusion, everything finally makes sense. Using Store EFB Copies to Texture and RAM fixes this notorious crash. However, wouldn't someone have tried that? The answer is yes! The problem is, that by the time we were changing settings, the bug had already occurred.
To understand this problem, we first need to understand how post-processing works on the GameCube and Wii. While rendering, the GPU rasterizes to the "Embedded Frame Buffer" (EFB), a 2MB chunk of extremely fast working memory that resides in the GPU itself. While some basic post-processing can be applied while the frame is in the EFB, any effect that involves reading then distorting the frame cannot be done while it is in the EFB. So for most post-processing effects, the game will execute an EFB Copy to transfer the frame to main memory, then read the frame as a texture from which to render again with the post effects on top. However, the consoles render in 24-bit RGBA6, but cannot read a texture in that format. To deal with this discrepancy, it is common for games to convert the frame to 32-bit RGBA8 during the EFB Copy to maintain full visual quality.
Crystal Chronicles makes copious use of this method for explosions and other effects. However, the game does something unusual. Crystal Chronicles is pushing memory hard and sometimes they may go a little over. So rather than always reserving space for an RGBA8 frame, the game will dynamically choose which format to do the EFB Copy in depending on how much memory is available. To facilitate this, Crystal Chronicles checks their custom memory allocator to see if there is enough free space in main memory for the larger frame. If the check clears, the game tells the GPU to convert the frame to RGBA8 during the EFB Copy. However if the check fails, the game tells the GPU to convert the frame to the small and lower quality 16-bit RGB565 texture format. With this trick, they can go a little over on their memory usage without worrying about an overrun.
The GameCube was almost always memory starved, so why don't we see lots of games using tricks like this? Because this method carries some risk. The GPU sends the frame into main memory without bounds detection, so if the EFB Copy is larger than the space allotted for it, the copy will just keep going even if it has to overwrite game data. The GameCube and Wii have no way to deal with this so this is literally game over. But it should be fine, right? The whole point of checking the available memory and copying in RGB565 is to prevent this very scenario from ever occurring. There's no way anything could go wrong...
The game relies on a helper function to get the size of the EFB Copy, which it then gives to the memory allocator. Usually this would be done using a helper function provided by Nintendo, however the developer has reimplemented this function on their own for some reason. It does the job very well, except for one texture function in the US release of the Crystal Chronicles. For this texture their helper function is incorrect and feeds the game slightly smaller values than the EFB Copy's actual size. If this happens while the game is strained just enough that the game's memory allocator calculates that there is room for an RGBA8 frame when there is not, we have the perfect recipe for a disaster.
And this occurs at the Goblin Wall, specifically in a post boss cutscene. The memory overrun leads to graphics data overwriting a data structure of the game's memory manager, a data structure the users will access later when the player opens their Moogle Mail in a following cutscene.
This is a catastrophic game bug. Yet, why wasn't it crashing on console? smurf3tte immediately assumed the data cache was saving it just like with Resident Evil 2 and 3. However, in hardware testing this proved to not be the case. In fact, the memory is overrun in exactly the same way on console! It turns out that the specific texture data that overwrites that memory manager data structure just so happens to start with 0xFF. The game interprets this as a flag and then ignores all of the other data, bypassing the data corruption entirely. That's how console avoids the crash. Luck, pure dumb luck. Sadly there was no luck left over for Dolphin.
The GameCube and Wii are unified memory model systems, so the CPU and GPU all share one pool of memory. This allows both the CPU and GPU to edit graphics data with zero performance penalty. Modern PCs on the other hand operate on a split memory model, where the CPU and GPU each have their own pools of memory. To accomodate for this difference, Dolphin has no choice but to have duplicate graphics data in both places and sync any changes across the memory pools. This syncing is by far one of the greatest bottlenecks in Dolphin, so we have developed tons of optimizations and workarounds for this. Our primary approach to this problem is Store EFB Copies to Texture Only. This mode isolates graphics data to the GPU side, preventing the sync from occurring and giving a huge speed up. However, the CPU can decide to tweak graphics data at any point and there has to be something on the CPU side, so in place of the graphics data we write all zeroes. While Store EFB Copies to Texture Only is definitely hacky, it works well enough in most games that it is enabled by default.
This takes us back to Crystal Chronicles. At the Goblin Wall when the bugged texture function is called and the memory overrun occurs, instead of a texture data overwriting the memory manager data structure, it is zeroed out. When Moogle Mail is opened in the following cutscene, the game reads 0x00 and decides to read the remaining zeroes as pointers. These are of course invalid and lead to tons of errors. There's our crash! But it turns out that to resolve this issue, all we have to do is disable Store EFB Copies to Texture Only by default for Crystal Chronicles. The graphics data then overwrites the memory manager data structure and the game reads 0xFF then ignores the rest of the data, just like real hardware! All of this analysis, research, and console testing, and our crash is fixed by changing one word in a GameINI.
Of course we had disabled Store EFB Copies to Texture Only while trying to debug this crash. In fact, it is one of the first things we tried. However, there's a long gap between when the bug is triggered and when it actually manifests. By relying on savestates close to the crash, as we usually do to speed up testing for issues like this, the bug had already occurred and everything we tried was irrelevant. Even though the final solution was a simple one, it took smurf3tte digging deep and truly understanding the problem for the solution to be found.
Are the casual readers gone? Yea they'd never make it this far into the Report. Good, ok technical detail lovers, let's get into the delicious exceptions you've all been waiting for!
So it turns out that, by default, disabling Store EFB Copies to Texture Only actually doesn't recreate the console behavior! The conclusion ignored a new-ish on-by-default EFB Copies to RAM optimization called Deferred EFB Copies to RAM where Dolphin delays EFB Copies to the host RAM so it can batch many EFB Copies to RAM together for a sizeable performance boost. However, the optimization has an optimization (oh yes) where it will notice if one EFB Copy is totally overriding another in a batch, and just, doesn't include the first EFB Copy in the batch of EFB Copies going to RAM. This bug just happens to be one of those cases. With Deferred EFB to RAM Copies enabled, the EFB Copy with the buffer overrun never happens on the RAM side, skipping the bug all together! While this is inaccurate, and we could disable deferring by default to allow the overrun to occur exactly as on console... the optimization is a sizeable performance boost, and having tons of error spam is never nice so smurf3tte decided to make a note of it and then leave it be. We couldn't figure out how to explain that exception without ruining our sweet "we tried EFB to RAM!" pay off so we left that out of the conclusion.
But there's more! smurf3tte was not done! They also created a patch to fix the game bug! Specifically, it fixes the bugged texture function's calculation for the memory the EFB Copy will use, which then allows the game to adapt correctly and avoid the overrun. However, there are borderline cases where this can lead to the game running in RGB565 mode where it previously ran in RGBA8 just fine. Since we have the EFB Copies to Texture and RAM solution that performs slightly better, the patch isn't really needed. But since smurf3tte already did the work, they included the patch but did not enable it by default.
With that our time is up, and you now know far too much about Final Fantasy: Crystal Chronicles. If you for some reason have any additional questions, you can meet us in the forum just down the hall.
Last Month's Contributors...¶
Special thanks to all of the contributors that incremented Dolphin from 5.0-13180 through to 5.0-13603!