Dolphin Progress Report: January 2017


Sometimes, it's easy to forget how much work there is left to do on a refined emulator. While the rush of getting a new game to boot or discovering a crazy feature hidden within an obscure gem never gets old, those moments do tend to get further and further apart as accuracy increases. As if to defy fate itself, excitement reigned over the month of January as a plethora of ancient bugs were fixed and many unbootable titles finally saw their day of reckoning come to be!

Among the new recruits are the final Virtual Console game, a massive Wii MMO that installs itself to USB, two games where we're almost certain the developers purposefully put code in to defeat Dolphin, and two channels developed by the remnants of Factor 5.



This is a massive Progress Report, so buckle up and enjoy this month's Notable Changes.


Notable Changes


5.0-1709 - Use Lower Left Origin for Point Texture Coordinates by stenzek

This is a painfully simple change that didn't cause as many issues as one would imagine. By incorrectly putting the origin for point textures at the top left instead of the bottom left, point textures were drawing upside down. The only reason this isn't more noticeable in games is because most point textures don't need any particular orientation to look okay. The fires in Mega Man Network Transmission are an example of this; they've been rendering upside down all along, but even after comparing before and after screenshots, it's basically impossible to see a difference without fifoci!


mmnetfire-broken.png

The fire effects are all drawn upside down, but they still look fine.

mmnetfire-fixed.png

Even knowing what's changed, it's hard to spot the difference!



But, that wasn't a known bug. The game that used this for a non-trivial texture is actually Mario Golf: Toadstool Tour. This game uses point textures for UI elements that look quite silly when flipped upside down.


mariogolftt-broken.png

Sure, it's not correct, but it does have entertainment value at the very least.

mariogolftt-fixed.png

Unfortunately the fixed version is a lot less funny.



As of this revision, all reported graphical issues in Mario Golf: Toadstool Tour are fixed! For a title that suffered from zfreeze issues, spiky vertices, line-width problems and more way back in Dolphin 4.0, it's come a long, long way.


5.0-1837 - Initial WFSI/WFSSRV implementation by delroth

This addition is the result of a huge reverse engineering effort! Despite that, it isn't going to have much of an effect on the average user. So what does all of this work actually accomplish? It's the first step of many on Dolphin's path to being able to run Dragon Quest X: Mezameshi Itsutsu no Shuzoku Online and allows for its installation process to complete.

You read that right, installation - Dragon Quest X is a still active MMO that actually saw release on the Wii. Unlike any other title released for Wii, it uses an included USB dongle to install itself off of two discs. Everything about Dragon Quest X is unique for the Wii; it even has its own IOS version; IOS59, which adds the WFS module. Up until last month, close to nothing was known about WFS and the service associated with it!

delroth once again put his reverse engineering skills to the test to document and understand the largest IOS module for the Wii. After buying the game brand new in a store while visiting Japan, instead of playing it, he wanted to unearth the secrets hidden inside.

On the theme of being unique, Dragon Quest X appears to be the only true two disc Wii title. While other products did ship with two discs, each disc was its own separate game with its own GameID. Dragon Quest X instead uses two discs for installation, the extracted files taking up around 5 GB + patches. There's actually no reason why it couldn't have fit on a Dual-layer disc. Our only guess as to why it'd be a double disc game is that maybe it was cheaper, and since it is installing itself, there's no need to disc-swap after the initial installation, more or less erasing the biggest annoyance with multi-disc console games. But if two discs just wasn't enough, it also has three expansion packs on top of that, each sold as a separate game!

But to install all of that to USB, it has its own file system known as WFS; which we can only assume means "Wii File System". In order to get Dragon Quest X to install, delroth had to implement enough of the file system so that it could write files to the USB successfully. Considering this is the largest single module and even contains encryption, this was no easy feat.

The result of countless hours of work is that you can now go through the installation process of Dragon Quest X to a virtual USB drive in Dolphin! The installation takes roughtly 30 minutes on Wii and ~16 minutes in Dolphin with the default settings. If you're impatient though, you can disable the framelimiter and enable fast disc transfers to get the job done in about three to four minutes on a fast PC. Note, this does not require a real USB stick; it emulates a USB stick within Dolphin's Wii NAND.


dqxsysmenu.png

All of that work, and this is about as far as you'll get... for now.


Remember: this is an initial implementation. Though the files are installed, the game can't actually read them yet! Installation doesn't finalize properly and the channel instantly crashes due to missing functionality. Why merge it if it's not done? Because it's such a huge change, there was a big potential for merge conflicts if it was left un-merged. It's not going to harm any of the other titles, so, merging it now just means that it's easier to work on in the future. Considering that Dragon Quest X for Wii is still active online and has a single player portion, it's definitely something we'd break a few rules to get done!


Bonus Features!

Because Dragon Quest X required some more IOS features to be implemented, some other games have gotten additional functionality. While the Mario Kart Channel is no longer functional due to Nintendo pulling the plug on it, you can belatedly install it from Mario Kart Wii assuming you have a complete NAND. Other titles, such as Wii Fit, can install channels that are still functional and they will work as expected.

Installing Wii System Updates + Channels in Dolphin


5.0-1945 - Only allow reset/power when core is running by leoetlino

Ever have Dolphin just randomly crash on you or do something weird without knowing why? Do you have the "reset" or "power" hotkey set to any of your commonly used keys?

A tester accidentally left the reset hotkey on a key he hit quite often, and, it turns out Dolphin really didn't like if you tried to resetting before a game was booted. Without having a game to reset, Dolphin would suffer some memory corruption, and the game would usually crash on boot, though memory corruption can result in nearly anything.

leoetlino's genius solution to this is to actually check if a game is running before sending reset and power commands. Such a brilliant check was definitely not something we should have thought about a long time ago to protect from something like this.


5.0-2023 - Allow setting GC/Wii System Language Per-Game by iwubcode

This is a simple change the adds functionality for specifying a Wii system language per-game. While this may not seem like a big deal, there are a lot of games that behave differently under various languages. Wii Party's NTSC-J version has a half-complete english translation that shows up when launched while the system language is set to English. Tales of Graces goes the other way and will refuse to boot at all if your system language is set to English.

Instead of having to modify your global system language for these cases and more, you can now just set them per-game and not have to worry about it.

For instructions, see the forum thread.


5.0-2123 - Set up IOS RAM information area with real values by booto

For the past month, booto has quietly embarked on a journey to remove hardcoded IOS-HLE values from Dolphin. Why? Many of Dolphin's values are from pre-2010 and even then are incorrect anyway! The Wii's Internal Operating System (IOS) has updated itself dozens of times since Dolphin's Wii emulation debuted, and while they've changed, Dolphin's IOS-HLE has not. These tiny differences didn't make much of a difference in most games, but, at least three downloadable titles absolutely used the memory map to its absolute fullest.

The focus was actually on Majora's Mask (VC). While it may not sound very interesting to get a Virtual Console game running in Dolphin, it was the only one known not to boot. Each region had its own way of failing: NTSC didn't boot at all, NTSC-J crashed on save-file creation, and PAL crashed on selecting a language. In the cases of the PAL and NTSC-J versions, if you imported a save from console you could at least get in-game, though it would crash whenever you saved.

For additional help in figuring out what Majora's Mask was doing, leoetlino extracted the symbol map from Twilight Princess to use with Majora's Mask. These symbol maps, when included with a game, make reverse engineering much easier. With a full symbol map, you can actually see what developers called a function in memory, making it much easier to understand what they're trying to do.

While not the most useful of the finds from the symbols, leoetlino was able to draw more log messages from games thanks to finding another logging function included in the symbols. He threw those symbols at Majora's Mask and...


cachef.png

Nintendo: a family friendly company.


This vulgar message is actually directly related to the issue, but, wasn't enough to solve anything. As such, they used their ultimate weapon: hardware testing. With his USB Gecko, JMC4789 was able to watch the Wii's memory while the game was running and gain valuable insight into what the game was doing differently. Leoetlino and booto instructed him on where to put memory breakpoints and that lead to the perhaps obvious conclusion that Dolphin's behavior was diverging as it setup the region in memory to store the savefile.

Many of Dolphin's IOS-HLE values are based off of IOS9; which is the earliest IOS used in commercial titles. Over time, new IOSes were created which added new features, including big ones like USB 2.0 and WFS. Majora's Mask used IOS33; which was right after some of the memory regions were very slightly modified. On a bit of a hunch, they used a homebrew game launcher to force Majora's Mask to boot on IOS9 and managed to make the game hang the same way on console.

From there, they knew for certain that Majora's Mask could work on Dolphin if IOS-HLE were more accurate. A hack to modify Dolphin's hardcoded values to that of a newer IOS would have been more than enough to fix it, but booto decided he wanted to end this once and for all.

Over the course of one week, booto traversed through each and every IOS he could get his hands on to extract useful values and find differences. The change that Dolphin was failing on in this case was visible in plain sight; the MEM1_ARENA_END was set too small, and Majora's Mask just so happens to use the very end of MEM1 for its savefile. When the game realized there wasn't enough there, it tried to copy the savefile to 0x0, which, in simple terms, is very, very bad and causes bad things to happen.

With this change, games will now use the real values from whatever IOS they'd load on console within Dolphin. This fixes Majora's Mask, regardless of its region and brings compatibility to both the Netflix Channel and Amazon Instant Video Channel! Unfortunately, Dolphin's lack of support for some WiiConnect24 features prevents them from actually connecting to their online services. These changes are also hooked into ES_Launch, so these games can be booted from the Wii Menu as well if you so desire.


dqxsysmenu.png

With the GameCube version being buggy even on console, this is the best way to play Majora's Mask in Dolphin!


5.0-2152 - Fix Savestates in Bluetooth Passthrough Mode by leoetlino

When it was first merged, one of the biggest sacrifices of using Bluetooth Passthrough was losing savestates. While it's not possible to make savestates work in every situation (we can't exactly savestate external hardware,) leoetlino returned to work on Bluetooth Passthrough and get savestates working in as many situations as possible.

As such, you can now use savestates in Bluetooth Passthrough under the following scenarios:



This means that if you're playing a game like Skyward Sword, it's easy enough to savestate before a boss battle and reload it to your heart's content, so long as you never disconnect the Wii Remote or close the game. If you do happen to close the game and still want to use the savestate, you can! Just disconnect any Wii Remotes before loading a bluetooth passthrough savestate and they're guaranteed to work.

Note: Savestates created in emulated Bluetooth mode are still not compatible with Bluetooth passthrough mode because the emulated Bluetooth adapter data is part of those savestates.


5.0-2178 - Enable Branch Following by degasus

This feature is huge for people playing certain games. It allows Dolphin to detect idle loops it previously could not in certain games, including Fire Emblem: Radiant Dawn and Xenoblade Chronicles.



This huge performance boost is actually just the beginning for these titles; we're still emulating the instructions, meaning that when we actually idle-skip them directly, there will be a second big performance boost.

The reason that branch following is such a huge help for these games is because their idle-loops can be combined into one JIT block. Combining instructions is one of the main advantages a JIT has over an interpreter, and in many scenes the idle loop can be well over half of the CPU time! The effect can be especially noticeable in games that are slow in light areas such as menus; because the idle-loop will be in much more use during those scenes. On that note, other games that could be affected by this that we don't mention are likely those that are slow in areas that you'd think would be fairly easy to emulate.


5.0-2192 - Auto-Adjust Window Size Improvements by AdmiralCurtiss

This is a simple change that should be nice to people who like to use "Auto-Adjust Window Size" in the graphics settings. This setting allows you to set the Window Size to exactly the internal resolution you're using. It's a nice way to get precise window sizes, especially handy when doing recordings and other work.


fzerobars

4:3 with black bars.

fzerowidebars

16:9 with black bars.



As you can see, while Dolphin would get the resolution of the game correct, it would not respect the aspect ratio of the game itself while setting the window size, resulting in a lot of black space around the edge of the window. AdmiralCurtiss returns to fix a friend's pet-peeve with the feature so that now the window size now takes into account the game's resolution.


fzerocorrect.png

4:3 with correct window size.

fzerowidecorrect.png

16:9 with correct window size



5.0-2204 - Hack to protect lower MEM1 from Malicious Game Code by booto

The Disney Trio of Destruction are three titles that are special. While a lot of games have strange behaviors, difficult to emulate quirks, and other potential pitfalls, these three games are the only ones where we're almost certain they were purposefully targeting Dolphin with odd behaviors.

While the original plan was to merge this change next month alongside a big article talking about the history of the games, their multiple levels of security, and why we believe they were targeting Dolphin rather than hacked Wiis with their anti-piracy, etc. Instead, it got merged by someone with an itchy trigger finger not even a few days ago. We'll have to get into the history of the Disney Trio of Destruction once all of that is prepared, but for now, let's talk about this change.


notphotoshopped.jpg

With their sinister covers, we should have known these games were trouble.


Firstly, this hack only affects two of the Disney Trio of Destruction: Disney's Cars 2 and Disney Infinity. The third, Toy Story 3, was older and had protection that was solved years ago by skidau. The remaining two titles were thought to be impossible. We had placed our hopes in Dynamic BATs to finally fix Cars 2 and Disney Infinity, but it didn't. When that wasn't enough, interest waned and the games were left behind once again.

booto did not give up on the games. Throughout the years, these games have been reverse-engineered to hell and back. We knew of this protection already, but, fixing didn't matter because we'd then just run into another layer of their security. But, because Dolphin has gotten so much more accurate, this hack is actually enough to get us in-game. So, what is this final layer of protection?

While we'd love to take a whole article talking about how this is nefarious and specifically targets Dolphin's way of emulating the Wii CPU, Parlane merged it early so you don't get that quite yet. In basic terms, Disney Infinity and Cars 2 do a very specific amount of writes to lower MEM1, where a lot of really important data is stored. But, it writes just enough that the CPU doesn't flush it to MEM1; it remains in the CPU L1/L2 caches. It then tells the CPU, nevermind and invalidates the CPU cache. Thus, even on a hacked Wii this would do exactly nothing; it's specifically targeting a weakness in emulators.

Emulating the CPU cache in Dolphin would likely slow-down Dolphin by a factor of ten or more. While we prefer not using hacks to "fix" games, we found it to be permissable in this case due to how specialized of an issue it is. Even if the L1/L2 caches are implemented in the future (which is definitely a possibility,) it will likely be something only used for testing, and thus this hack would remain so that people could actually play these two games. It's a compromise, one that no one is entirely happy with, but the game behavior necessitated it in this case.


Cars 2 in Dolphin (4K)


While we didn't know it at the time, it's worth noting that both Disney Infinity and Cars 2 require the Dynamic Block Address Translation change. Without it, the hack does not work, making these the second and third games to require the feature.

We'll have more information on the Disney Trio of Destruction in an upcoming Feature article.


Last Month's Contributors...

Special thanks to all of the contributors that incremented Dolphin from 5.0-1617 through to 5.0-2235!


Next entry

Previous entry

Similar entries