To the Screen with Hybrid XFB

Dolphin has been around for over 14 long years at this point. Goals, expectations and standards have shifted quite a bit since the beginning. At one point, just booting a game at all was good enough, regardless of what you would see or hear! Compatibility has gone from a few select titles to almost every game released across two consoles. Considering all of that, it should be no surprise that some solutions that worked in the past slowly came to be a burden going forward. In this case, we're talking about eXternal FrameBuffer emulation.

Frames output by the GPU are stored in game allocated RAM, where the CPU can make finishing touches before sending the frame to the screen. Since most games don't do anything interesting during this step, XFB emulation has actually lagged behind a bit. While the old solutions work, sometimes it's necessary for big rewrites to take a step forward. In this case, there wasn't much of a choice but to kick Dolphin's decade old XFB emulation to the curb in order to revolutionize how the emulator outputs frames.

Our new feature, Hybrid XFB, simultaneously simplifies code while making the fast path more accurate and the accurate path more suitable for general use. For those that were aware of it, Hybrid XFB has been awaited with the same anticipation as other big features for good reason. Hundreds of games will see either performance increases or better visual output thanks to this rewrite!

Rendering Frames on the GameCube and Wii

When the GameCube/Wii is rendering something, the GPU renders to the Embedded FrameBuffer (EFB), a special 2MB chunk of memory on the GPU itself, separate from the shared main memory. Once the GPU is done rendering a frame, it executes an EFB copy with a special bit to convert the frame into the YUV format (which the GPU can no longer modify) and sends it to a designated region in main memory known as the "eXternal FrameBuffer". In Dolphin, we refer to these as XFB copies. On console, these special EFB copies reside in the XFB region until the video interface (VI) is ready to output the frame to the screen, scanned line by line over the course of a frame's duration (scanout). Even the scanout process isn't static, as games can modify the VI registers to change how the data in the XFB region is displayed. For example, when switching from interlaced output to progressive output, a game changes the VI register controlling that.

In the default case, a game will have one frame being scanned out and another ready for scanout, all while the GPU renders the next one. However, the CPU is still free to do whatever it wants until scanout begins. After all, the XFB copies are just data sitting in RAM! However, a vast majority of games don't elect to take advantage of this feature, instead scanning out the frame from the XFB region unmodified.

How Dolphin Handled XFB Usage

Since most games don't modify what is placed into the XFB regions, early Dolphin developers were able to mostly ignore emulating XFB behaviors. The Legend of Zelda: The Wind Waker, Super Smash Bros. Melee, Luigi's Mansion along with most of the other early supported titles in Dolphin don't require much in the way of XFB emulation! Even with the library now mostly set, XFB Disabled worked fine for thousands of games across the GameCube/Wii library.

What did XFB Disabled do? When the emulated GPU does an XFB copy, Dolphin would immediately send the rendered frame to the screen, completely bypassing VI and any potential CPU modifications. In order to skip VI and not completely fail, Dolphin substituted default values that worked fine for most games.

XFB Disabled had a pretty glaring weakness though - if the XFB Copy Command was used for anything except whole frames, output would get mangled very quickly. This happened with games like Star Wars: Rogue Squadron 2, which uses a non-standard method to render frames in its menus.

This is how XFB Disabled displayed Rogue Squadron's menus.

While supporting the biggest use case was nice, having a path that replicates what console actually does was extremely important. That brings us to XFB Real, a solution that perfectly followed the pipeline so that every title would work correctly. It even went as far as to store XFB copies in emulated main memory, which greatly lowered performance and forced output to native resolution. Why does doing this limit output to native resolution? The GameCube/Wii have a specific amount of memory, and games are specifically coded to use the addresses and size of said memory. If Dolphin were to try to make the memory bigger, we'd have to translate all memory addresses to the larger space, which would be a nightmare, especially with address wrapping. Even if it is possible to translate all of these addresses (it may not be) so a game could use more memory, we still have to hack each individual game's rendering pipeline to actually use that larger space.

XFB Real sounds foolproof, but how isolated it was from the other solutions turned out to be a major flaw. Because it shared little with the other codepaths and very few people tested it, regressions were far too common and would sometimes take a long time to be reported. The most common reason was that XFB Real actually relied on VI emulation being correct, meaning in some cases XFB Disabled would look correct while XFB Real wouldn't. It wasn't until shortly before the 5.0 release that booto revolutionized VI emulation with a huge rewrite that allowed Dolphin to use real VI emulation provided by the game rather than hardcoded values. But even beyond that, all manner of bugs and rendering issues slipped into XFB Real because it didn't see enough testing. Thanks to the efforts of booto, phire and several others, XFB Real was finally devoid of major issues heading into the 5.0 release.

Despite working perfectly, most users ignored XFB Real due to resolution limitations.

Around the release of Dolphin 2.0, an alternative method was developed that would combine the best aspects of XFB Disabled and XFB Real: XFB Virtual. This option was an attempt to emulate most of the XFB pipeline by storing the XFB copy in a buffer outside of main memory. The downside to this is that the emulated CPU can't access this buffer, but, on the other hand, the buffer is outside main memory so it can hold arbitrary resolutions, while still applying VI to the XFB copies.

With higher accuracy than XFB Disabled and none of the limitations of XFB Real, XFB Virtual should have been a slamdunk of a feature! ...In theory, at least. XFB Virtual was more or less its own implementation and didn't really share much code from the other codepaths. As such, maintaining it was difficult and a lot of bugs slipped in. While it was good enough for a few key cases, XFB Virtual most often ended up marrying problems of both XFB Disabled and XFB Real into an unholy abomination of crazy problems. XFB Virtual handled a few cases it was designed for fine, but, wasn't very good at actually emulating the pipeline. Thus, a lot of cases would break it in tremendous ways.

XFB and Games

We've described how the process of rendering a frame works on console, and Dolphin's three solutions to the problem. Now it's time to actually look at some games and see how Dolphin handles their XFB behaviors.

No Special XFB Usage

As one of the earliest titles to run in Dolphin, it shouldn't be a surprise that XFB Disabled works without issues for The Legend of Zelda: The Wind Waker.

XFB Disabled bypassed VI, but it doesn't matter as the game doesn't use it!
XFB Virtual added some VI emulation, but because the game doesn't need it, nothing has changed.
Full emulation of the pipeline with XFB Real forced the game down to native resolution, but it looks the same otherwise.

With The Legend of Zelda: The Wind Waker, it was actually favorable to leave XFB emulation off during most of Dolphin's development. If you enabled XFB emulation and held B to enable progressive scan, you'd run into a very annoying issue: because Dolphin hardcoded VI registers, Wind Waker would suddently be outputting at a crisp 15 fps. Because most users were using XFB Disabled, it took until late into the 4.0 era to narrow down what was causing this and actually fix it.

Wind Waker works so well with XFB Disabled because it, and most other games, outputs whole frames in a single XFB copy and sends them to the screen without any modification.

Vertical Tiled Rendering

Vertical Tiled rendering is the most common case that XFB Virtual is used to quell. Games like Star Wars: Rogue Squadron II and its sequel Star Wars: Rogue Squadron III will output several XFB copies that make up a single frame instead of outputting it all at once.

The Rogue Squadron games use two primary XFB copies when rendering menus and the opening.

Why is that? Not content to leave any stone unturned, Factor 5 is going through extra steps during rendering in order to get antialiasing! These titles output XFB copies with three samples per pixel, allowing them to achieve 3xMSAA on the GameCube! Unfortunately, this comes with a tricky downside. At a comparatively massive 96 bits per pixel (vs 48 bits without) this copy can't fit into the 2MB Embedded Frame Buffer in the GameCube/Wii GPU. In order to overcome this, they must render the frame in chunks! By producing several separate XFB copies per frame and stitching them together with VI, games could achieve 3xMSAA on the GameCube!

Returning to this example, Rogue Squadron II outputs five XFB copies every frame during menus, including three of which that aren't directly used for rendering. XFB Disabled output every single one every frame.

Ironically, Dolphin doesn't even benefit from Rogue Squadron 2's method of getting 3xMSAA, as it doesn't support using the 96 bit EFB copies! That doesn't mean Dolphin gets away with bad emulation though, as that detail doesn't stop Rogue Squadron 2 from outputting the screen in chunks.

As a special note, a lot of users think that Dolphin has trouble with the GameCube Main Menu. Unlike most games, Dolphin cannot apply an XFB setting by default for the main menu, thus when booting a game from the main menu, Dolphin will adopt the settings of the game instead. Because XFB Disabled is the default, the main menu will look like a mess by default as it uses tiled rendering as well.

A ton of issue reports have come of XFB Disabled failing to handle this.
Who would have thought the GameCube Main Menu used anti-aliasing?

Horizontal Tiled Rendering

Nintendo ran into a bit of a problem when working on emulators for Sega Genesis/MegaDrive VC games. This is another case where the EFB copies couldn't fit into the EFB, so they had to break it apart into chunks. In order to compensate for this, the emulators would render most of the image in one XFB copy and the rest of the image in another XFB copy, then stitch it together and scale it to size with VI.

Sonic and Knuckles and the other MegaDrive/Genesis Virtual Console games render XFB copies in pieces.

XFB Disabled would just see two XFB Copies per frame and output each piece as it showed up. On a 60 FPS monitor with vsync, you'd only actually see one piece. Depending on your luck, the game may look playable with only a sliver of the screen missing! But on a 120hz display, it revealed its true form.

We've slowed this down so you can see it flipping between the two different XFB copies per frame.
XFB Virtual should have output this correctly but didn't.
XFB Real managed to do the correct output in modern builds. Before proper VI emulation was added, Dolphin would hard-crash!

All of this work is done so that these Genesis/MegaDrive games can run without having to compromise their aspect ratio or pixel resolution. So in a way, even if it makes things a bit harder for Dolphin, it's good to see the emulator devs at Nintendo working hard!

Panning

Panning is a feature of VI that allows games to only output a piece of the XFB copy to the screen. Only when in interlaced mode, many GC NES emulators, such as those in The Legend of Zelda: Collector's Edition and Animal Crossing, will use panning when outputting frames. For the longest time, the only way to make these NES games work in Dolphin was to activate the game's built-in progressive support by holding B during startup!

Again, this was a consequence of Dolphin hardcoding VI registers. Once that was finally fixed and Dolphin listened to what the game was telling it, this case started working in XFB Real. So what's going on?

This is how the XFB copy looks when output with the "default" VI values in XFB Disabled.

This is the mess of an XFB copy that the game gives the GameCube. Most of it is cropped out - only the left copy of the game is actually used. The reason the screen is so tall is because these games render in double strike. Namely, the interlaced lines are doubled to attain progressive output at the cost of half of the vertical resolution. Of course a 256x240 title like NES games is already at half the height of 640x480, so it works great! With that in mind, if you crop everything else out and halve the vertical resolution, the output makes a lot more sense.

XFB Virtual supported cropping and zooming... incorrectly.
XFB Real correctly handled this feature.

For a time, XFB Virtual actually correctly output this particular scenario for a while. Unfortunately, when correct emulation was added so that XFB Real would could support it, XFB Virtual was broken! Having three completely separate paths for XFB emulation was a nightmare to maintain.

Reading Old Frames

Super Smash Bros. Brawl (partially thanks to its many mods) is among the most popular games to play in Dolphin. Naturally we didn't want to force the game to use XFB Real, even though it was the only way to correctly see the classic mode results screen!

One of our favorite Dolphin exclusive Brawl characters: Green Bar!
XFB Real allowed the game to read the contents of the screen to pick out the character.

Super Smash Bros. Brawl reads a previously scanned out frame in the XFB region and integrates it during rendering. With XFB Disabled/Virtual, the XFB is outside of main memory, so all the game can read is some default values from that region of memory. Considering that the majority of users would rather have high resolution rendering and higher performance over a non-essential effect working, XFB Real was not imposed by default in Super Smash Bros. Brawl.

Rendering without the GPU

Running homebrew in Dolphin is a frustrating experience. Every single time you had to switch to XFB Real to see anything. This is actually why! A lot of homebrew render directly to the XFB region with the CPU, skipping the GPU entirely. Many games use this trick for rendering loading screens or FMVs, completely tripping up Dolphin's less accurate options. This can be seen even in early GameCube titles, such as Phantasy Star Online I & II.

We could have done a side-by-side with a blackscreen but that'd been silly.

Only XFB Real worked in this scenario because it was the only option that allowed the CPU to modify the XFB region in main memory. Considering that tons of games use this style of rendering for FMVs, this was a huge oversight.

Go Vacation

While this may sound like another shovelware Wii minigame bash, Go Vacation is actually a gigantic game that takes advantage of the Wii's unique set of controllers for many minigames and events spread out across four sizable overworlds. This level of mastery over the Wii extends beyond motion controls. At first glance, it looks as though the whole game works fine under XFB Disabled, but some minigames manage to trip it up. In the Pie Toss minigame results screen, XFB Disabled gets baited by extra XFB Copy commands that VI was never going to send to the screen.

You can see the process of the avatar rendering getting pushed to the screen!
This is how the game is supposed to scan out.

Speaking of frames never meant to be seen...

Extra/Incomplete Frames

If you've played any of these games, you may have noticed some garbage at transitions and loading screens. That's because these games send images to the XFB region that they never intend to be scanned to the screen! Instead, the game sends a frame to the XFB that it wants VI to just reuse and scan out over and over, while ignoring anything else that may happen in the XFB region.

  • Red Steel - On the initial ES_Launch of the main game, you can see a garbage frame because the game stops rendering but Dolphin does not.
  • Metal Gear Solid: The Twin Snakes - Certain cutscenes have loading screens where incomplete geometry is rendered. These XFB copies were never meant to be scanned out.
With XFB Disabled, this random flash of geometry interrupted the opening.
XFB Real handled this case correctly.
  • Wallace & Gromit in Project Zoo - There's a "Disc not inserted" ready to be scanned out when the game boots. Because this happens before a load, it actually stays on screen for several seconds when the game isn't actually rendering anything but a splash screen on XFB Real!
  • Battalion Wars 2 - Another disc not inserted image, but only for one frame. Same issue as Wallace and Gromit.
  • Metroid Prime: Trilogy - The last XFB copy sent to main RAM when loading any of the main games is never actually scanned out. Because we send it to the screen, you get to see it with some corruption.
Honestly wouldn't it have been cool if the Metroid Prime 3: Corruption logo did get corrupted on the last frame before loading?
When it's working correctly, the corruption is held back, never to see the light of day.

All of these issues can be fixed by enabling XFB Real, but, very few users are going to choose fixing some garbage during a loading screen over high resolution output.

Hybrid XFB - A Modern Solution to XFB emulation

Hybrid XFB is Dolphin's answer to all of these problems, and it's been in the works for quite a while. The original incarnation of Hybrid XFB wasn't actually that interesting, though. In early 2015, phire attempted to fix up XFB Virtual to handle some of the situations it should have supported but didn't. But after working with the XFB code for a while, he realized that just fixing it as it was wasn't going to help anything. He became demotivated to work on a subpar solution and began redesigning Hybrid XFB as something better.

The Ultra-Mega-Super-Hybrid-XFB branch was actually the first modern incarnation of Hybrid XFB, and turned up June of 2016. This was a massive branch that threw out all of Dolphin's old XFB code and wrote new code hooking up XFB emulation to the texture cache. Unfortunately, the project proved to be too much to do alone and he eventually got burned out. Hybrid XFB went into a deep slumber, seemingly another feature lost to time.


Development Timeline of Hybrid XFB

  • February 14th, 2015 - phire announces Hybrid XFB
  • March 2015 - Impending Feature Freeze for 5.0 delays Hybrid XFB
  • January 8th, 2016 - phire returns to work on Hybrid XFB
  • June 4th, 2016 - Ultra-Mega-Super-Hybrid XFB is announced, rewriting the feature and avoiding the problems of the first attempt.
  • September 13th, 2016 - Last activity on Ultra-Mega-Super-Hybrid XFB
  • May 29th, 2017 - iwubcode unexpectedly revives the project.


With the project silent for nearly a year, iwubcode unexpectedly revived Hybrid XFB with a new pull request. While iwubcode has made the Dolphin Progress Report for their work on the Vertex Offset Hack and other changes, they were fairly new to the project. Could someone with limited experience tackle the gargantuan task of rebasing and completing Hybrid XFB? Turns out the answer was yes!

After several months of work hooking up Hybrid XFB to today's codebase, stomping out the remaining bugs, and cleaning up the code, Hybrid XFB was finally merged. As of development build 5.0-5874 you can now use Hybrid XFB for yourself. So how does it work?

Revolutionizing XFB Emulation with Hybrid XFB

Hybrid XFB is surprisingly simple. On the GameCube and Wii, eXternal Frame Buffer copies are nearly identical to Embedded Frame Buffer (EFB) copies. In both cases, the console's GPU is storing an image from its embedded frame buffer to a buffer on main memory - EFB Copies store rendered effects for a frame the GPU is still working on; XFB copies store the final composited frame in YUV format to be scanned out. And both buffers are only able to work at native resolution.

While XFB emulation was stagnant, EFB emulation was being highly refined. Partial updates, stitching, hashing, high resolution support, and pretty much anything else we'd need for a better XFB solution was already in the EFB Copy codepath. Hybrid XFB throws out over 12 years of gnarled, neglected, hacky XFB emulation code and plugs XFB copies into the texture cache code of the EFB Copy codepath! And just like EFB copies, we now have two options for XFB copies to balance performance and accuracy, with their names taken from their EFB siblings.

Store XFB Copies to Texture and RAM

Hybrid XFB follows the pipeline accurately and copies a frame into emulated main memory every frame. While this may not sound any better than XFB Real, it actually has one key difference. Store XFB Copies to Texture and RAM is not limited to native resolution!

How is this possible? Well, much of the work was already done for Store EFB Copies to Texture and RAM. When the XFB Copy command is called, Dolphin moves the arbitrary resolution frame from the host GPU's framebuffer and places it in a texture cache Dolphin reserves in VRAM. Then Dolphin reads the texture cache on the host GPU and copies a 1x native version of the frame to the emulated GC/Wii main memory, and creates a "hash" from the properties of the frame. If the game hasn't chosen to modify the frame while its in the XFB region, when VI calls for the frame its request will match the hash Dolphin made earlier, allowing Dolphin to know that it's safe to use the arbitrary resolution XFB Copy in the texture cache. However, if the hash does not match, the emulated CPU must have modified the frame in some way and Dolphin will use the frame in emulated memory instead.

All of the above is straight from the EFB Copy code path. This is how Store EFB Copies to Texture and RAM can support various effects without dropping down everything to native resolution. The main downside of this technique, and why Store EFB Copies to Texture and RAM is so demanding as well, is the process of copying a frame from the texture cache to the emulated GC/Wii Main Memory. To do this, Dolphin must copy a frame from the host video memory to the host system memory, which is a slow and performance intensive action.

Store XFB Copies to Texture Only

Store XFB Copies to Texture Only exists to work around that performance issue. Just like Store EFB Copies to Texture Only, instead of doing an expensive copy from the texture cache on the host GPU, we simply give the emulated GC/Wii main memory default values. Specifically for XFB, we give it magenta.

The downside of this performance trick is that if the CPU tries to modify the frame in the XFB stage, that frame or section of a frame will return as magenta. We'll have some examples of that behavior below.

Hybrid XFB Test Cases

Tiled Rendering and Panning

Earlier we listed three different cases that required good VI emulation that all behaved differently under the old XFB code.

  • Star Wars: Rogue Squadron 2/3 - Vertical Tiled Rendering
  • Sonic and Knuckles (VC Genesis/MegaDrive titles) - Horizontal Tiled Rendering
  • The Legend of Zelda (GC NES emulation) - Panning

All of these cases fall into a codepath that puts together output based on VI registers. Tiled rendering is handled through the texture cache and the stitching code to put multiple pieces together. That same stitching code can be used to grab a piece out of a bigger XFB copy as well! That's how Dolphin handles panning, all in one codepath that's much easier to maintain. Now that everything is running through one codepath, Dolphin couldn't just special case each scenario and actually had to do things correctly.

Rogue Squadron 2's frames are made up of 5 XFB copies in some scenes!
NES GC games in Animal Crossing and The Legend of Zelda: Collector's Edition only use a small portion of the XFB copy for output.

Rendering without the GPU

Rendering without the GPU is an odd case, because the XFB Copy Command from the GPU doesn't actually contain anything we need for the finished frame. Since we need the RAM copy of the XFB to be correct, it may seem like Dolphin would need Store XFB Copies to Texture and RAM enabled for this to work. But it doesn't!

Store XFB Copies to Texture Only doesn't copy the host GPU's output to emulated main memory for performance reasons. But the CPU can still write to main memory all it wants. If a game is using the emulated CPU to render entire frames directly to the screen, it will completely overwrite the magenta with the correct output! Even in Store XFB Copies to Texture Only, Dolphin checks main memory to make sure it hasn't been changed, and falls back to the RAM copy if it has. The most common case for this is when the CPU renders FMVs without the aid of the GPU.

XFB Virtual output nothing during videos.
You no longer need a special setting to get Mushroom Men: The Spore War to render properly.

Reading Old Frames

Many games take advantage of the old XFB copies residing in the XFB copy regions by integrating them into new frames for effects and transitions. Because Store XFB Copies to Texture Only fills the XFB copy regions with magenta, these effects will not work correctly.

Effects like these require the more accurate option: Store XFB Copies to Texture and RAM. This isn't quite the annoyance that using XFB Real would represent to users, as you're no longer locked to native resolution.

Store XFB Copies to Texture Only doesn't have the image in memory to read from and gets predefined empty values.
Store XFB Copies to Texture and RAM has the old frame in data, and the game is able to integrate it fine.
Use Store XFB Copies to Texture Only to unlock a top tier clone character: Magenta Box!
Store XFB Copies to Texture and RAM gives correct output.

While Store XFB Copies to Texture and RAM still allows for rendering at any resolution, it still is quite a bit slower than Store XFB Copies to Texture Only. As such, even in these two titles that we know use older frames, we do not enforce this setting by default.

Partial XFB Copies

Two particular games actually manage to hit an edge case in Store XFB Copies to Texture. Chicken Little: Ace in Action and Super Mario Strikers have a bit of an awkward behavior to emulate. They both do XFB copies from the GPU, then use the CPU to overwrite part of the XFB, but not the whole thing.


As shown above, both options fail the hash check and use the frame data from Main Memory, as they should. However with Store XFB Copies to Texture Only, the host GPU cannot copy the frame into emulated main memory, so the CPU is writing on top of magenta.

XFB Virtual used only the XFB copies in its buffer, and didn't allow for or recognize CPU edits.
XFB to Texture Only allows the CPU edit and switches to main memory, but it doesn't have the frame from the GPU.
Store XFB Copies to Texture and RAM correctly combines GPU and CPU contributions.

While it may seem easy to simply add a hack for this case, it would result in much messier code. Considering that usefulness of this hack could be invalidated by another feature down the line, we decided against attempting a fix. The current handling in Store XFB Copies to Texture works for every case except this one, and in this case, both games work relatively well. The loading screens in Super Mario Strikers aren't essential to the game, and Chicken Little: Ace in Action displays FMVs correctly, albeit with a magenta bar in the areas it does not write the FMV.

Pace Yourself

Having the game run smoothly is extremely important for the user experience. In the case of one of our dedicated forum users, they weren't happy with how The Legend of Zelda: Twilight Princess ran in Dolphin. At two particular points in the opening scenes, Dolphin would seemingly stutter without actually slowing down. Normally, something like this would be brushed off as a computer issue, or perhaps a game bug, but this user kept investigating and playing with the issue, even verifying that it didn't happen on console along the way.

What does this have to do with Hybrid XFB? Everything! As we said before, XFB emulation is the last stage before a frame is sent to the screen. Alas, when they finally tried XFB Real, their issue was resolved, but at the cost of high resolution and performance. It made sense that this didn't work with XFB Disabled, as it was shipping XFB copies as soon as they were rendered, but this is yet another issue where XFB Virtual should have worked. Unfortunately, it didn't solve the issue due to other timing issues on that path.

Thankfully, that mess is a thing of the past! Regardless of which setting you pick, Hybrid XFB will follow the pipeline correctly and output the frames the game actually wants to output.

Another case of strange frame pacing can be found in Xenoblade Chronicles, where if you load up the game with XFB Disabled, you'll be waiting quite a while for the initial loading screen to pass. This isn't because Dolphin is struggling to emulate the loading screen, rather the game is outputting XFB copies as fast as it can during this initial load. Vsync locks the framerate to the monitor's refresh rate in a case where Dolphin is trying to render at a much higher framerate, making the problem even worse.

XFB Disabled plus Vsync was a little slower than it should be.
Click/tap to start.

Most of these XFB copies aren't actually sent out to the screen, so Hybrid XFB doesn't suffer from the issue regardless. This also applies to the other titles mentioned earlier where Dolphin was displaying XFB copies that weren't meant to see the screen!

The Last Stand of XFB Disabled

Sometimes hacks become loved due to some of the beneficial side-effects that they have. One of the benefits, in developers eyes, of Hybrid XFB was finally dropping the nasty XFB Disabled hack. But, to a lot of users, XFB Disabled was more than just a hack - it was a huge feature that provided a benefit over playing on console.

Faster than the Speed of Light

Input latency is a huge issue when it comes to emulators, often leading enthusiasts to claim that emulators "don't feel right". This is simply because emulators have a a lot of work to do! In a PC game or the original hardware, games are running in their home environment, and are able to simply run. Emulators have to take what the game is doing and turn it into something a modern computer can run.

With USB latency, bluetooth stacks, desktop compositing, graphics APIs, and the many other layers that add latency, emulators have an uphill battle trying to match the low latency of an older console. In order to make games feel right, Dolphin has a ton of features that can help limit latency and make input feel more like it is on console. These include, but are not limited to, exclusive fullscreen, which bypasses operating system compositors alongside LibUSB passthrough of GameCube controllers, Wii Remotes, and other Wii USB devices which allows controllers to directly communicate with Dolphin to reduce latency on that end.

Using all of these tricks together allows Dolphin to match a GameCube and Wii in latency when paired with a low latency monitor.

Then there's XFB Disabled, which lowered latency in a completely different way. By grabbing XFB copies before a game was ready to display them, you could essentially get lower latency than console for the games that don't require any special XFB emulation. While subconsole latency may sound like nothing more than an interesting quirk, it's actually become a very integral feature for a substantial part of Dolphin's userbase.

Netplay and the Everlasting Battle Against All Latency

Whether you play netplay against friends or competitively using matchmaking sites, your opponent isn't always your greatest adversary. That honor usually goes to your internet connection. Dolphin compensates for internet latency on netplay with an input buffer. When you hit a button, Dolphin won't immediately send that input to the game. Instead, it holds it in a buffer while Dolphin sends that input over to your partners and your partners send their input over to you. This allows all Dolphin instances to execute the same inputs on the same frame, which keeps them in sync. If Dolphin runs out of buffer (meaning an input hasn't arrived yet) you'll see a stutter or pause while it waits for that input to arrive.

Hundreds of people play Melee, Brawl, or one of their hacks on netplay every day.

XFB Disabled allows for users to make up for up to 2 frames of input latency depending on how the game outputs frames. Removing the ability to display frames early in those games would be a huge blow to competitive netplay. While they could rely on older versions and forks, that represents a maintenance nightmare for them and loses the project valuable testing. Thankfully, a new option could be added that brings back the latency of XFB Disabled without having to rely on a separate codepath.

Immediately Present XFB

One of the reasons that we wanted to renovate the old XFB code is that it was fairly ugly and hard to work with. All three options kind of did their own thing, which made it very easy to accidentally break one of the paths that relied on some hack or strange behavior in Dolphin. After working so hard to drop XFB Disabled, it may sound weird that we were willing to implement a hack to replace it, but there's a key difference between the old hack and the new one: Immediately Present XFB only costs two lines of code. All it does is skip the console's natural buffering, and most of Video Interface emulation. The rest of the pipeline is still intact!

By keeping most of the pipeline intact, not only is the code cleaner, but it has better compatibility too! In Immediately Present XFB to Texture and RAM, frames are still being sent to the GC/Wii Main Memory, so games that read the old frames (e.g. Brawl and Twin Snakes) will still work!

Unfortunately, all the other benefits of using Store XFB Copies to Texture and RAM are gone. Most of the cases described above will break much like they did on XFB Disabled when Immediately Present XFB is enabled. For the most part that doesn't matter - this is a feature designed to reduce latency at all costs.

Rock Band 3's test places "0" at actual console latency on a CRT. Negative latency means that Dolphin is lower latency than a GameCube/Wii hooked up to CRT - it isn't displaying what's happening before it happens!

Result Breakdown

All of these latency results are extremely positive. Even without latency reducing hacks, Dolphin's latency is fantastic for an emulator. Anything within one frame of console when taking into account having to emulate a CPU, GPU, handle inputs and everything else just seems crazy.

While it may sound enticing to simply turn on Immediately Present XFB to get even lower latency, we do not recommend this outside of extreme cases where latency would matter, such as competitive netplay. Many games use the frame-out process for pacing, so, you'll very often get a smoother, more enjoyable experience by leaving Immediately Present XFB off. Super Smash Bros. Melee, Super Smash Bros. Brawl, and other latency intensive competitive titles that do not require special XFB emulation have Immediately Present XFB enabled by default for convenience.

Note: There's a pretty obvious anomaly on the graph that we can't exactly explain. Dolphin's Store XFB Copies to Texture and RAM combined with Immediately Present XFB results in extremely low latency in Rockband 3's test. While we've managed to reproduce the test results consistently across several machines, we actually have no idea why we're getting these results. It could be a malfunction in Rockband 3's test or maybe there is some real reason it's lower latency. We'll need to investigate this further in the future.

Moving Forward

Just as Hybrid XFB simplified things within the codebase, in the end, the biggest benefit will be simplifying things for the userbase. It takes away having to make a choice between correct output and high resolution output while simultaneously making more games work without the stark CPU power that XFB Real previously required.

While there are still a few small tasks to button up, Hybrid XFB is probably the last implementation that Dolphin will need for XFB emulation.

You can continue the discussion in the forum thread of this article.

Next entry

Previous entry

Similar entries