One dev's quest to support every controller you can plug into a USB port

Controllers physically embody the "toy" aspect of video games.

While Sony and Microsoft have set a de facto standard with their console gamepads, there are guitars and drumsets and bongos and dancepads and many other toy-like controllers, designed for niche genres or one-off gimmick games.

Many devs love these oddball controllers, even as most collect dust once the owner loses interest in the specific game tailored to their physical specialties; specialized controllers aren't good for much else.

Nevertheless, the devs of Jelly Team—composed of three Northeastern University alumni: programmer and designer Mark Trueblood, artist and designer Oskar Strom, and programmer Liam Fratturo—set out to make use of as many of those oddball controllers as possible for their arena battle party game, Super Slime Arena.

Recently, they took time off from working on their upcoming training and single-player update to speak about supporting obscure Dreamcast fishing reel controllers in modern games.

Questions and answers have been edited for brevity and clarity.

One of the unique selling points of Super Slime Arena is support for any controller with a USB plug, including oddballs like fishing controllers and conga drums. How did you go about making that possible?

Mark Trueblood: It started from simplifying the inputs as much as possible. The core idea is a "minimalist fighting game", so at the core it's just two buttons and a d-pad (or joystick). We also have a "start" and "swap" button for pausing and changing characters, but those are secondary inputs that aren't typically needed during gameplay. Imagine an NES controller.

After that, the major challenge is just figuring out which button on the controller should be "attack", "jump", "left", "right", etc. And then we did that over and over again for literally every controller we could find. That stuff was in the game from very early.

Later on, I decided I really wanted the game to be able to auto-detect what controller you've plugged in, and automatically load the correct set of inputs for that controller. So, it can detect if you plug in an Xbox vs PlayStation vs Nintendo Switch, etc. controller and set up everything properly without any manual configuration. This was a bit harder and required a custom library (because most game engines don't even support this functionality).

Liam Fratturo: We knew we'd never be able to do custom button prompts for every controller we supported, so we leaned hard in the other direction and made the game tell you what 'action' you need to do instead of what button you need to press. [For example], on the end-of-round screen, we tell players to hold 'Jump' or 'Attack' to start a rematch, go back to mode select, etc.

In terms of menu design, every controller we support is guaranteed to have at least two axes (or four buttons) that are mapped to up/down/left/right, so navigating the menus should be possible no matter what you're using.

When playing locally, every player has control during the menus, and they use cursors to pick the game mode, characters, stages, etc. Even if one player is using a controller that's unwieldy for menu navigation, since everyone has control, the other players can get through the menus for them. When we demo the game Oskar likes to use the guitar controller, which has up/down mapped to a tiny d-pad at a weird angle, so I usually usually handle the menu stuff when he's using that.

Oskar Strom: One thing that I really like that we did also is having custom icons for each controller. I think I drew around 100 different ones spanning lots and lots of different systems that show up when you're on one of the early screens. Great way to identify who is who even before players are decided. It's also just fun to move around the little controllers!

Liam Fratturo: We can recognize the type of USB adapter, so we use the standard controller [icon] for that adapter: i.e. controllers connected to the Gamecube adapter will default to a standard Gamecube controller [icon], but you can go into config mode and change that to the DK Bongos if that's what you have connected. Changing that changes both the control bindings and the icon that's displayed.

How did you develop the library to recognize and support controllers, based on their USB serial number?

Liam Fratturo: I didn't create the library from scratch, I just had to track down an open-source library that actually supported all the features I needed, and worked on Windows, Mac, and Linux. That was surprisingly tough.

There's a lot of in-depth input libraries for Windows only, and there's a lot of shallow input libraries for multi-platform. But there's not many in-depth multi-platform libraries. Then since the library was written in C++ and our game is written in Haxe, I had to build a little interface that lets the Haxe code talk to the C++ code. This wasn't too difficult because Haxe is very flexible, and using non-Haxe libraries is actually a pretty common thing in the Haxe community. Haxe is relatively small as a programming language, so almost everybody who uses Haxe is doing that type of thing to interface with tools written in other languages.

It's also pretty nice working with a lot of open-source tech. Haxe is open-source, Haxeflixel (the engine we used) is open-source, Libstem Gamepad (the input library) is open-source, and Snowkit (the tool for using C++ libraries in haxe) is open-source too. It's a very welcoming community even if it's a bit more "DIY" than using a big mainstream engine like Unity.

Then we just had to try to make sure that we've got custom configurations for all of the popular controllers, and added a decent UI that will let you create your own button configs, and automatically load those configs when you plug them in.

There's still a few sticking points, though. For stuff like the Dreamcast Fishing Rod or DK Bongo controller, you need a cable adapter (Dreamcast-to-USB or GameCube-to-USB respectively). Unfortunately, those cable adapters don't give you any information beyond "a GameCube controller is plugged in". You can't tell whether it's a normal controller or a fishing rod or a dance pad or whatever. So the user still has to configure that stuff manually every time the game starts. Not really any way around that, unfortunately.

I've kinda given this whole lecture a few times before. (laughs) A lot of other devs are curious about it.

I talked to one of the people at Unity who works on their input system, [Corey Johnson]. I think we [originally] met at PAX West, but it could have been some other event. He was basically asking about our experience designing our input system, and asked some questions about what a developer would want out of Unity's input system. All three of us have some experience making stuff in Unity, even though Super Slime Arena isn't made in Unity.

How could Unity improve their input system?

Mark Trueblood: The biggest thing was just giving you the USB ID of a controller so that you can figure out what's plugged-in. Some game engines will tell you "it's an Xbox controller" or "it's a PlayStation controller" or "other", which is useful if you just want to display the correct "AB" vs "XO" icons, but not good if you want to support a broader range of controllers.

I know Unity has been updating a bunch of their stuff recently, so this might be something that they've fixed by now. And I think that you can find 3rd-party input systems on the Asset store to solve some of the issues with Unity's input.

I think a lot of game devs like Unity, but also have little wishlists of features or changes for it, so it was cool to directly talk to someone involved in making some of those decisions. Also, it's a nice feeling when you've built something interesting enough that a giant company wants to talk to you and learn from you.

So how did you folks arrive at the idea that you could support all these different controllers?

Oskar Strom: One of the big principles we wanted behind the game was accessibility, which mainly featured into our simple control scheme, like Mark mentioned before, but another part of it was with the controllers. We liked the idea of being able to use anything that you h