If at first you don't succeed, try something similar
I've been spending some time during the quarantine of 2020 working on various small Sega Genesis technical demos. I'm specifically noting 2020 for the sake of future readers who might also be living in a reality show nightmare world. I have low expectations and am not confident we will learn anything from this experience.
I'm a bad learner myself since I haven't made much progress in this time period. I started with a goal of creating an 8-player quiz-type game demo. I own two Team Players and a dozen controllers so it seemed like a good use of them. The Team Player is kind of a goofy device to program for. Sega must have given developers some sample code because trying brute-force to work with it is a bad idea. I got up to the dreaded point of "works on an emulator but not real hardware" before deciding to try something else.
For reasons unknown to me I thought trying the 4 Way Play would possibly be easier. Since it uses both controller ports I naively assumed it must map controller 1&2 to one port and 3&4 to the other. Then it would be something easy like polling each port 3-6 times like you would a 6-button controller.
That sounds logical and easy. I'm sure someone has figured this out already and documented it:
WHY!? I feel so sorry for anyone else searching for this that lands on my site.
So yeah. I didn't find anything. I started by writing a diagnostic rom that tried various polling combinations. Here's one of the lovely screenshots from it:
You don't want to see the source code to this. It helped me track down some code that kind of worked which I could refine into code that actually worked. To my surprise it even works on real hardware. Let's get to it already...
Minor note: I do not own a 4 Way Play but tested this code using a Team Player in 4 Way Play mode. Since the 4 Way Play does not fit every model of Sega Genesis I sort of assume the Team Player is what most would use for 4 Way Play games anyway.
I'm going to start with the bit of code that I'm least sure about. I ran into some issues where the 4 Way Play wasn't sending back data until it was initialized. On a typical controller sending #$40 to the control port around startup does the trick, the 4 Way Play seems to need a little more fidgeting. I am not sure if all these steps are necessary, once I had something working I stopped messing with it. There are other combinations that seemed to work too. Someone better at this can figure out something better I'm sure.
Then during vblank let's cycle through each controller. What I found is that all 4 controllers are read through the controller 1 port. The controller 2 port is used to tell the 4 Way Play which controller to pipe through the controller 1 port. This is done by sending #$[0-3]C to the control 2 data port. This is a neat approach. You could in theory support as many controllers as you have time to read during vblank this way.
; these values are totally arbitrary
bsr.w; Read4Way ; read controllers
; point a0 and a1 to controllers
; read controller 1
; read controller 2
; read controller 3
; read controller 4
The part to actually read a controller isn't different than the standard 3-button read except for starting with TH low (TH high & low is just barely explained in my 6 button tutorial). With a little more experimenting I'd probably find you could start with TH high just as well. This is another case of quitting once I found something that worked.
move.b #$00,(a0) ; set TH low
nop ; bus synchronization
nop ; bus synchronization
move.b (a0),d0 ; d0 has 00SA0000
move.b #$40,(a0) ; set TH high
lsl.b #$02,d0 ; d0 is now SA000000
andi.b #%11000000,d0 ; clear lowest 6 bits
move.b (a0),d1 ; d1 now has 00BCRLDU
andi.b #%00111111,d1 ; clear 2 highest bits
or.b d1,d0 ; merge the two reads into d0
; d0 now has SABCRLDU
not.b d0 ; flip bits so 0 means not pressed and 1 means pressed
Then I got lazy and decided not to write a real demo rom. I settled for a very ugly text screen that dumps the raw data.
This demonstrates that the code works at the very least. It's otherwise not a very exciting game.
Now, there is one good thing and one bad thing I ran into.
Bad: I can't figure out any way to poll for X/Y/Z/Mode buttons. As far as I can tell they are not supported with the the 4 Way Play. They are supported with the Team Player though. Based on my hazy memories of working at a game store in the 90s, the 4 Way Play slightly pre-dated 6 button Genesis controllers. Also the 4 Way Play was an EA accessory for (mostly) sports games. I don't think they were especially concerned about 6 button support if it even was a thing yet.
Good: This works fine with Atari 2600 controllers connected. The fire button is mapped to both A & B buttons which makes sense since they are read through the same pin. The d-pad is working exactly as expected. This of course is how an Atari 2600 controller works in a normal Genesis control port. I was slightly worried it might break with a 4 Way Play. This is great news for anyone considering a 4 player quiz-type game with minimal controls. Or a 4 player Snafu clone, that would be nice.
It's doubtful I'll post this on Github unless I use it in a larger demo.
I'd like to go back and try to get a Team Player demo working. I'd also sort of like to quit stalling on building another full demo game. So either one of these or something completely different I guess.