An Arduous Endeavor (Part 5): Buzzes and Beeps

Sound generation is probably the aspect of emulation that I am least familiar with. As part of my undergrad, I wrote simulators of simple computer architectures and interacted directly with framebuffers to display 2D graphics. But when it came to sound stuff, my experience was strictly at a high level, buffering WAV/MP3 files and relying on some mixing libraries written by others.

Sound emulation in Libretro, though, appears to require passing through integers at the “sample” level. In other words, the raw amplitude values for whatever sample rate we’re running the emulator at.

Sound is just vibrations, and one way it can be represented digitally is as a whole bunch of measurements of the amplitude of a wave. “CD quality” audio takes 44,100 of these measurements every second, with each measurement represented as a 16-bit integer (well, technically two 16-bit integers for stereo audio, one for each channel of left and right).

The ArduBoy uses a piezo-electric buzzer controlled by 2 digital pins on port C, pins 6 and 7, which simplifies things a little bit – there is no nuance/smoothness to the audio output, there are just square waves. Additionally, it appears that the two pins are maybe just voltages that get added together, so instead of having four possible values, we have three: negative, zero, and positive.

simavr also makes it straightforward to have a callback function run every time those pin values change via avr_irq_register_notify:

void Arduous::init(uint8_t* boot, uint32_t bootBase, uint32_t bootSize) {
    // ...
    pinCallbackParamTs = {PinCallbackParamT{.self = this, .speakerPin = 0},
                          PinCallbackParamT{.self = this, .speakerPin = 1}};

    avr_irq_register_notify(avr_io_getirq(cpu, AVR_IOCTL_IOPORT_GETIRQ('C'), 6), Arduous::soundPinCallback,
                            &pinCallbackParamTs[0]);
    avr_irq_register_notify(avr_io_getirq(cpu, AVR_IOCTL_IOPORT_GETIRQ('C'), 7), Arduous::soundPinCallback,
                            &pinCallbackParamTs[1]);
}

Note also that soundPinCallback must be a static method because avr_irq_register_notify expects a plain old function pointer, which is why we pass a helper struct as the third parameter to avr_irq_register_notify, which will get passed in to the callback function:

void Arduous::soundPinCallback(struct avr_irq_t* irq, uint32_t value, void* param) {
    auto* pinCallbackParamT = static_cast<PinCallbackParamT*>(param);
    Arduous* self = pinCallbackParamT->self;
    self->extendAudioBuffer();
    self->speakerPins[pinCallbackParamT->speakerPin] = value & 0x1;
}

So this is at least a defined, small-ish problem. The trick here is figuring out how to turn those digital pins into the right number of PCM values for each frame. Luckily we’ve got some information that can help – the CPU cycle count of the AVR itself.

The AVR (CPU) is set to a fixed speed of 16,000,000 Hz, and it also keeps track of what its current cycle is at cpu->cycle. With some math, we can figure out how many audio samples have passed in between the times that the speaker pin values change, and then store that many copies of whatever the last speaker pin value was in a per-frame audio buffer. We need to make sure we also clean things up at the end of each video frame to fill out the rest of the buffer, as libretro expects a consistent number of samples per video frame. Here’s the code that actually extends the audio buffer (called when a pin changes and also at the end of the video frame):

void Arduous::extendAudioBuffer() {
    int endSampleIndex = 2 * std::min((cpu->cycle - frameStartCycle) / cyclesPerAudioSample,
                                      static_cast<uint64_t>(audioSamplesPerVideoFrame));
    int16_t currentSample = getCurrentSpeakerSample();
    for (uint i = audioBuffer.size(); i < endSampleIndex; i++) {
        audioBuffer.push_back(currentSample);
    }
}

getCurrentSpeakerSample is my attempt to figure out how the pins get translated to a PCM value. I think that the appropriate math is to subtract the pin values from each other, because one of these pins is connected to the positive lead of the speaker, and the other is connected to the negative. So if both pins are high or both pins are low, that would be a 0, otherwise we get a positive or negative value. I’m not feeling super confident about this interpretation, though.

int16_t Arduous::getCurrentSpeakerSample() {
    switch (speakerPins.to_ulong()) {
        case 0:
        case 3:
            return 0;
            break;
        case 1:
            return INT16_MAX;
            break;
        case 2:
            return INT16_MIN;
        default:
            throw std::runtime_error("Invalid speaker pin value");
    }
}

I also created a separate audio_buffer array in libretro.cpp to keep the Libretro portion separate from the Arduous class-based implementation, partly because I’m not sure what gets done with that audio buffer outside of the callback.

int16_t audio_buffer[TIMING_SAMPLE_RATE / TIMING_FPS * 2];
// ...
void update_audio() {
    memcpy(audio_buffer, arduous->getAudioBuffer().data(), TIMING_SAMPLE_RATE / TIMING_FPS * 2);
    audio_cb(audio_buffer, TIMING_SAMPLE_RATE / TIMING_FPS);
}

So, put it all together, and we get… something that is a little grating, but it’s recognizable as sound!

WARNING – VERY LOUD SQUARE WAVES, CHECK YOUR VOLUME

Note that there are some weird things that are unexpected – specifically, the sound doesn’t always stop. I’m not sure what’s causing with that, whether it’s an issue with the Arduboy compiled code or my emulator, but I’m guessing it’s an issue with my emulator. I’ll go bug hunting another day, though.

Series Navigation

<< An Arduous Endeavor (Part 4): Input HandlingAn Arduous Endeavor (Part 6): Save States and Rewind >>