Raw files aren't playing, or are playing incorrectly - Oboe (Android-ndk)

Issue

I'm attempting to Play a Raw (int16 PCM) encoded audio file in my android application. I've been following and reading through the Oboe documentation/samples to try to get one of my own audio files to play.

The audio file I need to play is roughly 6kb, or 1592 frames (stereo).

Either no sound plays, or sound/jitter plays on startup (with varying output - see bellow)

Troubleshooting

update I have switched to floats for buffer queuing, instead of keeping everything to int16_t (and converting back to int16_t when done), although now I'm back to no sound.

The audio seems to be either not playing, or playing on startup (which is wrong). The sound should play after I press 'start'.

• When the app was implemented with int16_t only, the premature sound was relative to how big the buffer size was. If the buffer size is smaller than the audio file, the sound is very fast and clipped (more drone-like at lower buffer sizes). Bigger than the Raw audio size it seems like it plays on a loop and gets quieter at higher buffer sizes. The sound would also get "softer" when the start button is pressed. I'm not even entirely sure this means the raw audio was playing, it could just be random nonsense jitters from Android.

• When filling the buffers with floats, and converting to int16_t afterwards, no audio is played.

(I have tried running systrace, but I honestly don't know what I'm looking for)

• The stream opens fine.
• The buffer size fails to be ajusted in createPlaybackStream() (although somehow it still sets it to twice the burst size)
• The stream starts fine.
• The Raw resources are being loaded fine.

Implementation

What I am currently trying in the builder:

• Setting the callback to this, or onAudioReady()
• Setting the performance mode to LowLatency
• Setting the sharing mode to Exclusive
• Setting the buffer capacity to (anything bigger than my audio file frame count)
• Setting the burst size (frames per call back) to (anything equal to or lower than the buffer capacity / 2)

I am using the Player class and the AAssetManager class from the Rhythm Game sample here: https://github.com/google/oboe/blob/master/samples/RhythmGame. I am using these classes to load my resources and play the sound. Player.renderAudio writes the audio data to the output buffer.

Here are the relevant methods from my audio engine:

void AudioEngine::createPlaybackStream() {

//    // Load the RAW PCM data files into memory
    std::shared_ptr<AAssetDataSource> soundSource(AAssetDataSource::newFromAssetManager(assetManager, "sound.raw", ChannelCount::Mono));

    if (soundSource == nullptr) {
        LOGE("Could not load source data for sound");
        return;
    }

    sound = std::make_shared<Player>(soundSource);

    AudioStreamBuilder builder;

    builder.setCallback(this);
    builder.setPerformanceMode(PerformanceMode::LowLatency);
    builder.setSharingMode(SharingMode::Exclusive);
    builder.setChannelCount(mChannelCount);


    Result result = builder.openStream(&stream);

    if (result == Result::OK && stream != nullptr) {

        mSampleRate = stream->getSampleRate();
        mFramesPerBurst = stream->getFramesPerBurst();

        int channelCount = stream->getChannelCount();
        if (channelCount != mChannelCount) {
            LOGW("Requested %d channels but received %d", mChannelCount, channelCount);
        }

        // Set the buffer size to (burst size * 2) - this will give us the minimum possible latency while minimizing underruns
        stream->setBufferSizeInFrames(mFramesPerBurst * 2);
        if (setBufferSizeResult != Result::OK) {
            LOGW("Failed to set buffer size.  Error: %s", convertToText(setBufferSizeResult.error()));
        }

        // Start the stream - the dataCallback function will start being called

        result = stream->requestStart();
        if (result != Result::OK) {
            LOGE("Error starting stream. %s", convertToText(result));
        }

    } else {
        LOGE("Failed to create stream. Error: %s", convertToText(result));
    }
}

DataCallbackResult AudioEngine::onAudioReady(AudioStream *audioStream, void *audioData, int32_t numFrames) {
    int16_t *outputBuffer = static_cast<int16_t *>(audioData);

    sound->renderAudio(outputBuffer, numFrames);
    return DataCallbackResult::Continue;
}

// When the 'start' button is pressed, it calls this method with true
// There should be no sound on app start-up until this button is pressed
// Sound stops when 'stop' is pressed

setPlaying(bool isPlaying) {
    sound->setPlaying(isPlaying);
}

Solution

Setting the buffer capacity to (anything bigger than my audio file frame count)

You don't need to set the buffer capacity. This will be set automatically at a reasonable level for you. Typically ~3000 frames. Note that buffer capacity is different from buffer size which defaults to 2*framesPerBurst.

Setting the burst size (frames per call back) to (anything equal to or lower than the buffer capacity / 2)

Again, don't do this. onAudioReady will be called every time the stream requires more audio data and numFrames indicates how many frames you should supply. If you override this value with a value which isn't an exact ratio of the audio device's native burst size (typical values are 128, 192 and 240 frames depending on underlying hardware) then you may get audio glitches.

I have switched to floats for buffer queuing

The format which you need to supply data in is determined by the audio stream and it is only known after the stream has been opened. You can get it by calling stream->getFormat().

In the RhythmGame sample (at least the version you're referring to) here's how the formats work:

1. Source file is converted from 16-bit to float inside AAssetDataSource::newFromAssetManager (floats are the preferred format for any kind of signal processing)
2. If the stream format is 16-bit then convert it back inside onAudioReady

1592 frames (stereo).

You said that your source was stereo but you're specifying it as mono here:

std::shared_ptr soundSource(AAssetDataSource::newFromAssetManager(assetManager, "sound.raw", ChannelCount::Mono));

Without doubt that will cause audio problems because the AAssetDataSource will have a value for numFrames which is double the correct value. This will cause audio glitches because half the time you'll be playing random parts of system memory.

Answered By - donturner