ffmpeg convert AV_SAMPLE_FMT_S16 to AV_SAMPLE_FMT_FLTP in c++

Issue

I want to encode and decode the sound from my Android app to the Opus format using FFmpeg 4.2.2.

The problem is that my Android app provides a raw PCM sound in AV_SAMPLE_FMT_S16 format, but the FFmpeg opus encoder requires only AV_SAMPLE_FMT_FLTP. So, I decided to resample the sound using FFmpeg swr_convert() function but it crashes with SIGSEGV error and I can't understand why.

My code looks like this:

swrContext = swr_alloc();

av_opt_set_int(swrContext, "in_channel_layout", (int64_t) codecContext->channel_layouts, 0);
av_opt_set_int(swrContext, "out_channel_layout", (int64_t) codecContext->channel_layouts,  0);
av_opt_set_int(swrContext, "in_sample_rate", 8000, 0);
av_opt_set_int(swrContext, "out_sample_rate", 48000, 0);

av_opt_set_sample_fmt(swrContext, "in_sample_fmt", AV_SAMPLE_FMT_S16, 0);
av_opt_set_sample_fmt(swrContext, "out_sample_fmt", AV_SAMPLE_FMT_FLTP,  0);

swr_init(swrContext);

memcpy(frame->data[0], data, dataSize); 

uint8_t *outBuffer = (uint8_t *) malloc(sizeof(uint8_t) * frame->nb_samples);

swr_convert(swrContext, &outBuffer, frame->nb_samples, (const uint8_t **)frame->data, frame->nb_samples);

I am new to C++ so sorry for some mistakes if I made them.

Solution

Here are a few things that you need to take care of:

Make sure that the frame->data[0] contains enough memory (at least equal to dataSize) for the data to be copied from in this call:

memcpy( frame->data[0], data, dataSize );

Also, you need to set frame->nb_samples accordingly. Maybe, you already have but there's no indication in the code that you have posted.

You also need to allocate samples buffer using av_samples_alloc and free it including all the other allocated memory after use so that there won't be any memory leaks.

Here's an example (add the value for out_num_channels):

const int in_sample_rate = 8000;
const int out_sample_rate = 48000;

swrContext = swr_alloc();
av_opt_set_int(swrContext, "in_channel_layout", (int64_t) codecContext->channel_layouts, 0);
av_opt_set_int(swrContext, "out_channel_layout", (int64_t) codecContext->channel_layouts,  0);
av_opt_set_int(swrContext, "in_sample_rate", in_sample_rate, 0);
av_opt_set_int(swrContext, "out_sample_rate", out_sample_rate, 0);
av_opt_set_sample_fmt(swrContext, "in_sample_fmt", AV_SAMPLE_FMT_S16, 0);
av_opt_set_sample_fmt(swrContext, "out_sample_fmt", AV_SAMPLE_FMT_FLTP,  0);
swr_init(swrContext);

memcpy(frame->data[0], data, dataSize); // frame->nb_samples ???

const int out_num_samples = av_rescale_rnd(swr_get_delay(swrContext, in_sample_rate) + frame->nb_samples, out_sample_rate, in_sample_rate, AV_ROUND_UP);

uint8_t* out_samples = NULL;
av_samples_alloc(&out_samples, NULL, out_num_channels, out_num_samples, AV_SAMPLE_FMT_FLTP, 0);

out_num_samples = swr_convert(swrContext, &out_samples, out_num_samples, &frame->data[0], frame->nb_samples);

av_freep(&out_samples);    // free after use
swr_free(&swrContext);     // free after use

You might want to tinker out_sample_rate according to your requirements. I'd suggest converting your file on the command line using ffmpeg command and use the parameters that worked in your code later. The code iterations would be less and you'd have more flexibility working on the command line. See this and this thread on using command line ffmpeg utility.

Hope this helps!

Answered By - Azeem