歡迎加入 ijkplay 播放器學(xué)習(xí)群：490805051
本文討論ijkplay播放直播流延時(shí)現(xiàn)象產(chǎn)生的原因和解決方法。

原因

1，網(wǎng)絡(luò)抖動(dòng)

a)，推流端因?yàn)榫W(wǎng)絡(luò)變差，buffer queue 會(huì)越來(lái)越大，等網(wǎng)絡(luò)恢復(fù)正常時(shí)，再推流出去。當(dāng)然，推流端大家估計(jì)有不同的控制策略。
b)，CDN 源節(jié)點(diǎn)到邊緣節(jié)點(diǎn)轉(zhuǎn)發(fā)網(wǎng)絡(luò)抖動(dòng)
c)，播放器端拉流因?yàn)榫W(wǎng)絡(luò)變差，讀取不到數(shù)據(jù)，等網(wǎng)絡(luò)恢復(fù)正常時(shí)，會(huì)把之前的數(shù)據(jù)讀回來(lái)（CDN服務(wù)器緩存多少秒？），導(dǎo)致buffer queue變大.
設(shè)置player.shouldShowHudView=YES; 可以實(shí)時(shí)觀察音視頻緩沖區(qū)的大小，對(duì)于is中的videoq和audioq，存的是未解碼前數(shù)據(jù)包。

2，待補(bǔ)充

解決方法

推薦使用 cutv web播放端 播放直播流，或者使用 ffplay -fflags nobuffer -i 播放地址。
針對(duì)播放器端因網(wǎng)絡(luò)抖動(dòng)引起緩沖區(qū)變大，該怎么處理呢？

1，倍速播放

ijkplay雖然提供倍速播放的接口，但是需要Android 6.0以上，我這邊測(cè)試過(guò)效果并不好，沒(méi)有使用，但是這個(gè)倍速播放方法用戶體驗(yàn)會(huì)更好。
倍速播放，同時(shí)也需要解碼性能跟得上，當(dāng)然也可以只解碼I幀。

2，丟包

丟解碼前數(shù)據(jù)包還是解碼后的數(shù)據(jù)幀呢？
為了簡(jiǎn)單處理，這邊采用丟解碼前的數(shù)據(jù)包，策略如下：
a)，有音頻流和視頻流，或者只有音頻流情況下，當(dāng)audioq達(dá)到一定的duration，就丟掉前面一部分?jǐn)?shù)據(jù)包，因?yàn)槟J(rèn)是AV_SYNC_AUDIO_MASTER，視頻會(huì)追上來(lái)。
b)，只有視頻流情況，當(dāng)videoq達(dá)到一定的duration，就丟掉前面一部分?jǐn)?shù)據(jù)包。

下面是代碼，ijkplay版本是 0.5.1
ff_ffplay_def.h中添加最大緩存時(shí)長(zhǎng)

typedef struct VideoState {
    ...
    // Add by ljsdaya
    // for low delay time with live play(realtime), control videoq/audioq duration < max_cached_duration
    // realtime set to 0, max_cached_duration = 0 means is playback
    int max_cached_duration;
} VideoState;

ff_ffplay.c 添加控制緩存隊(duì)列的方法

static void drop_queue_until_pts(PacketQueue *q, int64_t drop_to_pts) {
    MyAVPacketList *pkt1 = NULL;
    int del_nb_packets = 0;
    for (;;) {
        pkt1 = q->first_pkt;
        if (!pkt1) {
            break;
        }
        // video need key frame? 這里如果不判斷是否是關(guān)鍵幀會(huì)導(dǎo)致視頻畫面花屏。但是這樣會(huì)導(dǎo)致全部清空的可能也會(huì)出現(xiàn)花屏
        // 所以這里推流端設(shè)置好 GOP 的大小，如果 max_cached_duration > 2 * GOP，可以盡可能規(guī)避全部清空
        // 也可以在調(diào)用control_queue_duration之前判斷新進(jìn)來(lái)的視頻pkt是否是關(guān)鍵幀，這樣即使全部清空了也不會(huì)花屏
        if ((pkt1->pkt.flags & AV_PKT_FLAG_KEY) && pkt1->pkt.pts >= drop_to_pts) {
//        if (pkt1->pkt.pts >= drop_to_pts) {
            break;
        }
        q->first_pkt = pkt1->next;
        if (!q->first_pkt)
            q->last_pkt = NULL;
        q->nb_packets--;
        ++del_nb_packets;
        q->size -= pkt1->pkt.size + sizeof(*pkt1);
        if (pkt1->pkt.duration > 0)
            q->duration -= pkt1->pkt.duration;
        av_free_packet(&pkt1->pkt);
#ifdef FFP_MERGE
        av_free(pkt1);
#else
        pkt1->next = q->recycle_pkt;
        q->recycle_pkt = pkt1;
#endif
    }
    av_log(NULL, AV_LOG_INFO, "233 del_nb_packets = %d.\n", del_nb_packets);
}

static void control_video_queue_duration(FFPlayer *ffp, VideoState *is) {
    int time_base_valid = 0;
    int64_t cached_duration = -1;
    int nb_packets = 0;
    int64_t duration = 0;
    int64_t drop_to_pts = 0;
    
    //Lock
    SDL_LockMutex(is->videoq.mutex);
    
    time_base_valid = is->video_st->time_base.den > 0 && is->video_st->time_base.num > 0;
    nb_packets = is->videoq.nb_packets;
    
    // TOFIX: if time_base_valid false, calc duration with nb_packets and framerate
    // 為什么不用 videoq.duration？因?yàn)橛龅竭^(guò)videoq.duration 一直為0，audioq也一樣
    if (time_base_valid) {
        if (is->videoq.first_pkt && is->videoq.last_pkt) {
            duration = is->videoq.last_pkt->pkt.pts - is->videoq.first_pkt->pkt.pts;
            cached_duration = duration * av_q2d(is->video_st->time_base) * 1000;
        }
    }
    
    if (cached_duration > is->max_cached_duration) {
        // drop
        av_log(NULL, AV_LOG_INFO, "233 video cached_duration = %lld, nb_packets = %d.\n", cached_duration, nb_packets);
        drop_to_pts = is->videoq.last_pkt->pkt.pts - (duration / 2);  // 這里刪掉一半，你也可以自己修改，依據(jù)設(shè)置進(jìn)來(lái)的max_cached_duration大小
        drop_queue_until_pts(&is->videoq, drop_to_pts);
    }
    
    //Unlock
    SDL_UnlockMutex(is->videoq.mutex);
}

static void control_audio_queue_duration(FFPlayer *ffp, VideoState *is) {
    int time_base_valid = 0;
    int64_t cached_duration = -1;
    int nb_packets = 0;
    int64_t duration = 0;
    int64_t drop_to_pts = 0;
    
    //Lock
    SDL_LockMutex(is->audioq.mutex);
    
    time_base_valid = is->audio_st->time_base.den > 0 && is->audio_st->time_base.num > 0;
    nb_packets = is->audioq.nb_packets;
    
    // TOFIX: if time_base_valid false, calc duration with nb_packets and samplerate
    if (time_base_valid) {
        if (is->audioq.first_pkt && is->audioq.last_pkt) {
            duration = is->audioq.last_pkt->pkt.pts - is->audioq.first_pkt->pkt.pts;
            cached_duration = duration * av_q2d(is->audio_st->time_base) * 1000;
        }
    }
    
    if (cached_duration > is->max_cached_duration) {
        // drop
        av_log(NULL, AV_LOG_INFO, "233 audio cached_duration = %lld, nb_packets = %d.\n", cached_duration, nb_packets);
        drop_to_pts = is->audioq.last_pkt->pkt.pts - (duration / 2);
        drop_queue_until_pts(&is->audioq, drop_to_pts);
    }
    
    //Unlock
    SDL_UnlockMutex(is->audioq.mutex);
}

static void control_queue_duration(FFPlayer *ffp, VideoState *is) {
    if (is->max_cached_duration <= 0) {
        return;
    }
    
    if (is->audio_st) {
        return control_audio_queue_duration(ffp, is);
    }
    if (is->video_st) {
        return control_video_queue_duration(ffp, is);
    }
    
}

ff_ffplay.c read_thread 線程中，在每次 av_read_frame后去判斷緩存隊(duì)列有沒(méi)有達(dá)到最大時(shí)長(zhǎng)。這里需要把原來(lái)的realtime設(shè)置為0

...
// 把原來(lái)的realtime設(shè)置為0，并從外部設(shè)置獲取max_cached_duration的值
//    is->realtime = is_realtime(ic);
    is->realtime = 0;
    AVDictionaryEntry *e = av_dict_get(ffp->player_opts, "max_cached_duration", NULL, 0);
    if (e) {
        int max_cached_duration = atoi(e->value);
        if (max_cached_duration <= 0) {
            is->max_cached_duration = 0;
        } else {
            is->max_cached_duration = max_cached_duration;
        }
    } else {
        is->max_cached_duration = 0;
    }

    if (true || ffp->show_status)
        av_dump_format(ic, 0, is->filename, 0);
...
...
        // 每次讀取一個(gè)pkt，都去判斷處理
        // TODO:優(yōu)化，不用每次都調(diào)用
        if (is->max_cached_duration > 0) {
            control_queue_duration(ffp, is);
        }
        if (pkt->stream_index == is->audio_stream && pkt_in_play_range) {
            packet_queue_put(&is->audioq, pkt);
        } else if (pkt->stream_index == is->video_stream && pkt_in_play_range
                   && !(is->video_st && (is->video_st->disposition & AV_DISPOSITION_ATTACHED_PIC))) {
            packet_queue_put(&is->videoq, pkt);
#ifdef FFP_MERGE
        } else if (pkt->stream_index == is->subtitle_stream && pkt_in_play_range) {
            packet_queue_put(&is->subtitleq, pkt);
#endif
        } else {
            av_packet_unref(pkt);
        }
...
...

iOS端使用實(shí)例代碼

    IJKFFOptions *options = [IJKFFOptions optionsByDefault];
    // Set param
    [options setFormatOptionIntValue:1024 * 16 forKey:@"probsize"];
    [options setFormatOptionIntValue:50000 forKey:@"analyzeduration"];
    [options setPlayerOptionIntValue:0 forKey:@"videotoolbox"];
    [options setCodecOptionIntValue:IJK_AVDISCARD_DEFAULT forKey:@"skip_loop_filter"];
    [options setCodecOptionIntValue:IJK_AVDISCARD_DEFAULT forKey:@"skip_frame"];
    if (_isLive) {
        // Param for living
        [options setPlayerOptionIntValue:3000 forKey:@"max_cached_duration"];   // 最大緩存大小是3秒，可以依據(jù)自己的需求修改
        [options setPlayerOptionIntValue:1 forKey:@"infbuf"];  // 無(wú)限讀
        [options setPlayerOptionIntValue:0 forKey:@"packet-buffering"];  //  關(guān)閉播放器緩沖
    } else {
        // Param for playback
        [options setPlayerOptionIntValue:0 forKey:@"max_cached_duration"];
        [options setPlayerOptionIntValue:0 forKey:@"infbuf"];
        [options setPlayerOptionIntValue:1 forKey:@"packet-buffering"];
    }

Android端使用實(shí)例代碼

ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT, "probesize", 1024 * 16);
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_FORMAT, "analyzeduration", 50000);
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_CODEC, "skip_loop_filter", 0);
ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_CODEC, "skip_frame", 0);
if (mIsLive) {
    // Param for living
    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "max_cached_duration", 3000);
    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "infbuf", 1);
    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "packet-buffering", 0);
} else {
    // Param for playback
    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "max_cached_duration", 0);
    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "infbuf", 0);
    ijkMediaPlayer.setOption(IjkMediaPlayer.OPT_CATEGORY_PLAYER, "packet-buffering", 1);    
}

@不鳴則已 提到的問(wèn)題請(qǐng)參考 @Gongjia 的文章 ijkplayer丟幀的處理方案