Silence.Slow.Simple
首页 标签 分类 归档

BufferQueueCore 详解

2021-04-04 知其所以然

首先感谢何小龙大佬写的这篇十分优秀的博客。

背景

当我们提起 BufferQueue 的时候，一般都会想起下面的这张经典的图：

BufferQueue

但是呢，由于 Android S BLASTBufferQueue 的加入，这张图对于现在的 BufferQueue 实际上已经不适用了。我将在下篇文章详细说明 BLASTBufferQueue，这一篇先了解一下 BufferQueue 的核心结构——BufferQueueCore。

BufferQueueCore 结构详解

BufferQueueCore 的结构其实比较简单，因为它的最主要的功能就是管理 Graphic Buffer 申请/释放/状态维护，所以它的所有内容都是围绕这个核心功能，其中最核心的结构就是 mSlots 和 mQueue 了，下面分别说明：

mSlots

// frameworks/native/libs/gui/include/gui/BufferQueueCore.h
// mSlots is an array of buffer slots that must be mirrored on the producer
// side. This allows buffer ownership to be transferred between the producer
// and consumer without sending a GraphicBuffer over Binder. The entire
// array is initialized to NULL at construction time, and buffers are
// allocated for a slot when requestBuffer is called with that slot's index.
BufferQueueDefs::SlotsType mSlots;

// frameworks/native/libs/gui/include/gui/BufferQueueDefs.h
namespace BufferQueueDefs {
    typedef BufferSlot SlotsType[NUM_BUFFER_SLOTS];
}

// frameworks/native/libs/ui/include/ui/BufferQueueDefs.h
namespace BufferQueueDefs {
    // BufferQueue will keep track of at most this value of buffers.
    // Attempts at runtime to increase the number of buffers past this
    // will fail.
    static constexpr int NUM_BUFFER_SLOTS = 64;
}

mSlots 是BufferQueueCore 的核心，是一个大小为 NUM_BUFFER_SLOTS（一般值为 64）的 BufferSlot 数组。

BufferSlot 这个结构很重要，他的最重要的成员是 Graphic Buffer 本身以及 Buffer 状态。

struct BufferSlot {
    // mGraphicBuffer points to the buffer allocated for this slot or is NULL
    // if no buffer has been allocated.
    sp<GraphicBuffer> mGraphicBuffer;

    // mBufferState is the current state of this buffer slot.
    BufferState mBufferState;
}

GraphicBuffer

GraphicBuffer 借由 Gralloc 接口，最终通过 ION 给 Buffer 分配内存。

Gralloc 内存分配器会进行缓冲区分配，并通过两个特定于供应商的 HIDL 接口来进行实现（请参阅 hardware/interfaces/graphics/allocator/ 和 hardware/interfaces/graphics/mapper/）。allocate() 函数采用预期的参数（宽度、高度、像素格式）以及一组用法标志。

ION 是谷歌推出的一种跨设备跨空间的内存分配方式，是为了终结OEM厂商各有各的接口的混乱局面。

Buffer 状态

Buffer 状态通过 BufferState 来表示，总共有下面几种：

  • FREE
  • DEQUEUED
  • QUEUED
  • ACQUIRED
  • SHARED

而这几个状态的表示和切换 BufferState 是通过内部三个计数器和一个标志位来实现的：

// BufferState tracks the states in which a buffer slot can be.
struct BufferState {

    // All slots are initially FREE (not dequeued, queued, acquired, or shared).
    BufferState()
    : mDequeueCount(0),
      mQueueCount(0),
      mAcquireCount(0),
      mShared(false) {
    }

具体的规则如下：

// frameworks/native/libs/gui/include/gui/BufferSlot.h
// A buffer can be in one of five states, represented as below:
//
//         | mShared | mDequeueCount | mQueueCount | mAcquireCount |
// --------|---------|---------------|-------------|---------------|
// FREE    |  false  |       0       |      0      |       0       |
// DEQUEUED|  false  |       1       |      0      |       0       |
// QUEUED  |  false  |       0       |      1      |       0       |
// ACQUIRED|  false  |       0       |      0      |       1       |
// SHARED  |  true   |      any      |     any     |      any      |

BufferState 内部提供了实用函数来方便状态的查询和切换：

inline bool isFree() const {
    return !isAcquired() && !isDequeued() && !isQueued();
}

inline bool isDequeued() const {
    return mDequeueCount > 0;
}

inline bool isQueued() const {
    return mQueueCount > 0;
}

inline bool isAcquired() const {
    return mAcquireCount > 0;
}

inline bool isShared() const {
    return mShared;
}

inline void reset() {
    *this = BufferState();
}

const char* string() const;

inline void dequeue() {
    mDequeueCount++;
}

inline void detachProducer() {
    if (mDequeueCount > 0) {
        mDequeueCount--;
    }
}

inline void attachProducer() {
    mDequeueCount++;
}

inline void queue() {
    if (mDequeueCount > 0) {
        mDequeueCount--;
    }
    mQueueCount++;
}

inline void cancel() {
    if (mDequeueCount > 0) {
        mDequeueCount--;
    }
}

inline void freeQueued() {
    if (mQueueCount > 0) {
        mQueueCount--;
    }
}

inline void acquire() {
    if (mQueueCount > 0) {
        mQueueCount--;
    }
    mAcquireCount++;
}

inline void acquireNotInQueue() {
    mAcquireCount++;
}

inline void release() {
    if (mAcquireCount > 0) {
        mAcquireCount--;
    }
}

这几个状态在后面的 API 部分 详细说明。

mFreeSlots/mFreeBuffers/mUnusedSlots/mActiveBuffers

这 4 个存放到的都是 mSlots 里的数组下标索引：

// mFreeSlots contains all of the slots which are FREE and do not currently
// have a buffer attached.
// BufferSlot 里没有已分配 Buffer，且状态为 FREE
std::set<int> mFreeSlots;

// mFreeBuffers contains all of the slots which are FREE and currently have
// a buffer attached.
// BufferSlot 里有已分配 Buffer，且状态为 FREE
std::list<int> mFreeBuffers;

// mUnusedSlots contains all slots that are currently unused. They should be
// free and not have a buffer attached.
std::list<int> mUnusedSlots;

// mActiveBuffers contains all slots which have a non-FREE buffer attached.
// BufferSlot 里有已分配 Buffer，且状态为非 FREE
std::set<int> mActiveBuffers;

mQueue

// mQueue is a FIFO of queued buffers used in synchronous mode.
Fifo mQueue;

typedef Vector<BufferItem> Fifo;

mQueue 则是一个使用 Vector 来模拟队列的，存储对象是 BufferItem：

class BufferItem : public Flattenable<BufferItem> {

BufferItem 继承了 Flattenable，因此可以进行跨进程传输。

BufferQueue API

我们已经知道 BufferQueue 是生产者和消费者之间的一个纽带，所以接下来我们来看看生产者和消费者都是怎么跟 BufferQueue 打交道的：

dequeueBuffer

dequeueBuffer() 作用是从 BufferQueue 里获取一块 Buffer。核心的获取逻辑在 BufferQueueProducer::waitForFreeSlotThenRelock()：

status_t BufferQueueProducer::waitForFreeSlotThenRelock(FreeSlotCaller caller,
        std::unique_lock<std::mutex>& lock, int* found) const {
		......
        *found = BufferQueueCore::INVALID_BUFFER_SLOT;

        // If we disconnect and reconnect quickly, we can be in a state where
        // our slots are empty but we have many buffers in the queue. This can
        // cause us to run out of memory if we outrun the consumer. Wait here if
        // it looks like we have too many buffers queued up.
        const int maxBufferCount = mCore->getMaxBufferCountLocked();
        bool tooManyBuffers = mCore->mQueue.size()
                            > static_cast<size_t>(maxBufferCount);
        if (tooManyBuffers) {
            BQ_LOGV("%s: queue size is %zu, waiting", callerString,
                    mCore->mQueue.size());
        } else {
            // If in shared buffer mode and a shared buffer exists, always
            // return it.
            if (mCore->mSharedBufferMode && mCore->mSharedBufferSlot !=
                    BufferQueueCore::INVALID_BUFFER_SLOT) {
                *found = mCore->mSharedBufferSlot;
            } else {
                if (caller == FreeSlotCaller::Dequeue) {
                    // If we're calling this from dequeue, prefer free buffers
                    int slot = getFreeBufferLocked();
                    if (slot != BufferQueueCore::INVALID_BUFFER_SLOT) {
                        *found = slot;
                    } else if (mCore->mAllowAllocation) {
                        *found = getFreeSlotLocked();
                    }
                } else {
                    // If we're calling this from attach, prefer free slots
                    int slot = getFreeSlotLocked();
                    if (slot != BufferQueueCore::INVALID_BUFFER_SLOT) {
                        *found = slot;
                    } else {
                        *found = getFreeBufferLocked();
                    }
                }
            }
        }

核心逻辑就是：优先从 mFreeBuffer 里找一个 Slot，其次再从 mFreeSlots 里找。这两个的区别是：从 mFreeBuffer 找到的 slot 就不需要重新分配 Buffer 空间了；而 mFreeSlots 则需要。

但是这里有一个问题是，当 App 第一次执行 dequeueBuffer() 的时候，此时 mFreeSlots 也是空的呀，那岂不就永远都分配不到 slot 了吗？其实不然，在 BufferQueueCore 的构造函数其实已经解决这个问题了：

BufferQueueCore::BufferQueueCore() {
	......
	// 将 [0, MaxBufferCount) 这个区间内的 index 设置为 mFreeSlots
    int numStartingBuffers = getMaxBufferCountLocked();
    for (int s = 0; s < numStartingBuffers; s++) {
        mFreeSlots.insert(s);
    }
	// 将 [MaxBufferCount, NUM_BUFFER_SLOTS) 这个区间内的 index 设置为 mUnusedSlots
    for (int s = numStartingBuffers; s < BufferQueueDefs::NUM_BUFFER_SLOTS;
            s++) {
        mUnusedSlots.push_front(s);
    }

可以看到，BufferQueueCore 在构造阶段的后面，会将 mSlots 分为 mFreeSlots 和 mUnusedSlots。

好，说完上面的小插曲以后，我们再接着来看，具体是在下面的逻辑里去分配空间的：

status_t BufferQueueProducer::dequeueBuffer(int* outSlot, sp<android::Fence>* outFence,
                                            uint32_t width, uint32_t height, PixelFormat format,
                                            uint64_t usage, uint64_t* outBufferAge,
                                            FrameEventHistoryDelta* outTimestamps) {
		......
        int found = BufferItem::INVALID_BUFFER_SLOT;
        while (found == BufferItem::INVALID_BUFFER_SLOT) {
            status_t status = waitForFreeSlotThenRelock(FreeSlotCaller::Dequeue, lock, &found);
            if (status != NO_ERROR) {
                return status;
            }
		......

        *outSlot = found;
        ATRACE_BUFFER_INDEX(found);

        if ((buffer == nullptr) ||
                buffer->needsReallocation(width, height, format, BQ_LAYER_COUNT, usage))
        {
            returnFlags |= BUFFER_NEEDS_REALLOCATION;
		}


    if (returnFlags & BUFFER_NEEDS_REALLOCATION) {
        BQ_LOGV("dequeueBuffer: allocating a new buffer for slot %d", *outSlot);
		// 真正分配 Buffer 的地方
        sp<GraphicBuffer> graphicBuffer = new GraphicBuffer(
                width, height, format, BQ_LAYER_COUNT, usage,
                {mConsumerName.string(), mConsumerName.size()});

        status_t error = graphicBuffer->initCheck();

        { // Autolock scope
            std::lock_guard<std::mutex> lock(mCore->mMutex);

            if (error == NO_ERROR && !mCore->mIsAbandoned) {
                graphicBuffer->setGenerationNumber(mCore->mGenerationNumber);
				// 将 Buffer 与 mSlot 结合
                mSlots[*outSlot].mGraphicBuffer = graphicBuffer;
}

queueBuffer

当 dequeueBuffer() 从 BufferQueue 里获取到 GraphicBuffer 并填充完以后，会通过 queueBuffer() 将 Buffer 送回 BufferQueue，具体的：

// frameworks/native/libs/gui/BufferQueueProducer.cpp
status_t BufferQueueProducer::queueBuffer(int slot,
        const QueueBufferInput &input, QueueBufferOutput *output) {
    ...
    sp<IConsumerListener> frameAvailableListener;
    sp<IConsumerListener> frameReplacedListener;
    ...
    BufferItem item;
    { // Autolock scope
        ...
        const sp<GraphicBuffer>& graphicBuffer(mSlots[slot].mGraphicBuffer);
        ...
        mSlots[slot].mBufferState.queue();  // 修改mBufferState为QUEUE
        ...
         
        // BUfferItem赋值
        item.mAcquireCalled = mSlots[slot].mAcquireCalled;
        item.mGraphicBuffer = mSlots[slot].mGraphicBuffer;
        ...
 
        if (mCore->mQueue.empty()) {  // 队列为空时将BufferItem放到mQueue的尾部
            // When the queue is empty, we can ignore mDequeueBufferCannotBlock and simply queue this buffer
            mCore->mQueue.push_back(item);
            frameAvailableListener = mCore->mConsumerListener;
        } else {  // 队列不为空是判断尾部BufferItem是否可以替换，如果可以则替换，否则放在队列尾部
            // When the queue is not empty, we need to look at the last buffer in the queue to see if we need to replace it
            const BufferItem& last = mCore->mQueue.itemAt(mCore->mQueue.size() - 1);
            if (last.mIsDroppable) {
                ...
                // Overwrite the droppable buffer with the incoming one
                mCore->mQueue.editItemAt(mCore->mQueue.size() - 1) = item;  // 替换末尾的BufferSlot
                frameReplacedListener = mCore->mConsumerListener;
            } else {
                mCore->mQueue.push_back(item);  // 将BufferSlot放在mQueue尾部
                frameAvailableListener = mCore->mConsumerListener;
            }
        }
        ...
    } // Autolock scope
    ...
 
    { // scope for the lock
        ...
        if (frameAvailableListener != nullptr) {
            frameAvailableListener->onFrameAvailable(item);  // 调用消费者的onFrameAvailable()通知队列中有BufferSlot可以消费使用
        } else if (frameReplacedListener != nullptr) {
            frameReplacedListener->onFrameReplaced(item);  // 调用消费者的onFrameReplaced()通知队列中有替换过的BufferSlot
        }
        ...
    }
    ..
 
    return NO_ERROR;
}

总结，queueBuffer() 将其放回mQueue，将 Buffer 状态置为 QUEUE 状态并通过回调函数通知 BufferQueueConsumer 开始进行消费。

acquireBuffer

从 mQueue 中获取 BufferSlot，将其状态置为 ACQUIRED 并从mQueue中移除

releaseBuffer

  1. BufferSlot有效性检查
  2. 修改 Buffer 状态为 FREE
  3. 将 Slot 从 mActivateBuffers 中移除并放回 mFreeBuffers

注意：该过程不会解除 GraphicBuffer 和 BufferSlot 的绑定，即 GraphicBuffer 不会被释放，这样的好处是显而易见的，后面的 dequeueBuffer() 就可以更快地获取到 Buffer 了，有效地提高了性能。

总结

用下面一张图总结各个 API 以及涉及到 Buffer 状态切换：

Buffer状态切换

Author: simowce

Permalink: https://blog.simowce.com/all-about-bufferqueuecore/

知识共享许可协议
本作品采用 知识共享署名-非商业性使用-禁止演绎 4.0 国际许可协议 进行许可。

updated at 2022-04-27

# BufferQueue
Previous Post
BufferQueue 详解
Next Post
析构函数的妙用