Let’s first use a picture to see the corresponding relationship between layers between various parts. Next, we will analyze how the conversion is performed based on this diagram.
Creation of Surface and Layer
For the process of app creating Surface, please refer to “09 SurfaceFinger layer creation process” for the previous introduction and introduction. BufferStateLayer or other types of Layers will be created in SurfaceFlinger;
And a transaction will be created and put into mTransactionQueue; 0 “10 Analysis of SurfaceComposerClient Submission Process” learned that when the app executes apply(), a transaction will also be submitted to SurfaceFlinger and put into mTransactionQueue;
OutputLayerCreation
When the vsync signal arrives, SurfaceFlinger::onMessageInvalidate will be executed; the specific reason why this is called is not the focus of this article and can be discussed later.
Let’s take a look at the implementation of onMessageInvalidate
void SurfaceFlinger::onMessageInvalidate(int64_t vsyncId, nsecs_t expectedVSyncTime) {
... ...
refreshNeeded = handleMessageTransaction();
... ...
onMessageRefresh();
}
handleMessageTransaction() is called here; as the name suggests, it handles message transactions
bool SurfaceFlinger::handleMessageTransaction() {
if (getTransactionFlags(eTransactionFlushNeeded)) {
flushTransactionQueues();
}
... ...
}
The eTransactionFlushNeeded flag is set when committing the transaction
Then execute flushTransactionQueues();
void SurfaceFlinger::flushTransactionQueues() {
std::vector<TransactionState> transactions;
auto it = mPendingTransactionQueues.begin();
while (it != mPendingTransactionQueues.end()) {
auto & amp; [applyToken, transactionQueue] = *it;
while (!transactionQueue.empty()) {
auto & amp; transaction = transactionQueue.front();
if (!transactionIsReadyToBeApplied(transaction.frameTimelineInfo,
transaction.isAutoTimestamp,
transaction.desiredPresentTime,
transaction.originUid, transaction.states,
bufferLayersReadyToPresent)) {
setTransactionFlags(eTransactionFlushNeeded);
break;
}
transaction.traverseStatesWithBuffers([ & amp;](const layer_state_t & amp; state) {
bufferLayersReadyToPresent.insert(state.surface);
});
transactions.emplace_back(std::move(transaction));
transactionQueue.pop();
}
if (transactionQueue.empty()) {
it = mPendingTransactionQueues.erase(it);
mTransactionQueueCV.broadcast();
} else {
it = std::next(it, 1);
}
}
while (!mTransactionQueue.empty()) {
auto & amp; transaction = mTransactionQueue.front();
bool pendingTransactions = mPendingTransactionQueues.find(transaction.applyToken) !=
mPendingTransactionQueues.end();
if (pendingTransactions ||
!transactionIsReadyToBeApplied(transaction.frameTimelineInfo,
transaction.isAutoTimestamp,
transaction.desiredPresentTime,
transaction.originUid, transaction.states,
bufferLayersReadyToPresent)) {
mPendingTransactionQueues[transaction.applyToken].push(std::move(transaction));
} else {
transaction.traverseStatesWithBuffers([ & amp;](const layer_state_t & amp; state) {
bufferLayersReadyToPresent.insert(state.surface);
});
transactions.emplace_back(std::move(transaction));
}
mTransactionQueue.pop();
ATRACE_INT("TransactionQueue", mTransactionQueue.size());
}
for (auto & amp; transaction : transactions) {
applyTransactionState(transaction.frameTimelineInfo, transaction.states,
transaction.displays, transaction.flags,
transaction.inputWindowCommands, transaction.desiredPresentTime,
transaction.isAutoTimestamp, transaction.buffer,
transaction.postTime, transaction.permissions,
transaction.hasListenerCallbacks, transaction.listenerCallbacks,
transaction.originPid, transaction.originUid, transaction.id);
}
}
This code is stinky and long. Mainly do three things:
1. Determine whether the current transaction can be applied. If it can be applied, take the transaction out of mPendingTransactionQueues and mTransactionQueue and put it into transactions;
2. If the transaction cannot be applied currently, put the transaction in mTransactionQueue into mPendingTransactionQueues
3. Traverse each transaction and execute applyTransactionState
void SurfaceFlinger::applyTransactionState(const FrameTimelineInfo & amp; frameTimelineInfo,
const Vector<ComposerState> & amp; states,
Vector<DisplayState> & displays, uint32_t flags,
const InputWindowCommands & amp; inputWindowCommands,
const int64_t desiredPresentTime, bool isAutoTimestamp,
const client_cache_t & uncacheBuffer,
const int64_t postTime, uint32_t permissions,
bool hasListenerCallbacks,
const std::vector<ListenerCallbacks> & amp; listenerCallbacks,
int originPid, int originUid, uint64_t transactionId) {
for (const ComposerState & amp; state : states) {
clientStateFlags |=
setClientStateLocked(frameTimelineInfo, state, desiredPresentTime, isAutoTimestamp,
postTime, permissions, listenerCallbacksWithSurfaces);
}
}
applyTransactionState calls setClientStateLocked to parse the surface properties we configured in 08 SurfaceComposerClient Transaction. and put in mCurrentState.layersSortedByZ.
uint32_t SurfaceFlinger::setClientStateLocked(
const FrameTimelineInfo & amp; frameTimelineInfo, const ComposerState & amp; composerState,
int64_t desiredPresentTime, bool isAutoTimestamp, int64_t postTime, uint32_t permissions,
std::unordered_set<ListenerCallbacks, ListenerCallbacksHash> & amp; outListenerCallbacks) {
if (what & amp; layer_state_t::eLayerChanged) {
// NOTE: index needs to be calculated before we update the state
const auto & amp; p = layer->getParent();
if (p == nullptr) {
ssize_t idx = mCurrentState.layersSortedByZ.indexOf(layer);
if (layer->setLayer(s.z) & amp; & idx >= 0) {
mCurrentState.layersSortedByZ.removeAt(idx);
mCurrentState.layersSortedByZ.add(layer);
flags |= eTransactionNeeded|eTraversalNeeded;
}
} else {
if (p->setChildLayer(layer, s.z)) {
flags |= eTransactionNeeded|eTraversalNeeded;
}
}
}
if (what & amp; layer_state_t::eRelativeLayerChanged) {
// NOTE: index needs to be calculated before we update the state
const auto & amp; p = layer->getParent();
const auto & amp; relativeHandle = s.relativeLayerSurfaceControl ?
s.relativeLayerSurfaceControl->getHandle() : nullptr;
if (p == nullptr) {
ssize_t idx = mCurrentState.layersSortedByZ.indexOf(layer);
if (layer->setRelativeLayer(relativeHandle, s.z) & amp; & amp;
idx >= 0) {
mCurrentState.layersSortedByZ.removeAt(idx);
mCurrentState.layersSortedByZ.add(layer);
flags |= eTransactionNeeded|eTraversalNeeded;
}
} else {
if (p->setChildRelativeLayer(layer, relativeHandle, s.z)) {
flags |= eTransactionNeeded|eTraversalNeeded;
}
}
}
}
It can be seen from the code that layer_state_t::eLayerChanged, layer_state_t::eRelativeLayerChanged is set and the layer will be put into layersSortedByZ when the layer has no parent.
In this article, we only parse the layer into layersSortedByZ. Don’t be distracted by other codes.
Then return to onMessageInvalidate()
void SurfaceFlinger::onMessageInvalidate(int64_t vsyncId, nsecs_t expectedVSyncTime) {
... ...
refreshNeeded = handleMessageTransaction();
... ...
onMessageRefresh();
}
Later onMessageRefresh() is called; there is still a lot of content inside, but it doesn’t matter, just delete some of it.
void SurfaceFlinger::onMessageRefresh() {
compositionengine::CompositionRefreshArgs refreshArgs;
const auto & amp; displays = ON_MAIN_THREAD(mDisplays);
refreshArgs.outputs.reserve(displays.size());
for (const auto & amp; [_, display] : displays) {
refreshArgs.outputs.push_back(display->getCompositionDisplay());
}
mDrawingState.traverseInZOrder([ & amp;refreshArgs](Layer* layer) {
if (auto layerFE = layer->getCompositionEngineLayerFE())
refreshArgs.layers.push_back(layerFE);
});
mCompositionEngine->present(refreshArgs);
}
Put all Display into refreshArgs.outputs; use mDrawingState.traverseInZOrder to traverse all Layers in mCurrentState.layersSortedByZ and put into refreshArgs.layers
Call mCompositionEngine->present (android12\frameworks\
ative\services\surfaceflinger\CompositionEngine\src\CompositionEngine.cpp)
void CompositionEngine::present(CompositionRefreshArgs & amp; args) {
for (const auto & amp; output : args.outputs) {
output->prepare(args, latchedLayers);
}
}
The output here is actually Display, but prepare is not implemented in Display because Display inherits from Output and uses the prepare method of the parent class Output.
void Output::prepare(const compositionengine::CompositionRefreshArgs & amp; refreshArgs,
LayerFESet & geomSnapshots) {
rebuildLayerStacks(refreshArgs, geomSnapshots);
}
void Output::rebuildLayerStacks(const compositionengine::CompositionRefreshArgs & amp; refreshArgs,
LayerFESet & layerFESet) {
compositionengine::Output::CoverageState coverage{layerFESet};
collectVisibleLayers(refreshArgs, coverage);
}
void Output::collectVisibleLayers(const compositionengine::CompositionRefreshArgs & amp; refreshArgs,
compositionengine::Output::CoverageState & amp; coverage) {
for (auto layer : reversed(refreshArgs.layers)) {
ensureOutputLayerIfVisible(layer, coverage);
}
}
We see that ensureOutputLayerIfVisible is finally executed.
void Output::ensureOutputLayerIfVisible(sp<compositionengine::LayerFE> & amp; layerFE,
compositionengine::Output::CoverageState & amp; coverage) {
auto result = ensureOutputLayer(prevOutputLayerIndex, layerFE);
}
ensureOutputLayerIfVisible will process the coverage area of all layers in sequence. How to calculate it is not the focus of this article. At present, we only focus on the relationship between layers, so this function is only reduced to one line, [manual dog head].
The ensureOutputLayer method is not implemented in Output.cpp. I went to android12\frameworks\
ative\services\surfaceflinger\CompositionEngine\src\Display.cpp and looked for it, and I found it.
std::unique_ptr<compositionengine::OutputLayer> Display::createOutputLayer(
const sp<compositionengine::LayerFE> & amp; layerFE) const {
auto outputLayer = impl::createOutputLayer(*this, layerFE);
if (const auto halDisplayId = HalDisplayId::tryCast(mId);
outputLayer & amp; & amp; !mIsDisconnected & amp; & amp; halDisplayId) {
auto & amp; hwc = getCompositionEngine().getHwComposer();
auto hwcLayer = hwc.createLayer(*halDisplayId);
outputLayer->setHwcLayer(std::move(hwcLayer));
}
return outputLayer;
}
You can see that Display::createOutputLayer uses impl::createOutputLayer to create OutputLayer
std::unique_ptr<OutputLayer> createOutputLayer(const compositionengine::Output & amp; output,
const sp<compositionengine::LayerFE> & amp; layerFE) {
return createOutputLayerTemplated<OutputLayer>(output, layerFE);
}
createOutputLayerTemplated is a template function, I will expand it directly here.
std::unique_ptr<OutputLayer > createOutputLayerTemplated(const Output & amp; output,
sp<LayerFE> layerFE) {
class OutputLayer final : public OutputLayer {
public:
OutputLayer(const Output & amp; output, const sp<LayerFE> & amp; layerFE)
: mOutput(output), mLayerFE(layerFE) {}
~OutputLayer() override = default;
private:
const Output & amp; getOutput() const override { return mOutput; }
LayerFE & getLayerFE() const override { return *mLayerFE; }
const OutputLayerCompositionState & amp; getState() const override { return mState; }
OutputLayerCompositionState & amp; editState() override { return mState; }
const Output & mOutput;
const sp<LayerFE> mLayerFE;
OutputLayerCompositionState mState;
};
return std::make_unique<OutputLayer>(output, layerFE);
}
Simply delete some unnecessary code, and you can see that OutputLayer inherits the OutputLayer in the same file, and adds Display mOutput to which several members belong, mLayerFE converted from which layer, and mState to save parameters. Create final OutputLayer and return.
Finally, I saw the creation of OutputLayer. Since there are a lot of codes, it is easy to be interfered by other codes. By deleting other codes, the analysis will be much clearer.
To summarize the general process:
- When the app creates a surface, it will create a transaction and put it into mTransactionQueue.
- When the next vsync comes and needs to be synthesized, put the transactions stored in mPendingTransactionQueues and mTransactionQueue into transactions
- Traverse each transaction and determine whether to put the current layer into mCurrentState.layersSortedByZ based on layer_state_t eLayerChanged and eRelativeLayerChanged
- Display traverses mCurrentState.layersSortedByZ and calculates the visible area to create an OutputLayer.
Creation of hwcLayer
Take a breath and look back at Display::createOutputLayer
std::unique_ptr<compositionengine::OutputLayer> Display::createOutputLayer(
const sp<compositionengine::LayerFE> & amp; layerFE) const {
... ...
if (const auto halDisplayId = HalDisplayId::tryCast(mId);
outputLayer & amp; & amp; !mIsDisconnected & amp; & amp; halDisplayId) {
auto & amp; hwc = getCompositionEngine().getHwComposer();
auto hwcLayer = hwc.createLayer(*halDisplayId);
outputLayer->setHwcLayer(std::move(hwcLayer));
}
return outputLayer;
}
Call hwc.createLayer(*halDisplayId); create hwcLayer
Put hwclayer into outputlayer
void OutputLayer::setHwcLayer(std::shared_ptr<HWC2::Layer> hwcLayer) {
auto & amp; state = editState();
if (hwcLayer) {
state.hwc.emplace(std::move(hwcLayer));
} else {
state.hwc.reset();
}
}
The state here is OutputLayerCompositionState mState;
In this way, SurfaceFlinger can process the layer-related information directly and get it directly to OutputLayer, which contains all image information.
I sigh, it’s too convoluted. . . . . .
Creation is not easy, please like and comment.