fwrite

Overview#

Official Introduction: Android-Graphics

• Image Stream Producers: Graphics producers that generate graphics buffers for graphics consumers to use. Examples include OpenGL ES, Canvas 2D, and mediaserver video decoders.

• Native Framework: Libgui library, which contains the basic elements of the graphics system, such as Surface, BufferQueue, GraphicBufferConsumer, GraphicBufferProducer, SurfaceComposer, and more.

• Window Positioning: WindowManager, used to control window objects, which are a collection of view objects. It provides SurfaceFlinger with buffers and window metadata, and SurfaceFlinger can use this information to compose Surfaces onto the screen.

• Image Stream Consumers: Graphics consumers, including SurfaceFlinger and some graphics applications.

  • The most common consumer of graphics streams is SurfaceFlinger, a system service that consumes visible Surfaces and uses the information provided by WindowManager to compose them onto the display. SurfaceFlinger uses OpenGL and Hardware Composer to compose a set of Surfaces.
  • Some OpenGL ES applications can also act as consumers, such as camera applications that consume camera preview image streams. Non-GL applications can also be consumers, such as the ImageReader class.

• HAL: Hardware abstraction implementation of the display subsystem, including Hardware Composer and Gralloc.

  • SurfaceFlinger can delegate some composition work to Hardware Composer to offload work from OpenGL and the GPU. In this case, SurfaceFlinger acts as another OpenGL ES client, and Hardware Composer performs the graphics rendering. Hardware Composer must support events, one of which is VSYNC (another is support for hot-plug HDMI).
  • Gralloc is used to allocate memory requested by graphics producers.

BufferQueue#

Official Introduction: Android-BufferQueue

The object on the left is the renderer that generates graphics buffers, such as the main screen, status bar, and system interface. SurfaceFlinger is the composer, and Hardware Composer is the producer. BufferQueue is an important component of the Android graphics system, responsible for data transfer:

The producer and consumer in the diagram run in different processes. BufferQueue is a data structure that combines a buffer pool with a queue, and it uses Binder IPC to transfer buffers between processes. Several important functions are as follows:

• Producers request an available buffer (GraphicBuffer) from BufferQueue using IGraphicBufferProducer.dequeueBuffer.
• After filling the buffer (GraphicBuffer) with data (drawing, etc.), producers enqueue the buffer (GraphicBuffer) into BufferQueue using IGraphicBufferProducer.queueBuffer.
• Consumers dequeue a buffer (GraphicBuffer) from BufferQueue using IGraphicBufferConsumer.acquireBuffer.
• After consuming the buffer (typically done by SurfaceFlinger for composition), consumers return the buffer (GraphicBuffer) to the queue using IGraphicBufferConsumer.releaseBuffer.

IGraphicBufferProducer is the producer interface of BufferQueue, and the implementation class is BufferQueueProducer. IGraphicBufferConsumer is the consumer interface of BufferQueue, and the implementation class is BufferQueueConsumer.

SurfaceFlinger#

SurfaceFlinger is used to manage the surfaces that are currently visible. All visible surfaces that are rendered will be submitted to the screen's back buffer by SurfaceFlinger using the information provided by WindowManager, and they will wait for the next Vsync signal to be displayed on the screen. SurfaceFlinger establishes a connection between the back buffer and the screen, and it also establishes a connection between the surfaces and the upper layer.

SurfaceFlinger Workflow

The this in SurfaceFlinger's registerCallback refers to the implementation of the ComposerCallback interface.

• onHotplugReceived

  Callback for hot-plug events, triggered when a display is connected or disconnected.

• onRefreshReceived

  Receives refresh requests from the underlying HWComposer. The implementation is as follows:

  Copy

  
  void SurfaceFlinger::onRefreshReceived(int sequenceId,
  
                                         hwc2_display_t /*display*/) {
  
      Mutex::Autolock lock(mStateLock);
  
      if (sequenceId !=getBE().mComposerSequenceId) {
  
          return;
  
      }
  
      repaintEverythingLocked();
  
  }
  
  
  void SurfaceFlinger::repaintEverythingLocked() {
  
      android_atomic_or(1, &mRepaintEverything);
  
      // Trigger refresh and recompose the display
  
      signalTransaction();
  
  }
  
  

• onVsyncReceived

  Callback for Vsync events, receives vertical sync signals reported by the underlying hardware.

For the need for vertical sync signals, refer to https://juejin.cn/post/6863756420380196877#heading-1

Composition Methods#

• Client Composition

The content of each layer is rendered to a temporary buffer using the GPU, and the temporary buffer is then transferred to the display hardware. Client composition uses RenderEngine for composition.

• Device Composition

Hardware Composer performs composition. The composition method is to pass all the data of each layer to the display hardware and inform it to read data from different buffers for different parts of the screen.

HWC Related Diagram#

Composition Diagram#

Dump Information#

adb shell dumpsys SurfaceFlinger

Layers#

Layers are the most important units of composition. Each layer has a set of attributes that define its interaction with other layers. Although the implementation of layers varies in each layer, the basic concept of layers is the same.

frameworks/native/services/surfaceflinger

├── Layer.h

├── Layer.cpp

├── ColorLayer.h

├── ColorLayer.cpp

├── BufferLayer.h

└── BufferLayer.cpp

Summary#

The entire graphics display system can be simply divided into two systems: the graphics system and the display system.

• The graphics system provides support for drawing and graphics processing. Whether it's 2D Skia, 3D OpenGL ES, or various image decoding libraries.
• The display system is responsible for displaying the graphics. It involves window composition, and that's where SurfaceFlinger comes in. The upper layer of the display system consists of View, ActivityManagerService, and WindowManagerService. A window is represented by a Surface, and SurfaceFlinger uses Layers to describe a window.