fwrite

Overview#

Official Introduction: Android-Graphics

• Image Stream Producers: Graphics producers that generate graphic buffers for graphic consumers. For example, OpenGL ES, Canvas 2D, and mediaserver video decoders.

• Native Framework: The Libgui library, which contains the basic elements of the graphics system's underlying architecture, such as Surface, BufferQueue, GraphicBufferConsumer, GraphicBufferProducer, SurfaceComposer, etc.

• Window Positioning: WindowManager, which controls window objects; a window is a collection of view objects. It provides buffers and window metadata to SurfaceFlinger, which uses this information to compose Surfaces onto the screen.

• Image Stream Consumers: Graphic consumers, including SurfaceFlinger and some graphic applications.

  • The most common consumer of graphic streams is SurfaceFlinger, which consumes currently visible Surfaces and uses information provided by WindowManager to compose them onto the display. SurfaceFlinger uses OpenGL and Hardware Composer to compose a set of Surfaces.
  • Some OpenGL ES applications can also act as consumers, such as camera applications that consume camera preview image streams. Non-GL applications can also be consumers, such as the ImageReader class.

• HAL: The hardware abstraction layer implementation of the display subsystem, including Hardware Composer and Gralloc.

  • SurfaceFlinger can delegate some compositing work to Hardware Composer to offload the workload from OpenGL and the GPU. In this case, SurfaceFlinger acts as another OpenGL ES client, while Hardware Composer handles the graphics rendering. Hardware Composer must support events, one of which is VSYNC (the other is hot-plugging support for plug-and-play HDMI).
  • Gralloc is used to allocate memory requested by graphic producers.

BufferQueue#

Official Introduction: Android-BufferQueue

The object on the left is the renderer that generates graphic buffers, such as the home screen, status bar, and system interface. SurfaceFlinger is the compositor, while Hardware Composer is the producer. BufferQueue is a crucial component of the Android graphics system, responsible for data transfer:

The producers and consumers in the diagram run in different processes. BufferQueue is a data structure that combines a pool of buffers with a queue, using Binder IPC to transfer buffers between processes. Several important functions are as follows:

• Producers request a free buffer (GraphicBuffer) through BufferQueue: IGraphicBufferProducer.dequeueBuffer method.
• After filling the buffer (GraphicBuffer) with data (drawing, etc.), producers enqueue the buffer (GraphicBuffer) into BufferQueue: IGraphicBufferProducer.queueBuffer method.
• Consumers dequeue a buffer (GraphicBuffer) from BufferQueue: IGraphicBufferConsumer.acquireBuffer method.
• After consumption (typically by SurfaceFlinger for composited data), consumers return the buffer (GraphicBuffer) to the queue: IGraphicBufferConsumer.releaseBuffer method.

Among them, IGraphicBufferProducer is the producer interface of BufferQueue, with the implementation class being BufferQueueProducer; IGraphicBufferConsumer is the consumer interface of BufferQueue, with the implementation class being BufferQueueConsumer.

SurfaceFlinger#

SurfaceFlinger is used to manage the consumption of currently visible Surfaces. All rendered visible Surfaces are composed by SurfaceFlinger using information provided by WindowManager (using OpenGL and Hardware Composer) and submitted to the screen's back buffer, waiting for the next Vsync signal from the screen to display on the screen. SurfaceFlinger establishes a connection with the screen through the back buffer while connecting with the upper layer through Surfaces, acting as a bridge.

SurfaceFlinger Workflow

In the registerCallback of SurfaceFlinger, this refers to the implementation of the ComposerCallback interface by SurfaceFlinger.

• onHotplugReceived

  Hot-plug event callback when the display screen is connected or disconnected.

• onRefreshReceived

  Receives refresh requests from the underlying HWComposer. The implementation method is as follows:

  Copy

  void SurfaceFlinger::onRefreshReceived(int sequenceId,
                                         hwc2_display_t /*display*/) {
      Mutex::Autolock lock(mStateLock);
      if (sequenceId != getBE().mComposerSequenceId) {
          return;
      }
      repaintEverythingLocked();
  }
  
  void SurfaceFlinger::repaintEverythingLocked() {
      android_atomic_or(1, &mRepaintEverything);
      // Trigger refresh and recompose display
      signalTransaction();
  }
  

• onVsyncReceived

  Vsync event reporting, receiving vertical sync signals reported by the underlying hardware.

Why vertical sync signals are needed, refer to https://juejin.cn/post/6863756420380196877#heading-1

Composition Methods#

• Client Composition

Render the contents of each Layer into a temporary buffer using the GPU, and finally transfer the temporary buffer to the display hardware. Client composition uses RenderEngine for composition.

• Device Composition

The hardware composer performs composition using HWComposer, which composes by sending all Layer data to the display hardware and informing it to read different parts of the screen from different buffers.

HWC Related Diagrams#

Composition Diagram#

Dump Information#

adb shell dumpsys SurfaceFlinger

Layers#

Layers are the most important unit of composition; each layer has a set of attributes that define how it interacts with other layers. The implementation of Layer in each layer's code varies, but the basic concept of Layer is the same.

frameworks/native/services/surfaceflinger
├── Layer.h
├── Layer.cpp
├── ColorLayer.h
├── ColorLayer.cpp
├── BufferLayer.h
└── BufferLayer.cpp

Summary#

The entire graphics display system can be simply divided into two systems: graphics and display.

• The graphics system provides support for drawing and graphic processing. Whether it's 2D Skia, 3D OpenGLES, or various image decoding libraries.
• The display system, after graphics are drawn, needs to display them, which requires merging windows. At this point, SurfaceFlinger is needed. The upper display system consists of View, ActivityManagerService, and WindowManagerService. A window is a Surface, and SurfaceFlinger uses Layers to describe a window.