Flutter tips and different ideas to achieve cool 3D page turning and folding animation

So what are the benefits of using Fragment Shader directly instead of using Dart matrix transformation? To put it simply, it can reduce the time consuming of the CPU and directly send instructions to the GPU through the graphics language (GLSL). The performance will undoubtedly be improved, and the implementation will be simpler.

However, the act of loading a shader can be expensive, so it must be compiled into the appropriate platform-specific shader at runtime.

Of course, there are conditions for using Fragment Shader in Flutter. For example, it is generally necessary to introduce the #include header file, because when writing shader code, we all need Know the value of the local coordinates of the current fragment, and flutter/runtime_effect.glsl provides FlutterFragCoord().xy; to support accessing local coordinates, which is not a standard GLSL API.

In addition, Fragment Shader only supports .frag format files and does not support vertex shading files .vert. It also has the following restrictions:

• UBO and SSBO are not supported
• sampler2D is the only supported sampler type
• texture only supports two parameter versions (sampler and uv)
• Cannot declare additional mutable inputs
• Unsigned integers and boolean values are not supported

Therefore, if you need to move some existing GLSL effects, such as the code on shadertoy, then some necessary “code modifications” cannot be escaped. For example, the following code is a shader for a gradient animation:

void mainImage( out vec4 fragColor, in vec2 fragCoord ){
float strength = 0.4;
    float t = iTime/3.0;
    
    vec3 col = vec3(0);
    vec2 fC = fragCoord;

    for(int i = -1; i <= 1; i + + ) {
        for(int j = -1; j <= 1; j + + ) {

            fC = fragCoord + vec2(i,j)/3.0;
            vec2 pos = fC/iResolution.xy;
            pos.y /= iResolution.x/iResolution.y;
            pos = 4.0*(vec2(0.5) - pos);
            for(float k = 1.0; k < 7.0; k + =1.0){
                pos.x + = strength * sin(2.0*t + k*1.5 * pos.y) + t*0.5;
                pos.y + = strength * cos(2.0*t + k*1.5 * pos.x);
            }
            col + = 0.5 + 0.5*cos(iTime + pos.xyx + vec3(0,2,4));
        }
    }
    col /= 9.0;
    col = pow(col, vec3(0.4545));
    fragColor = vec4(col,1.0);
}

In Flutter, it needs to be converted into the following code:

• The first is the essential flutter/runtime_effect.glsl
• Next define the main() function
• Then we need to move the out vec4 fragColor; defined in mainImage to the global declaration
• Because in GLSL iResolution is used to represent the canvas pixel height and width, iTime is the time when the program is running, and here resolution and iTime are defined directly through uniform Used to accept input from Dart side, the rest of the logic remains unchanged
• Corresponding to fragCoord, you can obtain coordinates in Flutter through FlutterFragCoord

#version 460 core
#include <flutter/runtime_effect.glsl>

out vec4 fragColor;

uniform vec2 resolution;
uniform float iTime;

void main(){
    float strength = 0.25;
    float t = iTime/8.0;
    vec3 col = vec3(0);
    vec2 pos = FlutterFragCoord().xy/resolution.xy;
    pos = 4.0*(vec2(0.5) - pos);
    for(float k = 1.0; k < 7.0; k + =1.0){
        pos.x + = strength * sin(2.0*t + k*1.5 * pos.y) + t*0.5;
        pos.y + = strength * cos(2.0*t + k*1.5 * pos.x);
    }
    col + = 0.5 + 0.5*cos(iTime + pos.xyx + vec3(0,2,4));
    col = pow(col, vec3(0.4545));
    fragColor = vec4(col,1.0);
}

The first line #version 460 core specifies the OpenGL language version used.

It can be seen that converting a piece of GLSL code is not particularly troublesome. It mainly involves changes in coordinates and input parameters. However, these existing fragment shaders can provide us with extremely rich rendering effects, as shown in the following code:

• Introduce the above shaders code in pubspec.yaml

• Load the corresponding 'shaders/warp.frag' file through ShaderBuilder to obtain FragmentShader

• Use FragmentShader‘s setFloat to pass data

• By adding a shader to draw through Paint()..shader , the rendering can be completed

flutter:
  shaders:
    - shaders/warp.frag

·············
  
  late Ticker _ticker;

  Duration _elapsed = Duration.zero;

  @override
  void initState() {<!-- -->
    super.initState();
    _ticker = createTicker((elapsed) {<!-- -->
      setState(() {<!-- -->
        _elapsed = elapsed;
      });
    });
    _ticker.start();
  }

  @override
  Widget build(BuildContext context) => ShaderBuilder(
        assetKey: 'shaders/warp.frag',
        (BuildContext context, FragmentShader shader, _) => Scaffold(
          appBar: AppBar(
            title: const Text('Warp')
          ),
          body: CustomPaint(
            size: MediaQuery.of(context).size,
            painter: ShaderCustomPainter(shader, _elapsed)
          ),
        ),
      );

class ShaderCustomPainter extends CustomPainter {<!-- -->
  final FragmentShader shader;
  final Duration currentTime;

  ShaderCustomPainter(this.shader, this.currentTime);

  @override
  void paint(Canvas canvas, Size size) {<!-- -->
    shader.setFloat(0, size.width);
    shader.setFloat(1, size.height);
    shader.setFloat(2, currentTime.inMilliseconds.toDouble() / 1000.0);
    final Paint paint = Paint()..shader = shader;
    canvas.drawRect(Offset.zero & amp; size, paint);
  }

  @override
  bool shouldRepaint(CustomPainter oldDelegate) => true;
}

The only thing that needs explanation here is the shader.setFloat process, because it actually corresponds to the variables we have in the .frag file through indexing. To put it simply:

Here we define uniform vec2 resolution; and uniform float iTime; in GLSL, then vec2 resolution occupies index 0 and 1, and float iTime occupies index 2.

The general understanding is that vec2 means two float type values are stored together, so the vec2 resolution declared first occupies indexes 0 and 1. For example, as shown in the figure below, vec2 and vec3 are divided at this time. It occupies the index 0-4.

By defining the value in the GLSL shader through uniform , the corresponding data can be passed through the index of setFloat in Dart, thus forming a complete closed loop of data interaction.

The complete code of the gradient animation here in Flutter can be found in warp.frag in Github https://github.com/tbuczkowski/flutter_shaders.

At the same time, for the entire gradient animation mentioned above, the author also provides a comparison of the corresponding pure Dart code to achieve the same effect in the warehouse. From the data, it can be seen that the implementation using shaders has greatly improved the performance.

So looking back, the folding shader in the riveo_page_curl project looks like this, except for a bunch of unintelligible matrix changes, such as scale scaling, translate translation and project In addition to projection conversion, there are various incomprehensible trigonometric function calculations. The simple core is to calculate the radian of the curved part when the matrix changes, and to add shadow projection to improve the visual effect.

#include <flutter/runtime_effect.glsl>

uniform vec2 resolution;
uniform float pointer;
uniform float origin;
uniform vec4 container;
uniform float cornerRadius;
uniform sampler2D image;

const float r = 150.0;
const float scaleFactor = 0.2;

#definePI 3.14159265359
#define TRANSPARENT vec4(0.0, 0.0, 0.0, 0.0)

mat3 translate(vec2 p) {
    return mat3(1.0, 0.0, 0.0, 0.0, 1.0, 0.0, p.x, p.y, 1.0);
}

mat3 scale(vec2 s, vec2 p) {
    return translate(p) * mat3(s.x, 0.0, 0.0, 0.0, s.y, 0.0, 0.0, 0.0, 1.0) * translate(-p);
}

vec2 project(vec2 p, mat3 m) {
    return (inverse(m) * vec3(p, 1.0)).xy;
}

struct Paint {
    vec4 color;
    bool stroke;
    float strokeWidth;
    int blendMode;
};

struct Context {
    vec4 color;
    vec2p;
    vec2 resolution;
};


bool inRect(vec2 p, vec4 rct) {
    bool inRct = p.x > rct.x & amp; & amp; p.x < rct.z & amp; & amp; p.y > rct.y & amp; & amp; p.y < rct.w;
    if (!inRct) {
        return false;
    }
    // Top left corner
    if (p.x < rct.x + cornerRadius & amp; & amp; p.y < rct.y + cornerRadius) {
        return length(p - vec2(rct.x + cornerRadius, rct.y + cornerRadius)) < cornerRadius;
    }
    // Top right corner
    if (p.x > rct.z - cornerRadius & amp; & amp; p.y < rct.y + cornerRadius) {
        return length(p - vec2(rct.z - cornerRadius, rct.y + cornerRadius)) < cornerRadius;
    }
    // Bottom left corner
    if (p.x < rct.x + cornerRadius & amp; & amp; p.y > rct.w - cornerRadius) {
        return length(p - vec2(rct.x + cornerRadius, rct.w - cornerRadius)) < cornerRadius;
    }
    // Bottom right corner
    if (p.x > rct.z - cornerRadius & amp; & amp; p.y > rct.w - cornerRadius) {
        return length(p - vec2(rct.z - cornerRadius, rct.w - cornerRadius)) < cornerRadius;
    }
    return true;
}

out vec4 fragColor;

void main() {
    vec2 xy = FlutterFragCoord().xy;
    vec2 center = resolution * 0.5;
    float dx = origin - pointer;
    float x = container.z - dx;
    float d = xy.x - x;

    if (d > r) {
        fragColor = TRANSPARENT;
        if (inRect(xy, container)) {
            fragColor.a = mix(0.5, 0.0, (d-r)/r);
        }
    }

    else
    if (d > 0.0) {
        float theta = asin(d / r);
        float d1 = theta * r;
        float d2 = (3.14159265 - theta) * r;

        vec2 s = vec2(1.0 + (1.0 - sin(3.14159265/2.0 + theta)) * 0.1);
        mat3 transform = scale(s, center);
        vec2 uv = project(xy, transform);
        vec2 p1 = vec2(x + d1, uv.y);

        s = vec2(1.1 + sin(3.14159265/2.0 + theta) * 0.1);
        transform = scale(s, center);
        uv = project(xy, transform);
        vec2 p2 = vec2(x + d2, uv.y);

        if (inRect(p2, container)) {
            fragColor = texture(image, p2 / resolution);
        } else if (inRect(p1, container)) {
            fragColor = texture(image, p1 / resolution);
            fragColor.rgb *= pow(clamp((r - d) / r, 0.0, 1.0), 0.2);
        } else if (inRect(xy, container)) {
            fragColor = vec4(0.0, 0.0, 0.0, 0.5);
        }
    }
    else {
        vec2 s = vec2(1.2);
        mat3 transform = scale(s, center);
        vec2 uv = project(xy, transform);

        vec2 p = vec2(x + abs(d) + 3.14159265 * r, uv.y);
        if (inRect(p, container)) {
            fragColor = texture(image, p / resolution);
        } else {
            fragColor = texture(image, xy / resolution);
        }
    }

}

In fact, I know that everyone does not care about its implementation logic, but more about how to use it. The key information here is uniform sampler2D image. By introducing sampler2D, we can Dart passes ui.Image to GLSL through setImageSampler(0, image); , so that the above-mentioned folding animation logic can be implemented for the Flutter control.

Corresponding to the Dart layer, in addition to ShaderBuilder, you can also use flutter_shaders’ AnimatedSampler to implement more concise shader and image and canvas, the biggest role of AnimatedSampler is to take a screenshot of the entire child through PictureRecorder and convert it into ui .Image is passed to GLSL to complete the UI delivery interactive effect.

 Widget _buildAnimatedCard(BuildContext context, Widget? child) {<!-- -->
    return ShaderBuilder(
      (context, shader, _) {<!-- -->
        return AnimatedSampler(
          (image, size, canvas) {<!-- -->
            _configureShader(shader, size, image);
            _drawShaderRect(shader, size, canvas);
          },
          child: Padding(
            padding: EdgeInsets.symmetric(vertical: cornerRadius),
            child: widget.child,
          ),
        );
      },
      assetKey: 'shaders/page_curl.frag',
    );
    
    void _configureShader(FragmentShader shader, Size size, ui.Image image) {<!-- -->
    shader
      ..setFloat(0, size.width) // resolution
      ..setFloat(1, size.height) // resolution
      ..setFloat(2, _animationController.value) // pointer
      ..setFloat(3, 0) // origin
      ..setFloat(4, 0) // inner container
      ..setFloat(5, 0) // inner container
      ..setFloat(6, size.width) // inner container
      ..setFloat(7, size.height) // inner container
      ..setFloat(8, cornerRadius) // cornerRadius
      ..setImageSampler(0, image); // image
  }

  void _drawShaderRect(FragmentShader shader, Size size, Canvas canvas) {<!-- -->
    canvas.drawRect(
      Rect.fromCenter(
        center: Offset(size.width / 2, size.height / 2),
        width: size.width,
        height: size.height,
      ),
      Paint()..shader = shader,
    );
  }
    

The complete project can be found at: https://github.com/Rahiche/riveo_page_curl

So it can be seen thatcompared to implementing such 3D page folding in the Dart layer, the code implemented using FragmentShader will be simpler, and the performance experience will be better than the pure Dart implementation< /strong>The most important thing is that some shader codes similar to ShaderToy can be directly applied to Flutter through simple transplantation and adaptation, which is undoubtedly very friendly to the implementation of Flutter in game scenes.