Understanding Flutter's

handling of Gestures

Guillaume

Diallo-Mulliez

@Guitoof

@guitoof

Head of Flutter Engineering

Classic gestures handling

# GestureDetector

Gesture

 

Detector

GestureDetector

Higher level API for gesture detection

GestureDetector

Gesture Detector

Tap

# GestureDetector

Double Tap

Horizontal Drag

Long Press

Vertical Drag

Tap

Long Press

Horizontal Drag

Vertical Drag

Double Tap

Scale

Force Press

# GestureDetector

Pan

Need to handle some other custom gestures ?

# RawGestureDetector

Raw

Detector

GestureDetector

RawGestureDetector

RawGestureDetector

Detect gestures from custom gesture recognizers

Gesture

 

GestureDetector

GestureRecognizer

<<interface>>

RawGestureDetector

...

# Custom GestureRecognizer

TapGestureRecognizer

DoubleTapGestureRecognizer

LongPressGestureRecognizer

HorizontalDragGestureRecognizer

VerticalDragGestureRecognizer

PanGestureRecognizer

ScaleGestureRecognizer

Gesture detection/recognition

flow

Pointers & Events

Gesture events life cycle

# Gesture Detection Flow

  • PointerDownEvent

  • PointerMoveEvent

  • PointerUpEvent

  • PointerRemovedEvent

  • PointerAddedEvent

Pointer Events

+ hitTest

GestureBinding

+ handlePointerEvent

1

PlatformDispatcher

Entry point for

platform related events

1

Propagate Hit Test

# Gesture Detection Flow

Hit Testing

Hit testing down the Render Tree

hitTest ?

hitTest ?

hitTest ?

HitTestResult = 

# Gesture Detection Flow

Pointer Events

+ hitTest

handleEvent()

handleEvent()

RenderPointerListener

GestureBinding

+ handlePointerEvent

1

2

Call handleEvent on each target

Listener

RawGestureDetector

3

<Any>GestureRecognizer

+ addPointer

_handlePointerDown

Call addPointer on each recognizer

Register route 

PointerRouter

+ addRoute

+ dispatchEvent

+ handleEvent

Call all registered routes

<_handleAnyPointerEvent>

Actually handle event

4

RendererBinding

Propagate hitTest through the render tree

+ hitTest

PlatformDispatcher

Entry point for

platform related events

1

Propagate Hit Test

2

Dispatch Event

3

Register pointer events routes

4

Call registered routes to handles the event from recognizers

+ handleEvent

<<HitTestTarget>>

# Gesture Detection Flow

Custom Gesture Recognizer

GestureDetector

GestureRecognizer

<<interface>>

TapGestureRecognizer

RawGestureDetector

...

CustomGestureRecognizer

# Custom GestureRecognizer

DoubleTapGestureRecognizer

LongPressGestureRecognizer

HorizontalDragGestureRecognizer

VerticalDragGestureRecognizer

PanGestureRecognizer

ScaleGestureRecognizer


class CustomGestureRecognizer extends GestureRecognizer { 
}

Custom GestureRecognizer

Implement

# Custom GestureRecognizer

class CustomGestureRecognizer extends GestureRecognizer {

  @override
  void acceptGesture(int pointer) {}
  @override
  void rejectGesture(int pointer) {}
  @override
  String get debugDescription () => 'Custom Gesture';
  
}
			
# Custom GestureRecognizer

Custom GestureRecognizer

Implement


class CustomGestureRecognizer extends GestureRecognizer {
  @override 
  void addAllowedPointer(PointerEvent event) {
    GestureBinding.instance.pointerRouter.addRoute(
      event.pointer, _handleEvent, event.transform
    );
  }
  
  void _handleEvent(PointerEvent event) {
    if(event is PointerDownEvent) {
      /// handle pointer down event
    } else if (...) {
      /// or handle any other pointer event as you need here
    }
  }
  
  ...

  @override
  void acceptGesture(int pointer) {}
  @override
  void rejectGesture(int pointer) {}
  @override
  String get debugDescription () => 'Custom Gesture';
  
}
			
# Custom GestureRecognizer

Custom GestureRecognizer

Implement


class CustomGestureRecognizer extends GestureRecognizer {

  void Function(CustomGestureDetails details)? onCustomGestureDetected;

  @override 
  void addAllowedPointer(PointerEvent event) {
    GestureBinding.instance.pointerRouter.addRoute(
      event.pointer, _handleEvent, event.transform
    );
  }
  
  void _handleEvent(PointerEvent event) {
    if(event is PointerDownEvent) {
      /// handle pointer down event
    } else if (...) {
      /// or handle any other pointer event as you need here
    } else if(event is PointerUpEvent) {
      onCustomGestureDetected?.call(event);
    }
  }
  
  ...

  @override
  void acceptGesture(int pointer) {}
  @override
  void rejectGesture(int pointer) {}
  @override
  String get debugDescription () => 'Custom Gesture';
  
}
			
# Custom GestureRecognizer

Custom GestureRecognizer

Implement


...
return RawGestureDetector(
  gestures: {
    CustomGestureRecognizer: GestureRecognizerFactoryWithHandlers<CustomGestureRecognizer>(
      () => CustomGestureRecognizer(),
      (CustomGestureRecognizer instance) {
        instance.onCustomGestureDetected = ( CustomGestureDetails details) {
          /// Do stuff to handle the custom gesture here...
    	}
      },
    ),
  }
);
			
# Custom GestureRecognizer

Custom GestureRecognizer

Use with a RawGestureDetector

👇👇👇 TripleTapGestureRecognizer

🔄 RotationGestureRecognizer

return RawGestureDetector(
  gestures: {
    TripleTapGestureRecognizer: GestureRecognizerFactoryWithHandlers<CustomGestureRecognizer>(
      () => TripleTapGestureRecognizer(),
      (TripleTapGestureRecognizer instance) {
        instance.onTripleTap = ( TapUpDetails details) {...}
    ),
    RotationGestureRecognizer: GestureRecognizerFactoryWithHandlers<CustomGestureRecognizer>(
      () => RotationGestureRecognizer(),
      (RotationGestureRecognizer instance) {
        instance.onRotation = ( RotationDetails details) {...}
      },
    ),
  }
);
			
# Custom GestureRecognizer

Custom GestureRecognizer

Examples

What happens with multiple targets ?

Hit Testing

Hit Test Behavior

1. Should I pass the hit test through to my children?

2. Should I add myself to the list of widgets that were hit?

3. Should I tell my parent that I considered the hit test to hit me or my children?

deferToChild

translucent

opaque

# Hit Test Behavior

Hit Testing

Actual: Hit Test Behavior

HitTestResult = 

RenderObject 1

RenderObject 2

deferToChild

transluscent

opaque

deferToChild

transluscent

opaque

# Hit Test Behavior

Hit Testing

Hit Test Behavior: example

deferToChild

transluscent

opaque

Stack

A

B

Circle

Stack

A

B

Circle (opaque)

HitTestBehavior for B

# Hit Test Behavior

Who actually handles the event?

OR

Disambiguation

Gesture Arena

A

B

HitTestResult =

GestureArena

class AnyGestureRecognizer extends GestureRecognizer {
  ...
  GestureBinding.instance.gestureArena.add(pointer, this);
  ...
}
			
class AnyGestureRecognizer extends GestureRecognizer {
  ...
  gestureArenaEntry.resolve(GestureDisposition.accepted|rejected);
  ...
}

1 winner

OR

# Gesture Arena

How to handle gesture from multiple recognizers

AND

# Listener

Listener

GestureDetector

RawGestureDetector

Listener

Lowest gestures handling API

deals with pointer events

Listener

GestureDetector

GestureRecognizer

<<interface>>

RawGestureDetector

...

# Custom GestureRecognizer

TapGestureRecognizer

DoubleTapGestureRecognizer

LongPressGestureRecognizer

HorizontalDragGestureRecognizer

VerticalDragGestureRecognizer

PanGestureRecognizer

ScaleGestureRecognizer

Listener

GestureDetector

Listener

# Listener

Listener

Listener vs GestureDetector ?

Listener

Handle raw pointer events


return Listener(
  child: child,
  onPointerDown: (PointerDownEvent event) => {}
  onPointerMove: (PointerMoveEvent event) => {}
  onPointerUp: (PointerUpEvent event) => {}
  onPointerSignal: (PointerSignalEvent event) => {}
  onPointerHover: (PointerHoverEvent event) => {}
  onPointerCancel: (PointerCancelEvent event) => {}
  behavior: HitTestBehavior.deferToChild|translucent|opaque,
);

			
# Listener

return Scaffold(,
  floatingActionButton: FloatingActionButton(
    child: const Icon(...),
    onPressed: () {...}
  ),
);
        
# Listener

Listener

Detect gestures events without disturbance


return Listener(
  child: Scaffold(,
    floatingActionButton: FloatingActionButton(
      child: const Icon(...),
      onPressed: () {...}
    ),
  ),
);
        
# Listener

Listener

Detect gestures events without disturbance

return Listener(
  /// Handle any pointer related events from the Listener...
  onPointerDown: (PointerDownEvent event) => {}
  onPointerMove: (PointerMoveEvent event) => {}
  onPointerUp: (PointerUpEvent event) => {}
  child: Scaffold(,
    floatingActionButton: FloatingActionButton(
      child: const Icon(...),
      /// ...and handle any regular gesture normally
      onPressed: () {...}
    ),
  ),
);
        
# Listener

Listener

Detect gestures events without disturbance

# Listener

Listener

Detect gestures events without disturbance

How to Test?

# Gesture Testing

WidgetTester

WidgetController

Simulate high level gestures in a testing environment 

Testing

High level WidgetTester API : classic gestures

testWidgets(
  'should call "onTap" when receiving a tap gesture',
  (WidgetTester tester) async {
    final tapCallback = MockGestureCallback();
    await tester.pumpWidget(GestureDetector(
      onTap: tapCallback,
    ));
    
    await tester.tap(find.byType(GestureDetector));

    verify(() =>tapCallback()).called(1);
  },
);
			
tap(Finder finder);
longPress(Finder finder);
drag(Finder finder, Offset offset);
fling(Finder finder, Offset offset,
double speed);

Emulate gestures on Widgets

Emulate gestures at/from locations

tapAt(Offset position);
longPressAt(Offset position);
dragFrom(Offset position, Offset offset);
flingFrom(Offset position, Offset offset,
double speed);
			
# Gesture Testing
# Gesture Testing

WidgetTester

WidgetController

Simulate high level gestures in a testing environment 

TestGesture

+ createGesture

Higher level test gesture API

Lower level test gesture API

...

+ down

+ up

+ moveBy

+ moveTo

+ tap

+ longPress

+ drag

+ fling

+ tapAt

+ longPressAt

+ dragFrom

+ flingFrom

testWidgets(
  'should call "onVerticalDrag..." callbacks when receiving a drag gesture oriented vertically',
  (WidgetTester tester) async {
    final gestureCallback = MockGestureCallback();
    await tester.pumpWidget(GestureDetector(
      onVerticalDragStart: gestureCallback,
      onVerticalDragUpdate: gestureCallback,
      onVerticalDragEnd: gestureCallback,
    ));

    const startPosition = Offset(100, 100);    
    final gesture = await tester.startGesture(startPosition);
    gesture.moveBy(const  Offset(0, 100));
    gesture.up();

    verify(() =>gestureCallback(any())).called(3);
  },
);
			
# Gesture Testing

Testing

Low level WidgetTester API : custom gestures

TestWidgetsFlutterBinding

Test gestures can be recognized by any detector that needs to be tested

WidgetTester

WidgetController

Simulate high level gestures in a testing environment 

GestureBinding

+ handlePointerEvent

+ sendEventToBinding

....

TestGesture

+ createGesture

Higher level test gesture API

Lower level test gesture API

All test gestures end up in the GestureBinding's handlePointerEvent

...

+ down

+ up

+ moveBy

+ moveTo

GestureDetector

RawGestureDetector

Listener

# Gesture Testing

+ tap

+ longPress

+ drag

+ fling

+ tapAt

+ longPressAt

+ dragFrom

+ flingFrom

Wrap up

GestureDetector

GestureRecognizer

<<interface>>

RawGestureDetector

CustomGestureRecognizer

Listener

Detect most usual gestures

Detect & Recognize any custom gestures

Handle multiple gestures simultaneously without interfering

Test usual & custom gestures

with a high & low level gesture testing API

WidgetTester

# Wrap up

Useful links

  • Gesture Disambiguation documentation
  • https://github.com/guitoof/gestures_handling_sandbox
  • Issue #74733: HitTestBehavior.opaque documentation is wrong (@Hixie's answer)
  • GestureBinding documentation

Thank you

@Guitoof

@guitoof

Understanding Flutter's handling of Gestures - Flutter Vikings 2022

By Guillaume Diallo-Mulliez

Made with Slides.com

Understanding Flutter's handling of Gestures - Flutter Vikings 2022

  • 380

More from Guillaume Diallo-Mulliez