Understanding Flutter's

handling of Gestures

Guillaume DIALLO

Head of

Flutter Engineering

1

Handling Classic Gestures

2

Handling
Custom Gestures

3

Overlapping Gesture Handlers

2.2

Writing a custom GestureRecognizer

2.1

The Gesture
Detection Flow

4

Testing Gestures

3.1

HitTest
behaviors

3.2

Gesture Disambiguation

Classic Gestures Handling

# GestureDetector

Gesture

 

Detector

GestureDetector

Higher level API for gesture detection

GestureDetector

Gesture Detector

Tap

# GestureDetector

Double Tap

Horizontal Drag

Long Press

Vertical Drag

Tap

Long Press

Horizontal Drag

Vertical Drag

Double Tap

Scale

Force Press

# GestureDetector

Pan

Handling Custom Gestures

# RawGestureDetector

Raw

Detector

GestureDetector

RawGestureDetector

RawGestureDetector

Detect gestures from custom gesture recognizers

Gesture

 

GestureDetector

GestureRecognizer

<<interface>>

RawGestureDetector

...

# Custom GestureRecognizer

TapGestureRecognizer

DoubleTapGestureRecognizer

LongPressGestureRecognizer

HorizontalDragGestureRecognizer

VerticalDragGestureRecognizer

PanGestureRecognizer

ScaleGestureRecognizer

GestureDetector

GestureRecognizer

<<interface>>

TapGestureRecognizer

RawGestureDetector

...

CustomGestureRecognizer

# Custom GestureRecognizer

DoubleTapGestureRecognizer

LongPressGestureRecognizer

HorizontalDragGestureRecognizer

VerticalDragGestureRecognizer

PanGestureRecognizer

ScaleGestureRecognizer

Gesture detection/recognition

flow

Pointers & Events

Gesture events life cycle

# Gesture Detection Flow
  • PointerDownEvent

  • PointerMoveEvent

  • PointerUpEvent

  • PointerRemovedEvent

  • PointerAddedEvent

Pointer Events

+ hitTest

GestureBinding

+ handlePointerEvent

1

PlatformDispatcher

Entry point for

platform related events

1

Propagate Hit Test

# Gesture Detection Flow

Hit Testing

Hit testing down the Render Tree

hitTest ?

hitTest ?

hitTest ?

HitTestResult =

# Gesture Detection Flow

Pointer Events

+ hitTest

handleEvent()

handleEvent()

RenderPointerListener

GestureBinding

+ handlePointerEvent

1

2

Call handleEvent on each target

Listener

RawGestureDetector

3

<Any>GestureRecognizer

+ addPointer

_handlePointerDown

Call addPointer on each recognizer

Register route 

PointerRouter

+ addRoute

+ dispatchEvent

+ handleEvent

Call all registered routes

<_handleAnyPointerEvent>

Actually handle event

4

RendererBinding

Propagate hitTest through the render tree

+ hitTest

PlatformDispatcher

Entry point for

platform related events

1

Propagate Hit Test

2

Dispatch Event

3

Register pointer events routes

4

Call registered routes to handles the event from recognizers

+ handleEvent

<<HitTestTarget>>

# Gesture Detection Flow

Custom Gesture Recognizer

# Custom GestureRecognizer

Custom GestureRecognizer

V-Shape Gesture Recognizer

V


class VShapeGestureRecognizer extends GestureRecognizer { 
}
			

Custom GestureRecognizer

Implement

# Custom GestureRecognizer
# Custom GestureRecognizer

Custom GestureRecognizer

Implement


class VShapeGestureRecognizer extends GestureRecognizer {

  @override
  void acceptGesture(int pointer) {}
  @override
  void rejectGesture(int pointer) {}
  
}
			

class VShapeGestureRecognizer extends GestureRecognizer {
  @override 
  void addAllowedPointer(PointerEvent event) {
    GestureBinding.instance.pointerRouter.addRoute(
      event.pointer, _handleEvent, event.transform
    );
  }
  
  void _handleEvent(PointerEvent event) {
    if(event is PointerDownEvent) {
      /// handle pointer down event
    } else if (...) {
      /// or handle any other pointer event as you need here
    }
  }
  
  ...

  @override
  void acceptGesture(int pointer) {}
  @override
  void rejectGesture(int pointer) {}
  
}
			
# Custom GestureRecognizer

Custom GestureRecognizer

Register route to handle pointer events


class VShapeGestureRecognizer extends GestureRecognizer {
  final _tracker = ShapeTracker<VShape>();

  @override 
  void addAllowedPointer(PointerEvent event) {
    GestureBinding.instance.pointerRouter.addRoute(
      event.pointer, _handleEvent, event.transform
    );
  }
  
  void _handleEvent(PointerEvent event) {
    if(event is PointerDownEvent || event is PointerMoveEvent) {
      _tracker.track(event);
    } else if (...) {
      /// or handle any other pointer event as you need here
    }
  }
  
  ...

  @override
  void acceptGesture(int pointer) {}
  @override
  void rejectGesture(int pointer) {}
  
}
			
# Custom GestureRecognizer

Custom GestureRecognizer

Add some V-shape tracker


class VShapeGestureRecognizer extends GestureRecognizer {

  final _tracker = ShapeTracker<VShape>();

  void Function(VShapeDetails details)? onVShapeDetected;

  @override 
  void addAllowedPointer(PointerEvent event) {
    GestureBinding.instance.pointerRouter.addRoute(
      event.pointer, _handleEvent, event.transform
    );
  }
  
  void _handleEvent(PointerEvent event) {
    if(event is PointerDownEvent || event is PointerMoveEvent) {
      _tracker.track(event);
    } else if(event is PointerUpEvent && _tracker.hasValidShape) {
      onVShapeDetected?.call(_tracker.shapeDetails);
    }
  }
  
  ...

  @override
  void acceptGesture(int pointer) {}
  @override
  void rejectGesture(int pointer) {}
  @override
  String get debugDescription () => 'Custom Gesture';
  
}
			
# Custom GestureRecognizer

Custom GestureRecognizer

Trigger callback if tracked shape is valid on pointer Up event


...
return RawGestureDetector(
  gestures: {
    VShapeGestureRecognizer: GestureRecognizerFactoryWithHandlers<VShapeGestureRecognizer>(
      () => VShapeGestureRecognizer(),
      (VShapeGestureRecognizer instance) {
        instance.onVShapeDetected = ( VShapeDetails details) {
          /// Do stuff to handle the detected V-shape here...
    	}
      },
    ),
  }
);
			
# Custom GestureRecognizer

Custom GestureRecognizer

Use with a RawGestureDetector

# Custom GestureRecognizer

Custom GestureRecognizer

V-Shape Gesture Recognizer

Overlapping

Gesture handlers

Button
Triggers some action in the app

Transparent Gesture
Handler layer

Trigger some Analytics event

Button
Triggers some action in the app

Transparent Gesture
Handler layer

✅ Trigger some Analytics event

# Overlapping Gesture Handlers

What happens with multiple targets ?

Hit Testing

Hit Test Behavior

1. Should I pass the hit test through to my children?

2. Should I add myself to the list of widgets that were hit?

3. Should I tell my parent that I considered the hit test to hit me or my children?

deferToChild

translucent

opaque

# Hit Test Behavior

Hit Testing

Actual: Hit Test Behavior

HitTestResult = 

RenderObject 1

RenderObject 2

deferToChild

transluscent

opaque

deferToChild

transluscent

opaque

# Hit Test Behavior

Hit Testing

Hit Test Behavior: example

deferToChild

transluscent

opaque

Stack

A

B

Circle

Stack

A

B

Circle (opaque)

HitTestBehavior for B

# Hit Test Behavior

Hit Testing

Hit Test Behavior: example

deferToChild

transluscent

opaque

HitTestBehavior

for B

# Hit Test Behavior

Who actually handles the event?

OR

Disambiguation

Gesture Arena

A

B

HitTestResult =

GestureArena

class AnyGestureRecognizer extends GestureRecognizer {
  ...
  GestureBinding.instance.gestureArena.add(pointer, this);
  ...
}
			
class AnyGestureRecognizer extends GestureRecognizer {
  ...
  gestureArenaEntry.resolve(GestureDisposition.accepted|rejected);
  ...
}
			

1 winner

OR

# Gesture Arena

Disambiguation

Gesture Arena

# Gesture Arena

Craig Labenz
Gesture Arena | Decoding Flutter

How to handle gesture from multiple recognizers ?

AND

# Listener

Listener

GestureDetector

RawGestureDetector

Listener

Lowest gestures handling API

deals with pointer events

Listener

GestureDetector

GestureRecognizer

<<interface>>

RawGestureDetector

...

# Custom GestureRecognizer

TapGestureRecognizer

DoubleTapGestureRecognizer

LongPressGestureRecognizer

HorizontalDragGestureRecognizer

VerticalDragGestureRecognizer

PanGestureRecognizer

ScaleGestureRecognizer

Listener

GestureDetector

Listener

# Listener

Listener

Listener vs GestureDetector ?

Listener

Handle raw pointer events


return Listener(
  child: child,
  onPointerDown: (PointerDownEvent event) => {}
  onPointerMove: (PointerMoveEvent event) => {}
  onPointerUp: (PointerUpEvent event) => {}
  onPointerSignal: (PointerSignalEvent event) => {}
  onPointerHover: (PointerHoverEvent event) => {}
  onPointerCancel: (PointerCancelEvent event) => {}
  behavior: HitTestBehavior.deferToChild|translucent|opaque,
);

			
# Listener

return Scaffold(,
  floatingActionButton: FloatingActionButton(
    child: const Icon(...),
    onPressed: () {...}
  ),
);
        
# Listener

Listener

Detect gestures events without disturbance


return Listener(
  child: Scaffold(,
    floatingActionButton: FloatingActionButton(
      child: const Icon(...),
      onPressed: () {...}
    ),
  ),
);
        
# Listener

Listener

Detect gestures events without disturbance

return Listener(
  /// Handle any pointer related events from the Listener...
  onPointerDown: (PointerDownEvent event) => {}
  onPointerMove: (PointerMoveEvent event) => {}
  onPointerUp: (PointerUpEvent event) => {}
  child: Scaffold(,
    floatingActionButton: FloatingActionButton(
      child: const Icon(...),
      /// ...and handle any regular gesture normally
      onPressed: () {...}
    ),
  ),
);
        
# Listener

Listener

Detect gestures events without disturbance

# Listener

Listener

Detect gestures events without disturbance

How to Test

Gestures?

# Gesture Testing

WidgetTester

WidgetController

Simulate high level gestures in a testing environment 

Testing

High level WidgetTester API : classic gestures

testWidgets(
  'should call "onTap" when receiving a tap gesture',
  (WidgetTester tester) async {
    final tapCallback = MockGestureCallback();
    await tester.pumpWidget(GestureDetector(
      onTap: tapCallback,
    ));
    
    await tester.tap(find.byType(GestureDetector));

    verify(() =>tapCallback()).called(1);
  },
);
			
tap(Finder finder);
longPress(Finder finder);
drag(Finder finder, Offset offset);
fling(Finder finder, Offset offset,
double speed);
			

Emulate gestures on Widgets

Emulate gestures at/from locations

tapAt(Offset position);
longPressAt(Offset position);
dragFrom(Offset position, Offset offset);
flingFrom(Offset position, Offset offset,
double speed);
			
# Gesture Testing
# Gesture Testing

WidgetTester

WidgetController

Simulate high level gestures in a testing environment 

TestGesture

+ createGesture

Higher level test gesture API

Lower level test gesture API

...

+ down

+ up

+ moveBy

+ moveTo

+ tap

+ longPress

+ drag

+ fling

+ tapAt

+ longPressAt

+ dragFrom

+ flingFrom

testWidgets(
  'should call "onVerticalDrag..." callbacks when receiving a drag gesture oriented vertically',
  (WidgetTester tester) async {
    final gestureCallback = MockGestureCallback();
    await tester.pumpWidget(GestureDetector(
      onVerticalDragStart: gestureCallback,
      onVerticalDragUpdate: gestureCallback,
      onVerticalDragEnd: gestureCallback,
    ));

    const startPosition = Offset(100, 100);    
    final gesture = await tester.startGesture(startPosition);
    gesture.moveBy(const  Offset(0, 100));
    gesture.up();

    verify(() =>gestureCallback(any())).called(3);
  },
);
			
# Gesture Testing

Testing

Low level WidgetTester API : custom gestures

TestWidgetsFlutterBinding

Test gestures can be recognized by any detector that needs to be tested

WidgetTester

WidgetController

Simulate high level gestures in a testing environment 

GestureBinding

+ handlePointerEvent

+ sendEventToBinding

....

TestGesture

+ createGesture

Higher level test gesture API

Lower level test gesture API

All test gestures end up in the GestureBinding's handlePointerEvent

...

+ down

+ up

+ moveBy

+ moveTo

GestureDetector

RawGestureDetector

Listener

# Gesture Testing

+ tap

+ longPress

+ drag

+ fling

+ tapAt

+ longPressAt

+ dragFrom

+ flingFrom

Wrap up

GestureDetector

GestureRecognizer

<<interface>>

RawGestureDetector

CustomGestureRecognizer

Listener

Detect most usual gestures

Detect & Recognize any custom gestures

Handle multiple gestures simultaneously without interfering

Test usual & custom gestures

with a high & low level gesture testing API

WidgetTester

# Wrap up

Useful links

Thank you

Questions ?

Understanding Flutter's handling of Gestures - FlutterCon Berlin

By Guillaume Diallo-Mulliez

Understanding Flutter's handling of Gestures - FlutterCon Berlin

  • 80