Understanding Flutter's
handling of Gestures
Guillaume
Diallo-Mulliez
Head of Flutter Engineering
Classic gestures handling
# GestureDetector
Gesture
Detector
GestureDetector
Higher level API for gesture detection
GestureDetector
Gesture Detector
Tap
# GestureDetector
Double Tap
Horizontal Drag
Long Press
Vertical Drag
Tap
Long Press
Horizontal Drag
Vertical Drag
Double Tap
Scale
Force Press
# GestureDetector
Pan
Need to handle some other custom gestures ?
# RawGestureDetector
Raw
Detector
GestureDetector
RawGestureDetector
RawGestureDetector
Detect gestures from custom gesture recognizers
Gesture
GestureDetector
GestureRecognizer
<<interface>>
RawGestureDetector
...
# Custom GestureRecognizer
TapGestureRecognizer
DoubleTapGestureRecognizer
LongPressGestureRecognizer
HorizontalDragGestureRecognizer
VerticalDragGestureRecognizer
PanGestureRecognizer
ScaleGestureRecognizer
Gesture detection/recognition
flow
Pointers & Events
Gesture events life cycle
# Gesture Detection Flow
-
PointerDownEvent
-
PointerMoveEvent
-
PointerUpEvent
-
PointerRemovedEvent
-
PointerAddedEvent
Pointer Events
+ hitTest
GestureBinding
+ handlePointerEvent
1
PlatformDispatcher
Entry point for
platform related events
1
Propagate Hit Test
# Gesture Detection Flow
Hit Testing
Hit testing down the Render Tree
hitTest ?
hitTest ?
hitTest ?
HitTestResult =
# Gesture Detection Flow
Pointer Events
+ hitTest
handleEvent()
handleEvent()
RenderPointerListener
GestureBinding
+ handlePointerEvent
1
2
Call handleEvent on each target
Listener
RawGestureDetector
3
<Any>GestureRecognizer
+ addPointer
_handlePointerDown
Call addPointer on each recognizer
Register route
PointerRouter
+ addRoute
+ dispatchEvent
+ handleEvent
Call all registered routes
<_handleAnyPointerEvent>
Actually handle event
4
RendererBinding
Propagate hitTest through the render tree
+ hitTest
PlatformDispatcher
Entry point for
platform related events
1
Propagate Hit Test
2
Dispatch Event
3
Register pointer events routes
4
Call registered routes to handles the event from recognizers
+ handleEvent
<<HitTestTarget>>
# Gesture Detection Flow
Custom Gesture Recognizer
GestureDetector
GestureRecognizer
<<interface>>
TapGestureRecognizer
RawGestureDetector
...
CustomGestureRecognizer
# Custom GestureRecognizer
DoubleTapGestureRecognizer
LongPressGestureRecognizer
HorizontalDragGestureRecognizer
VerticalDragGestureRecognizer
PanGestureRecognizer
ScaleGestureRecognizer
class CustomGestureRecognizer extends GestureRecognizer {
}
Custom GestureRecognizer
Implement
# Custom GestureRecognizer
class CustomGestureRecognizer extends GestureRecognizer {
@override
void acceptGesture (int pointer) {}
@override
void rejectGesture (int pointer) {}
@override
String get debugDescription () => 'Custom Gesture';
}
# Custom GestureRecognizer
Custom GestureRecognizer
Implement
class CustomGestureRecognizer extends GestureRecognizer {
@override
void addAllowedPointer (PointerEvent event) {
GestureBinding .instance .pointerRouter .addRoute (
event .pointer , _handleEvent , event.transform
);
}
void _handleEvent (PointerEvent event) {
if (event is PointerDownEvent ) {
/// handle pointer down event
} else if (...) {
/// or handle any other pointer event as you need here
}
}
...
@override
void acceptGesture (int pointer) {}
@override
void rejectGesture (int pointer) {}
@override
String get debugDescription () => 'Custom Gesture';
}
# Custom GestureRecognizer
Custom GestureRecognizer
Implement
class CustomGestureRecognizer extends GestureRecognizer {
void Function(CustomGestureDetails details)? onCustomGestureDetected ;
@override
void addAllowedPointer (PointerEvent event) {
GestureBinding .instance .pointerRouter .addRoute (
event .pointer , _handleEvent , event.transform
);
}
void _handleEvent (PointerEvent event) {
if (event is PointerDownEvent ) {
/// handle pointer down event
} else if (...) {
/// or handle any other pointer event as you need here
} else if (event is PointerUpEvent ) {
onCustomGestureDetected ?.call(event);
}
}
...
@override
void acceptGesture (int pointer) {}
@override
void rejectGesture (int pointer) {}
@override
String get debugDescription () => 'Custom Gesture';
}
# Custom GestureRecognizer
Custom GestureRecognizer
Implement
...
return RawGestureDetector (
gestures : {
CustomGestureRecognizer : GestureRecognizerFactoryWithHandlers <CustomGestureRecognizer >(
() => CustomGestureRecognizer (),
(CustomGestureRecognizer instance) {
instance.onCustomGestureDetected = ( CustomGestureDetails details) {
/// Do stuff to handle the custom gesture here...
}
},
),
}
);
# Custom GestureRecognizer
Custom GestureRecognizer
Use with a RawGestureDetector
👇👇👇 TripleTapGestureRecognizer
🔄 RotationGestureRecognizer
return RawGestureDetector (
gestures : {
TripleTapGestureRecognizer : GestureRecognizerFactoryWithHandlers <CustomGestureRecognizer >(
() => TripleTapGestureRecognizer (),
(TripleTapGestureRecognizer instance) {
instance.onTripleTap = ( TapUpDetails details) {...}
),
RotationGestureRecognizer : GestureRecognizerFactoryWithHandlers <CustomGestureRecognizer >(
() => RotationGestureRecognizer (),
(RotationGestureRecognizer instance) {
instance.onRotation = ( RotationDetails details) {...}
},
),
}
);
# Custom GestureRecognizer
Custom GestureRecognizer
Examples
What happens with multiple targets ?
Hit Testing
Hit Test Behavior
1. Should I pass the hit test through to my children?
2. Should I add myself to the list of widgets that were hit?
3. Should I tell my parent that I considered the hit test to hit me or my children?
deferToChild
translucent
opaque
# Hit Test Behavior
Hit Testing
Actual: Hit Test Behavior
HitTestResult =
RenderObject 1
RenderObject 2
deferToChild
transluscent
opaque
deferToChild
transluscent
opaque
# Hit Test Behavior
Hit Testing
Hit Test Behavior: example
deferToChild
transluscent
opaque
Stack
A
B
Circle
Stack
A
B
Circle (opaque)
HitTestBehavior for B
# Hit Test Behavior
Who actually handles the event?
OR
Disambiguation
Gesture Arena
A
B
HitTestResult =
GestureArena
class AnyGestureRecognizer extends GestureRecognizer {
...
GestureBinding .instance .gestureArena .add (pointer, this);
...
}
class AnyGestureRecognizer extends GestureRecognizer {
...
gestureArenaEntry .resolve (GestureDisposition .accepted |rejected );
...
}
1 winner
OR
# Gesture Arena
How to handle gesture from multiple recognizers
AND
# Listener
Listener
GestureDetector
RawGestureDetector
Listener
Lowest gestures handling API
deals with pointer events
Listener
GestureDetector
GestureRecognizer
<<interface>>
RawGestureDetector
...
# Custom GestureRecognizer
TapGestureRecognizer
DoubleTapGestureRecognizer
LongPressGestureRecognizer
HorizontalDragGestureRecognizer
VerticalDragGestureRecognizer
PanGestureRecognizer
ScaleGestureRecognizer
Listener
GestureDetector
Listener
# Listener
Listener
Listener vs GestureDetector ?
Listener
Handle raw pointer events
return Listener (
child : child ,
onPointerDown : (PointerDownEvent event) => {}
onPointerMove : (PointerMoveEvent event) => {}
onPointerUp : (PointerUpEvent event) => {}
onPointerSignal : (PointerSignalEvent event) => {}
onPointerHover : (PointerHoverEvent event) => {}
onPointerCancel : (PointerCancelEvent event) => {}
behavior : HitTestBehavior .deferToChild |translucent |opaque ,
);
# Listener
return Scaffold ( ,
floatingActionButton : FloatingActionButton (
child : const Icon (...),
onPressed : () {...}
),
);
# Listener
Listener
Detect gestures events without disturbance
return Listener (
child : Scaffold ( ,
floatingActionButton : FloatingActionButton (
child : const Icon (...),
onPressed : () {...}
),
),
);
# Listener
Listener
Detect gestures events without disturbance
return Listener (
/// Handle any pointer related events from the Listener...
onPointerDown : (PointerDownEvent event) => {}
onPointerMove : (PointerMoveEvent event) => {}
onPointerUp : (PointerUpEvent event) => {}
child : Scaffold ( ,
floatingActionButton : FloatingActionButton (
child : const Icon (...),
/// ...and handle any regular gesture normally
onPressed : () {...}
),
),
);
# Listener
Listener
Detect gestures events without disturbance
# Listener
Listener
Detect gestures events without disturbance
How to Test?
# Gesture Testing
WidgetTester
WidgetController
Simulate high level gestures in a testing environment
Testing
High level WidgetTester API : classic gestures
testWidgets (
'should call "onTap" when receiving a tap gesture',
(WidgetTester tester) async {
final tapCallback = MockGestureCallback ();
await tester. pumpWidget (GestureDetector (
onTap : tapCallback,
));
await tester. tap (find.byType (GestureDetector ));
verify(() => tapCallback ()).called (1);
},
);
tap (Finder finder);
longPress (Finder finder);
drag (Finder finder, Offset offset);
fling (Finder finder, Offset offset,
double speed);
Emulate gestures on Widgets
Emulate gestures at/from locations
tapAt (Offset position);
longPressAt (Offset position);
dragFrom (Offset position, Offset offset);
flingFrom (Offset position, Offset offset,
double speed);
# Gesture Testing
# Gesture Testing
WidgetTester
WidgetController
Simulate high level gestures in a testing environment
TestGesture
+ createGesture
Higher level test gesture API
Lower level test gesture API
...
+ down
+ up
+ moveBy
+ moveTo
+ tap
+ longPress
+ drag
+ fling
+ tapAt
+ longPressAt
+ dragFrom
+ flingFrom
testWidgets (
'should call "onVerticalDrag..." callbacks when receiving a drag gesture oriented vertically',
(WidgetTester tester) async {
final gestureCallback = MockGestureCallback ();
await tester. pumpWidget (GestureDetector (
onVerticalDragStart : gestureCallback,
onVerticalDragUpdate : gestureCallback,
onVerticalDragEnd : gestureCallback,
));
const startPosition = Offset (100, 100);
final gesture = await tester. startGesture (startPosition);
gesture. moveBy (const Offset (0, 100));
gesture. up ();
verify(() => gestureCallback (any())).called (3);
},
);
# Gesture Testing
Testing
Low level WidgetTester API : custom gestures
TestWidgetsFlutterBinding
Test gestures can be recognized by any detector that needs to be tested
WidgetTester
WidgetController
Simulate high level gestures in a testing environment
GestureBinding
+ handlePointerEvent
+ sendEventToBinding
....
TestGesture
+ createGesture
Higher level test gesture API
Lower level test gesture API
All test gestures end up in the GestureBinding's handlePointerEvent
...
+ down
+ up
+ moveBy
+ moveTo
GestureDetector
RawGestureDetector
Listener
# Gesture Testing
+ tap
+ longPress
+ drag
+ fling
+ tapAt
+ longPressAt
+ dragFrom
+ flingFrom
Wrap up
GestureDetector
GestureRecognizer
<<interface>>
RawGestureDetector
CustomGestureRecognizer
Listener
Detect most usual gestures
Detect & Recognize any custom gestures
Handle multiple gestures simultaneously without interfering
Test usual & custom gestures
with a high & low level gesture testing API
WidgetTester
# Wrap up
Useful links
Thank you
Understanding Flutter's handling of Gestures - Flutter Vikings 2022
By Guillaume Diallo-Mulliez
Understanding Flutter's handling of Gestures - Flutter Vikings 2022
- 367