Lindsay Kay
BioDigital Tech Talk
September 22, 2017
Picking
3D Objects
within
Human
Four Flavors of Picking
Human's picking system supports four types of picking:
- Pick object at canvas coords
- Pick object surface position at canvas coords
- Pick object with arbitrary 3D ray
- Pick object surface position with arbitrary 3D ray
Flavor 1: Pick Object at Canvas Coords
Find object at mouse coordinates:
Flavor 1: Pick Object at Canvas Coords (cont)
Usage:
var hit = Human.renderer.pick({
canvasPos: [500,400]
});
if (hit) { // Picked an object
var object = Human.scene.objects[hit.objectId];
//...
}
Flavor 1: Pick Object at Canvas Coords (cont)
Algorithm:
- User picks at canvas coordinates.
- Render scene to auxiliary image buffer, filling each object with a unique color that is the RBGA-encoded index of its position within the internal display list.
- Read color from the image buffer at the canvas coordinates, map the color back to the object in the display list.
- Return a hit result containing ID of the object.
Flavor 2: Pick Object Surface at Canvas Coords
Find surface intersection at mouse coordinates:
Flavor 2: Pick Object Surface at Canvas Coords (cont)
var hit = Human.renderer.pick({
canvasPos: [500,400],
pickSurface: true, // <<----- Indicates that we want to pick on surface
});
if (hit) { // Picked an object
var object = hit.objectId; // ID of object we picked
var primitive = hit.primitive; // Type of primitive, usually "triangles"
var primIndex = hit.primIndex; // Index of triangle within object's indices
var indices = hit.indices; // Value of each of the triangle's indices
var localPos = hit.localPos; // Local-space coordinates
var worldPos = hit.worldPos; // World-space coordinates
var viewPos = hit.viewPos; // View-space coordinates
var bary = hit.bary; // Barycentric coordinates within the triangle
var normal = hit.normal; // Interpolated normal vector within the triangle
var uv = hit.uv; // Interpolated UVs within the triangle
}Usage:
Flavor 2: Pick Object Surface at Canvas Coords (cont)
Algorithm:
- User ray-picks at canvas coordinates.
- Render scene to auxiliary image buffer, filling each object with unique color that is the RBGA-encoded index of its position within the display list.
- Read color from image buffer at canvas coords, map to object in the display list.
- Clear image buffer, render object's triangles, filling each with unique color as RBGA-encoded index of triangle within object’s geometry.
- Read color from image buffer at canvas coords, map to the picked triangle.
- Make ray originating at eye and passing through near projection plane where we picked, then unproject to get ray in object’s local coordinate space.
- Find local-space intersection of ray with triangle.
Flavor 3: Pick Objects with Arbitrary Ray
Same as Flavor 1, but using a ray in 3D space:
Flavor 3: Pick Objects with Arbitrary Ray (cont)
var hit = Human.renderer.pick({
origin: [0,0,-5], // Ray origin
direction: [0,0,1], // Ray direction
pickSurface: false // Don't want intersect info
});
if (hit) { // Picked an object with the ray
var object = Human.scene.objects[hit.objectId]
}Usage:
Flavor 3: Pick Objects with Arbitrary Ray (cont)
Algorithm:
- User picks using a ray in World-space.
- Position the camera to look along the ray.
- Render the scene to an auxiliary image buffer, filling each entity with a unique color that is the RBGA-encoded index of its position within the internal display list.
- Restore the camera to its previous position.
- Read the color from the image buffer at the canvas coordinates, map the color back to an object in the display list.
- Return a hit result containing the object.
Flavor 4: Pick Object Surfaces with Arbitrary Ray
Same as Flavor 2, but using a ray in 3D space:
Flavor 4: Pick Object Surfaces with Arbitrary Ray (cont)
var hit = Human.renderer.pick({
origin: [0,0,-5], // Ray origin
direction: [0,0,1], // Ray direction
pickSurface: true // Pick on surface
});
if (hit) { // Picked an object with the ray
var objectId = hit.objectId;
var primitive = hit.primitive;
var primIndex = hit.primIndex;
var indices = hit.indices;
var localPos = hit.localPos;
var worldPos = hit.worldPos;
var viewPos = hit.viewPos;
var bary = hit.bary;
var normal = hit.normal;
var uv = hit.uv;
}Usage:
Flavor 4: Pick Object Surfaces with Arbitrary Ray (cont)
Algorithm:
- User picks using a ray in World-space.
- Position camera to look along the ray.
- Render scene to image buffer, filling each entity with unique color that is RBGA-encoded index of its position within display list.
- Restore camera to previous position.
- Read color from image buffer at canvas coord, map to object in display list.
- Clear image buffer, render triangles of object, each with unique color mapping to triangle within geometry indices.
- Read color from the image buffer at the canvas coords, map back to the triangle.
- Restore camera to previous position.
- With object and the triangle, make ray in clip-space, originating at the eye position and passing through the near projection plane at a position corresponding to where we picked, then unproject that ray to get a ray in the entity’s local coordinate space.
- Find the intersection of the ray with the triangle in local space.
- Find the barycentric coordinates of the local-space intersection, then use those to interpolate within the triangle to find the normal vector and UV coordinates at that position.
- Return a hit result containing the picked object, the triangle, and the ray-triangle intersection info.
Flavor 4: Pick Object Surfaces with Arbitrary Ray (cont)
Advantages:
- Linear time complexity O(n) - time taken increases linearly with increase in number of objects and triangles
- Done mostly on GPU - way faster than JS on CPU
Disadvantages:
- Only finds closest intersection
- Requires dedicated lazy-computed vertex colors and indices to support individually-colored triangles for picking