Loading

Graph Training 3

XN Logic Corporation

This is a live streamed presentation. You will automatically follow the presenter and see the slide they're currently on.

Advanced Graph Traversal Using Pacer

Brought to you by

Routes

Collections/streams

  • Composable
  • Lazily-loaded
  • Reusable
  • Implement Ruby's Enumerable interface

One of Pacer's most fundamental building blocks

Routes

Two phases:

A route can be executed multiple times

  1. Build
  2. Execute

Routes

We have seen some basic routes:

  • out_e, in_e, both_e
  • out_v, in_v, both_v

These routes are used for traversals.

Next, we will look at routes that are used for filtering ...

Filters - Property Match

filter(foo: 'a')
filter(foo: 'a', bar: 'b')
filter(foo: Set['a', 'b'])
filter(foo: Set['a', 'b'], bar: 'c')

foo is 'a' and bar is 'b'

foo is 'a' or 'b'

foo is 'a' or 'b'

and bar is 'c'.

foo is 'a'

Filters - Where Condition

g.v.where("type = 'person' and age > 21")
g.e.where("timestamp > :t", {t: (Time.now - 60) } )

 * Safe way to handle user input and avoid injection attacks.

< > <= >= == !=       # comparisons
=                     # used as a comparison where syntactically allowed
and or not && || !    # boolean logic
+ - * / %             # simple mathematical expressions
( )                   # expression grouping
:symbol               # symbols are replaced by user values
123 123.45            # numeric constants
"abc" 'abc'           # string constants
true false nil [] {}  # boolean, nil, array, or hash constants

The condition is a Ruby boolean expression that uses:

Optimized, runs as a traversal

Filters - Block Of Code

# Find vertices with palindromic names
g.v.filter {|v| v[:name] == v[:name].reverse }

Filter each element based on the result of a block of code.

Filters

Property Match

Where Condition

Block of Code



filter(foo: 'a')


where("age > 21")

filter do |v|
    depends_on(v)
end

Fastest

Fast

Slower

Least expressive

expressive

Most

expressive

Exercise

Take a look at the Pacer documentation.

You may find some useful route methods that will help you solve this exercise.

Implement the functions in ex2.rb, until you pass all test cases.


# Return (a route containing) posts 10 or more comments

def trending(posts)



    # Your code goes here ...

    

end

# Return (a route containing) posts 10 or more comments

def trending(posts)

    posts.filter do |post|
        post.in_e(:IS_ABOUT)
            .out_v(type: 'comment')
            .count >= 10
    end

end

Problem: Filtering using a block of code is inefficient.


# Return (a route containing) posts 10 or more comments

def trending(posts)

    posts.filter do |post|

        # Decide if we should filter the post ...

    end

end

Filters

Filters - Lookahead

# Return (a route containing) posts 10 or more comments

def trending(posts)

    posts.lookahead(min: 10) do |post|
        post.in_e(:IS_ABOUT).out_v(type: 'comment')
    end

end

Filter items based on a traversal

Exercise

Hints:

  1. Re-use functions from previous exercises.
  2. Before you implement the last function of this exercise, read about negative-lookahead in the lookahead section of the docs.  

Implement the functions in ex3.rb, until you pass all test cases.

only / except
only(collection_or_route)
except(collection_or_route)

* Route argument is evaluated immediately into a Set of elements.

Filter based on a collection of elements

is / is_not
is(element)
is_not(element)

Filter based on a single of elements

Filters - By Elements

Filters - Example

# Return followers of followers of the given user.
# 2nd degree only! The result does not include the 
# user or any of their direct followers.
def followers_of_followers(user)

    followers = user.in_e(:FOLLOWS).out_v

    followers
        .in_e(:FOLLOWS).out_v
        .except(followers)
        .is_not(user)

end

Problem: The followers route gets executed twice:

  1. During the build phase - In order for except to work, Pacer converts the route to a set.
  2. During the execution phase.

Filters

# Return followers of followers of the given user.
# 2nd degree only! The result does not include the 
# user or any of their direct followers.
def followers_of_followers(user)

    user.in_e(:FOLLOWS).out_v
      

      

end
# Return followers of followers of the given user.
# 2nd degree only! The result does not include the 
# user or any of their direct followers.
def followers_of_followers(user)

    user.in_e(:FOLLOWS).out_v
        .as(:followers) 
        .in_e(:FOLLOWS).out_v
        .is_not(:followers)

end
# Return followers of followers of the given user.
# 2nd degree only! The result does not include the 
# user or any of their direct followers.
def followers_of_followers(user)

    user.in_e(:FOLLOWS).out_v
        .as(:followers) 

      

end
# Return followers of followers of the given user.
# 2nd degree only! The result does not include the 
# user or any of their direct followers.
def followers_of_followers(user)

    user.in_e(:FOLLOWS).out_v
        .as(:followers) 
        .in_e(:FOLLOWS).out_v
      

end
# Return followers of followers of the given user.
# 2nd degree only! The result does not include the 
# user or any of their direct followers.
def followers_of_followers(user)

    user.in_e(:FOLLOWS).out_v
        .as(:followers) 
        .in_e(:FOLLOWS).out_v
        .is_not(:followers)
        .is_not(user)
end
# Return followers of followers of the given user.
# 2nd degree only! The result does not include the 
# user or any of their direct followers.
def followers_of_followers(user)

   
    # Your code goes here ...   

      

end

Give this intermediate point of the route the name :followers.

Vertices that went through the :followers point of the route are excluded from the result.

Filters

# Return followers of followers of the given user.
# 2nd degree only! The result does not include the 
# user or any of their direct followers.
def followers_of_followers(user)

    user.as(:me)
        .in_e(:FOLLOWS).out_v.as(:followers) 
        .in_e(:FOLLOWS).out_v
        .is_not(:me).is_not(:followers)

end

One more improvement ...

The user argument can be either a vertex or a route, and the method will run efficiently.

Exercise

Hints: 

  1. Re-use functions from previous exercises.
  2. You can implement complex traversals efficiently by combining a number of the advanced filtering techniques we've seen.

Implement the functions in ex4.rb, until you pass all test cases.

What we've seen so far ...

  • Basic traversal routes
    • out_e, in_e, out_v, etc.
  • Filtering routes
    • Property match, where-condition, block of code
    • lookahead
    • only, except, is, is_not

Next, we will look at advanced traversal routes ...

Traversals - Branching

person
  .branch do |p|
    p.out_e(:POSTED).in_v(type: 'post')
  end
person
person
  .branch do |p|
    p.out_e(:POSTED).in_v(type: 'post')
  end
  .branch do |p|
    p.out_e(:POSTED).in_v(type: 'comment')
     .out_e(:IS_ABOUT).in_v(type: 'post')
  end
person
  .branch do |p|
    p.out_e(:POSTED).in_v(type: 'post')
  end
  .branch do |p|
    p.out_e(:POSTED).in_v(type: 'comment')
     .out_e(:IS_ABOUT).in_v(type: 'post')
  end
  .merge

Traversals - Loops

Looping

Repeating Traversal Patterns

=

Example: For a given post in a social network, get its comment, and their comments, and their comments, and their comments, and so on ...

Traversals - Loops

def get_all_comments(post)
    comments = []
    c = post.in_e(:IS_ABOUT).out_v(type: 'comment')

    while (c.count > 0)
        comments += c.as_a
        c = c.in_e(:IS_ABOUT).out_v(type: 'comment')
    end
end

Naive example:

Q: Can we build (and execute!) a loop as a single route?

Traversals - Loops

.loop { |route| arbitrary_steps(route) }.while { |element, depth| }
.loop { |route| arbitrary_steps(route) }

Traversals - Loops

def get_all_comments(post)

    post
    .loop do |comment_or_post|
        comment_or_post
          .in_e(:IS_ABOUT).out_v(type: 'comment')
    end

    .while do |comment, depth| 
        if depth == 0
            :loop
        else depth <= MAX_DEPTH
            :loop_and_emit
        else
            false
        end
    end

end

Exercise

Implement the functions in ex5.rb, until you pass all test cases.

Traversals - Sections

def sample_posts(people)
    posts = []

    people.each do |person|
        posts += person
                  .out_e(:POSTED).in_v(type: 'post')
                  .limit(3).to_a
    end

    return posts
end

Given a route of people, we want to get their posts.

But ... we want to get at most 3 posts per person.

Problem:   Performance overhead of building routes

Goal:         Do the same using a single route

Traversals - Sections

def sample_posts(people, posts_per_person)
    people.as(:person)
          .out_e(:POSTED).in_v(type: 'post')
          .limit_section(:person, posts_per_person)
end

Using as and limit_section ...

  • As we've seen before, the as method names an intermediate point in the route.
  • Every element that goes through this point of the route "marks a section" of the traversal.
  • Let's see a picture ...

Traversals - Sections

Traversals - Sections

There are a number of methods that take advantage of sections:

  • limit_section
  • sort_section
  • count_section
  • uniq_in_section

The names of these methods are fairly self-descriptive, and you can read more about them in the docs.

Exercise

Implement the functions in ex6.rb, until you pass all test cases.

What's next?

At this point, you know a fair bit about graph traversals and Pacer routes.

Next, we will look at the problem of data modeling, and introduce Pacer extensions.

Extensions & routes go hand in hand:

  • Routes let you traverse the graph efficiently.
  • Extensions let you write these traversals in your own domain-specific language.
Made with Slides.com