Does GoogleBot Scroll?

This meme-free presentation brought to you by:

Adam Terlson

github.com/adamterlson

Creating SEO-friendly Apps with JavaScript (and without Node)

From: Minnesota

We're hiring!

 

adam.terlson@gmail.com

skype adam.terlson.internations

DISCLAIMER

The seo information and seo advice in this presentation is provided “as is” without any representations or warranties, express or implied. I make no representations or warranties in relation to the seo advice and information in this presentation.  You must not rely on the information in this presentation as an alternative to actually testing shit for yourself. If you have any specific questions about any seo matter you should consult your local development provider. You should never delay seeking developer advice, disregard developer advice, or commence or discontinue any development action because of information in this presentation.

I am not an SEO expert.

I do not have all the answers.

Your results may vary.

Instead, this presentation is about how

to fish.

THE MISSION:
RELAUNCH

#1
Rewrite our 40K+ externally-facing websites

#2
Reuse existing client-rendered components

#3
Don't totally mess up our page rank

DECISIONS

Which search engines do we care about?

if (engines.length > 1 || 
    engines[0] !== 'Google') {
    return "You're fucked";
}

Kind of....

Progressive

(SEO) Enhancement

Search Engine

Fundamental content is available for every search engine

Engines that have more JS support get an enhanced "experience"

Target: standards

At InterNations, Google is still king

RESEARCH

What exactly is GoogleBot capable of?

DEPRECATED

TOO NARROW

TOO VAGUE

There's not much out there....

KNOWLEDGE GRAPH & RICH SNIPPETS

The Knowledge Graph is a web of connected semantic-search information.

"Fancy" search results

Source: https://developers.google.com/search/docs/data-types/data-type-selector

Creative Works:

  • Articles
  • Music
  • Recipes
  • Reviews
  • TV & Movies
  • Videos

Commerce:

  • Local Businesses
  • Events
  • Products

 

Others:

  • Bread Crumbs
  • Site-specific Search
  • Social media links
  • Logos

In the beginning....

"Google will ignore content that isn't visible to human users, so you should mark up the text that visitors will see on your web pages."

Source

<div class="hreview">
    <div class="item">
        <div class="fn">
            <strong>Contoso Café</strong>
        </div>
    </div>
    <div class="summary">
        Chef Dan Jump seems to be on top of his game!
    </div>
    <div class="description">
        Contoso Café provides a variety of comfort 
        food made to order with the freshest ingredients.
    </div>
    <div>
        <strong>Reviewed by:</strong> 
        <span class="reviewer">Jennifer Anderson</span> 
        on 
        <span class="dtreviewed">
            November 10, 2010
            <span class="value-title" title="2010-11-10"></span>
        </span>.
    </div>
    <div class="rating">
        <strong>Rating:</strong>
        ★★★☆☆ 
        (<span class="value">3</span> 
         out of <span class="best">5</span>)
    </div>
</div>

Review: Microformats

  • Oldest
  • Most well-known
  • Defines representation and available data types
  • Uses existing HTML attributes (class & ref)
  • Relations and data overlap are hard/impossible
  • Is no longer used
<div vocab="http://schema.org/" typeof="Review">
  <div property="itemReviewed" typeof="Restaurant">
    <img 
        property="image" 
        src="seafood-restaurant.jpg" 
        alt="Catcher in the Rye"/>
    <span property="name">Legal Seafood</span>
  </div>
  <span 
    property="reviewRating" 
    typeof="http://schema.org/Rating">
    <span property="ratingValue">4</span>
  </span> stars -
  <b>"<span property="name">A good seafood place.</span>" </b>
  <span 
    property="author" 
    typeof="http://schema.org/Person">
    <span property="name">Bob Smith</span>
  </span>
  <span 
    property="reviewBody">
    The seafood is great.
  </span>
  <div property="publisher" typeof="Organization">
    <meta property="name" content="Washington Times">
  </div>
</div>

Review: RDFa

(Resource Description Format)
 

  • Linked Data in HTML
  • Uses custom attributes
  • No defined schema types
  • Rather complicated
<div itemscope itemtype="http://schema.org/Review">
  <div 
    itemprop="itemReviewed" 
    itemscope itemtype="http://schema.org/Restaurant">
    <img 
        itemprop="image" 
        src="seafood-restaurant.jpg" 
        alt="Catcher in the Rye"/>
    <span itemprop="name">Legal Seafood</span>
  </div>
  <span 
    itemprop="reviewRating" 
    itemscope itemtype="http://schema.org/Rating">
    <span itemprop="ratingValue">4</span>
  </span> stars -
  <b>"<span itemprop="name">A good seafood place.</span>" </b>
  <span 
    itemprop="author" 
    itemscope itemtype="http://schema.org/Person">
    <span itemprop="name">Bob Smith</span>
  </span>
  <span itemprop="reviewBody">The seafood is great.</span>
  <div 
    itemprop="publisher" 
    itemscope itemtype="http://schema.org/Organization">
    <meta itemprop="name" content="Washington Times">
  </div>
</div>

Review: Microdata

  • "Simpler" Linked Data in HTML
  • No RDF support
  • No defined schema types

Where do we get the defined schema types?

Syntax
"How to speak"
Vocabulary
"What you say"
Microformat
Microdata
RDFa
Schema.org

February, 2016

"[Structured data] should be an accurate reflection of the topic and content."

JSON-LD

Represent semantic data types in a standards-based, linkable JSON format

No longer tied to HTML!

Google knows when our page reflects an event by looking at JSON-LD, regardless of rendering.

<script type="application/ld+json">
{
  "@context": "http://schema.org/",
  "@type": "Review",
  "itemReviewed": {
    "@type": "Restaurant",
    "image": "http://www.example.com/seafood-restaurant.jpg",
    "name": "Legal Seafood"
  },
  "reviewRating": {
    "@type": "Rating",
    "ratingValue": "4"
  },
  "name": "A good seafood place.",
  "author": {
    "@type": "Person",
    "name": "Bob Smith"
  },
  "reviewBody": "The seafood is great.",
  "publisher": {
    "@type": "Organization",
    "name": "Washington Times"
  }
}
</script>

Review: JSON-LD

JSON-LD

IT'S TIME TO EXPERIMENT

Experiment Strat:

  1. Create a page with really unique links and content
  2. Tell GoogleBot to crawl it
  3. Check the index via targeted (quoted) search

If the link/content is found, Pass.  Otherwise, Fail.

Caveats

  • No account for affect on page rank
  • No account for "Link Juice"
  • YMMV

Experimental Control:

A JS-Rendered Link

Features to Test:

Links (variable delay)

AJAX Content (variable delay)

Popovers

Manual Pagination

Automatic Pagination

Breakpoints

InterNations
SEO Test Suite

DEPLOY!

Check Fetch & Render

Fail.

FAIL

Total and complete

Why is the page taking so long to load?!

Lesson #1: Google doesn't support ES6 Promises yet.

DEPLOY

AGAIN!

Verify the control....

FAIL

What about the content...

PASS

WTF

On the subject of Links

Apparently, Google takes a longer time to index JS-rendered links than it does static ones.

What about the other tests?

AJAX Suite
(Delayed content)

4 Second Delayed Content

PASS

7 Second Delayed Content

FAIL

Eventually, the same held true for links

Critical* load time:
~ 5 seconds

 

* This is merely indexation, ignoring any impact on page rank

Breakpoint Suite

@ 1024

PASS

> 1300

FAIL

Don't have content targeting wider than 1024

JS Component Suite

Popover Controls

FAIL

Manual Pagination

FAIL

More Testing Required

Other JS control behavior (modals), full ES6 compatibility, infinite scroll length, automatic redirects via JS, affect upon link juice distribution, etc....

But the most important test of all...

TEST
ROUND 2:
JSON-LD

Can Google render rich snippets from JSON-LD with client-side rendering?

InterNations
SEO Test Suite

Event

<script type="application/ld+json" id="events-0">
[{
  "@context": "http://schema.org",
  "@type": "Event",
  "name": "InterNations London 'Fashion Week' Social @ Mahiki",
  "startDate" : "2016-02-19T19:00",
  "url" : "https://www.internations.org/event/promotion/details/1007258",
  "location" :
  {
    "@type" : "Place",
    "sameAs" : "http://mahiki.com/",
    "name" : "Mahiki",
    "address" :
    {
      "@type" : "PostalAddress",
      "streetAddress" : "1 Dover Street",
      "addressLocality" : "London",
      "addressRegion" : "Greater London",
      "postalCode" : "W1S 4LD"
    }
  }
}]
</script>

DiscussionForumPosting

(No present Rich Snippet support)

<script type="application/ld+json" id="forum-0">
[{
  "@context": "http://schema.org",
  "@type": "DiscussionForumPosting",
  "headline": "Govmnt Contacts in Barbados/St. Lucia/Antigua",
  "discussionUrl": "https://www.internations.org/lesser-antilles-expats/forum/govmnt-contacts-in-barbados-st-lucia-antigua-975808",
  "url": "https://www.internations.org/lesser-antilles-expats/forum/govmnt-contacts-in-barbados-st-lucia-antigua-975808",
  "datePublished": "2013-09-02T17:33",
  "image": "https://inassets1-internationsgmbh.netdna-ssl.com/image/60_60/2016/02/29/7f4cce8cf3ba8a270c3ece32886cd02d881b51fd276ce3a1b1f819b0828a9131.jpeg"
}]
</script>

SHIP IT

<24 hours>

Rich Snippets Work!

PASS

Forum Post Indexation

PASS

JSON-LD seems to affect indexation speed, independent of rich snippet support!

And it's "future-proof"

Use JSON-LD :)

SUMMARY

SEO optimization is reverse-engineering

via trial and error

and a lot of black magic

Google will not help you.

Progressive (SEO) Enhancement is key

Does GoogleBot Scroll?

Automatic Pagination

PASS

Yes.

Does your website need to be rendered server-side  to be "SEO friendly"?

No.

(but the really critical bits probably still should be)

Cheers.

FORK &


email: adam.terlson@gmail.com
skype: adam.terlson.internations


In partnership with NFQ:
roberta@nfq.lt

WE'RE HIRING

Does GoogleBot Scroll?

By Adam Terlson

Does GoogleBot Scroll?

As our applications become ever-more client-side heavy, the saying goes that if you care about SEO, you have to do your rendering on the server. Or do you? Google claims to support crawling our client-side applications, but how far does that support go? Will Google see JS-rendered links and content? What breakpoint will it browse the page at? Will it scroll down and trigger infinite pagination? At InterNations, we experimented extensively to answer these questions and more to prepare for the relaunch of 40K externally facing pages. This talk may lead to more questions than answers, but will provide solid tips for what you can do to make your client-side application SEO friendly today, as well as things to avoid.

  • 2,101