30 Minutes On

 

Creating a Custom Search Engine

Simon Elichko (they/them)
Social Sciences & Data Librarian

Agenda

  • How do search engines work?
     
  • Exploring alternative search engines like Marginalia, SearXNG, Carrot2, and Ecosia
     
  • Creating your own custom search engine using Google's Programmable Search
infographic text is on linked site

Alternative search engines

  • Direct alternative to Google, works similarly but can bring up different results (including for images):  Bing
  • Wider selection of sites included, less tracking of your data, no shopping or AI overview, just links: SearXNG
  • Privacy-focused, has some Google-like features: DuckDuckGo
  • Find off-beat, unique, non-commercial sites: Marginalia
  • Cluster your search results by topic: Carrot2
  • Public benefit corporation, plants trees for every search,
    uses Bing's search index: Ecosia
  • Regional search engines like Yandex (70% of the market in Russia) and Baidu (50% of market in China)

Learn more about the global search engine market (Statista)

What's a custom search engine?

  • We'll be using Google's Programmable Search Engine.
     
  • Instead of searching the entire web, this tool lets you curate the sources (sites) that are included
     
  • You also get some control over how the search functions (e.g. how keywords are weighted, how results are ranked and displayed)

Custom search engine examples

Create your custom search engine

  1. Go to Google Programmable Search
  2. Add a new search
  3. Give it a name
  4. Enter the URLs of the sites you want to search. Formatting matters!

    To search an entire site, format URLs like this:
    *.threadtheory.ca/*
    *.    inquirer.com/*
    Start with an asterisk, then a period. Leave out the scheme (https) and the subdomain (including www), unless you only want to search that specific subdomain. Follow the URL with a slash and an asterisk.
     
  5. Choose your search settings (e.g. include image search)
  6. Try it out! For best results, add 1-2 sites and test, then add more. It's an iterative process!
anatomy of a URL, follow link for accessible version

Anatomy of a URL

Help & advice

 

  • Make an appointment with Simon
     
  • Email them with questions or to set up a time to talk: selichk1@swarthmore.edu

30 Minutes On: Creating a custom search engine

By Swarthmore Reference

30 Minutes On: Creating a custom search engine

Learn about alternative search engines like SearXNG and Marginalia, and how to create your own custom search engine where you specify which sites are included. We'll talk about use cases ranging from comparing think tanks to finding quality sewing patterns.

  • 74

More from Swarthmore Reference