Search engine

• The first text-based search engine is considered Veronica, used as a search tool in Gopher.

• The first search engine ever developed is considered Archie, used to find files stored on anonymous FTP sites.

Web search engine:

• A software that is designed to search for information(web pages, images and other types of files) on the World Wide Web.

• Eg.  Google, Duck Duck Go, Bing etc.

• They used crawlers to go through every page, creates a huge index, receives the search request, compares it to the entries in the index, and returns results.

Search files/pattern in file system

1. find

2. locate

3. grep

Search Engine Programs

• Whoosh: A fast, featureful full-text indexing and searching library implemented in pure Python.

• Xapian: Open source search engine library, which allows developers to easily add advanced indexing and search facilities to their own applications

Text/ Multimedia search engine for

user contents

• A search engine for text, image, and audio files.

• It searches audio files according to the artist, album, genre and year.

• It searches image files according to date, month, year, locality, city, state, country, and postal code.

• It searches textual files according to the given search word and returns path/to/file containing number of iterations of the word.

Text files

• Yielding a list of all text files from given directory.     

• Looking for search word in saved indexes

• Saving the indexes

• Creating indexes of files along with the repetition of words.

Audio files

• Yielding files ('.mp3', '.ogg', '.wav', '.flac', '.wma') from given directory.
• Extracting metadata using pyexiftool from files.
• Saving required metadata -- (file_type, file_size, artist, album, genre, year)
• Index metadata along with their respective files and save them.
• Look for given artist, album, genre, year in saved files.

Image files

• Yielding a list of files ('.png', '.tif', '.jpg', '.gif', '.JPEG') from given directory.
• Extracting metadata using pyexiftool from files.
• Saving required metadata -- (created-date, g_p_s_latitude, g_p_s_longitude)
• Index metadata along with their respective files and save them.
• Look for given date, month, year, locality, city, state, country, and postal code in saved files.

Major challenges in Image files

• Extracting date, month and year  from create_date metadata

• Exception handling  

• Searching for all keywords.

• Extracting village, city, state, country and postal code from gps_latitude and gps_longitude.

Increasing efficiency of searching

{code}