Focused Crawler

1
Focused Crawler

• Ben Markines
• Mira Stoilova
• Fulya Erdinc

2
Introduction

• Based from the paper presented the first week of
  class
• Accelerated Focused Crawling through Online
  Relevance Feedback by Chakrabarti presented by
  Mark Meiss
• Implemented a focused crawler and a focused
  crawler with an apprentice
• Apprentice analyzes words around a link

3
Crawler Implementation

• Feature extraction
• Using document frequency and mutual information
• Baseline crawl using a classifier
• Naïve Bayesian
• Cosine Similarity
• Support Vector Machine
• Crawl with trained apprentice
• Again using the same types of classifiers

4
Baseline Precision/Recall Target Pages
5
Baseline Precision/Recall DMOZ Description
6
Apprentice Precision/Recall Target Pages
7
Apprentice Precision/Recall DMOZ Description