Crawl Frontier
The queue of URLs a crawler has discovered but not yet fetched.
Definition
Why it matters
How we apply it
Related glossary entries
Frequently asked questions