English (en)
Web crawler
A web crawler, or spider, is a
type of bot that find information
on the hundreds of millions of Web
pages that exist.
Webcrawlers index the content of
websites all across the Internet
so that those websites can appear
in search engine results.
- Saƙagon injin bincike
Decomposition: a bot for search engine
[Total Votes: 0]