Are the crawled URLs stored in the database then they are traversed in BFS/DFS manner .... or is it something else ?
Please Help!
apanimesh061
0
Junior Poster
LastMitch
@apanimesh061
Are the crawled URLs stored in the database then they are traversed in BFS/DFS manner .... or is it something else ?
You are asking about a theory of how it works?
Read this:
http://activities.tjhsst.edu/sct/lectures/1112/bfsdfs100711.pdf
If you want to try how crawl works then read this:
https://developers.google.com/search-appliance/documentation/68/admin_crawl/database_crawl_serve
https://developers.google.com/search-appliance/documentation/68/help_gsa/crawl_database
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.