I am intended to use heritrix to crawl some sites but want to skip all the ads that are linked in the pages also want to make my crawler to crawl only specific links form seed URL which I am getting from my database. Will you please guide me for this?
Is there any plugin to heritrix like metacrawler or similar to that??
dipsn
0
Newbie Poster
Be a part of the DaniWeb community
We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.