I would like to know if it is technically possible to develop an automated internet search algorithm.
Let me give you a random example of what I have in mind.

For example, let’s say I am the largest disaster relief organization in the world and I want to run an automated google search every single day to see if there is any crisis/disaster happening anywhere around the world. The reason I want an automated search (not manual) because I may have many different key word combinations that I want to use (let’s say 1000 different key word combinations) and it would be hectic to do it manually. Let me give few key word examples that this disaster relief organization might want to use every day:

  • “Disaster” + “need help urgently”
  • “Massive disaster” + “no help”
  • “People are suffering” + “earthquake”
  • “Where is the UN”
  • “We are trapped”
  • “Africa” + “hunger” + “no food”
  • Etc.
    There might be hundreds of key word combinations that the disaster relief organization want to type every single day and they want an automated search algorithm to run all these key word combinations daily and then extract the key links in a page or table. Of course in Google you can also restrict the date – for e.g. ask Google to only give new results for the last day … since the organization will be running the automated algorithm each day and it only wants to see the new results.

The above was a random example for a disaster relief organization, but it can be for any agency or company.

My two most important questions are:

  • Is this technically feasible to do?
  • If it is possible, then who is the type of person that can do it? What is his “job title” or expertise or qualifications?

Basically you are building Google Alerts, but for more sources and with more restrictions. You'll need to integrate with news sources, or search engine APIs to extract the information you'd want to see. You need to build a service type application that runs daily and intelligently.

Is this technically feasible to do?


who is the type of person that can do it?

IMO you'll need a software engineer and a data analist.

Thank you for your reply Pritaeas.
What about alogrithm developers, do they do this type of job?
In general, are the experts who can do this rare? Or is it simple and widely available?
It would be great if you could provide the name of companies or individuals that provide this service, and customers that implemented such service for their business.

are the experts who can do this rare?

Any decent software engineer should be able to build this. I mentioned a data analist, because they have a clearer understanding of how to generate the results you need, using the bulk of news available each day.

It would be great if you could provide the name of companies or individuals that provide this service

I doubt we are in the same region, but any software house or freelance software engineer could build this if you have clear specifications.

No matter where you are, you may try freelancer.com

But I must say my feelings about freelance.com aren't that good enough.

Here's something that won't rile folk too much.

Someone had an idea for an app but didn't want to pay much if anything. So they went to the local colleges and picked up "interns" that would code for them in exchange for credits at the college. You can't vouch for the quality but you can get code for cheap(er)? this way.

As to whether it works for you, that's why you spent so much time writing the software design documents and how to test the app.

I don't have problems paying for the service, so let me know if you know good companies providing the same. I guess location will not be an issue, since the work can be done remotely.

Location issues can be mitigated by having your design documents complete before you begin development. Your top post is not a design document so you would have that ready as you shop developers.