For the last 5 months I've tried to organize myself and my friends in order to collect some relevant information on news sites (information about global warming, pollution, ecology etc.)
The only solution that I've found was to use a local search engine from the news site and type in adequate keywords which are mentioned inside the parenthesis above. After that, articles would appear and the only way to collect some relevant sentences is to read the whole text or again use the CTRL+F and type in the criteria words.
Unfortunately, it worked only for a few days because we were getting tired of doing this. I've calculated that it would take us a long time to achieve what we wanted.
Before the project had started I was thinking that the best thing would be to come up with an artificial intelligence algorithm which would save us 80% of our time. I don't see any other good programming approach apart from looping through the text and getting the senteces with the relevant words, but the same thing is done with CTRL+F.
I don't want to make this post too long. I would be very glad if someone could tell me how to approach/tackle such problem (efficiently collect data on news site). Is AI worth it? If yes what type of AI would I use to do this or should I use another approach.
I would also consider tagged languages to solve my problem. If there is a better language, please suggest them.
