My company is planning to develop a web search engine with a crawler, and we're considering using three languages, namely C++, Java and Python. Now we're a bit not sure which language is best suitable for features like web crawling, extracting keywords & indexing, ranking of indexed pages and searching.
We're aware that some programming languages are most suitable for performing certain tasks, and we want to make the right choices. Someone suggested we use C++ for features that require absolute speed and Python for glue code that isn't all that time-critical. But we're not sure of the exact features that require absolute speed.
Now my questions are:
- Which language (C++ or Java) is most suitable for developing a web crawler and why?
- Which language is best suited for developing a search ranking algorithm - C++ or Java?
- Which features of the search engine should C++ be used for?
- Which features should Java be used for?
- Where should Python come in? Which features should it be used for?
- Do these three languages make a good combination when developing a search application?
Getting some enlightenment on these issues will enable us to get down to work. And your suggestions will be much appreciated.