I am doing a project on developing an intranet search engine. My search engine uses MySQL as database and it is developed on PHP which is why i am posting this article here.... :D As my search engine is for an intranet and just like an search engine i will be using a crawler to get all the data avaliable on the server. My crawler comes across various file where i will have to decode the file to get text content from it. for spellchecking i was being advised by my HOD to use WordNet libraries. As there are more than 1 lakh 55 thousand words in it. I am facing a difficulty in adding all the words in to the database. Its taking to much of time and i am not inserting all the words into one table either. Based in the POS (part of speech) like verb, noun, adj, adverb i have created tables for each for easier processing....if it helps. How can i increase the speed of insertion of these many words into the table?
Edited by unikorndesigns: spelling mistake