Member Avatar for umshere

Searching keywords inside webpages

--------------------------------------------------------------------------------

hi all
my problem is searching :rolleyes: ...i want to search words within webpage which is stored in one directory..:surprised

it is different from normal directory searching :eek: ...beacause i am searching inside the webpage ..so i want to omit all that html tags or other asp codes...:-|
also i want to know that is there any way to store webpage inside the database:confused: ...it will be efficient than storing inside the directory....:surprised
seeking ur help :lol:
ums

  • 2 Contributors
  • 2 Replies
  • 104 Views
  • 22 Hours Discussion Span
  • Latest Post Latest Post by umshere

Dani AI

Generated

Brief, practical path to a reliable intranet help search (F1-style), building on ’s goal and the pointers from .

A robust approach: convert each help page to plain text with the browser DOM (safer than fragile regex), store that text in a simple index table, then run searches against the index. In classic ASP you can read a file with ADODB.Stream (so you control encoding) and use the HTML DOM object to get visible text:

<%
Function HtmlToText(filePath, charset)
  If charset = "" Then charset = "utf-8"

  Dim stm, html, doc
  Set stm = Server.CreateObject("ADODB.Stream")
  stm.Type = 2                 ' text
  stm.Charset = charset
  stm.Open
  stm.LoadFromFile filePath
  html = stm.ReadText
  stm.Close
  Set stm = Nothing

  Set doc = Server.CreateObject("htmlfile")
  doc.open
  doc.write html
  doc.close

  If Not doc.body Is Nothing Then
    HtmlToText = doc.body.innerText
  Else
    HtmlToText = ""
  End If

  Set doc = Nothing
End Function
%>

Indexing workflow (high level)

  • Crawl the help directory (FileSystemObject or scheduled job), call HtmlToText, normalize (lowercase, strip punctuation), store rows like (doc_id, path, title, plain_text).
  • For small collections a simple SQL lookup on the plain_text column is fine. For relevance, compute token frequencies or use the DB/external engine to rank results. For larger sets, use a dedicated indexer/search engine.

Troubleshooting and cautions

  • Ensure the IIS account can read files. Match ADODB.Stream charset to file encoding to avoid mojibake.
  • The htmlfile DOM usually yields visible text, but test pages with scripts/styles. If worried about execution or odd tags, remove script/style nodes via the DOM before reading innerText.
  • Reindex on updates or run a scheduled incremental indexer for performance.

This gives a maintainable, reproducible index and avoids brittle pattern matching while keeping the implementation classic-ASP friendly.

Recommended Answers

All 2 Replies

Member Avatar for hollystyles

You can use windows find to search for text in files in a directory. If you want to do it from ASP page and skip HTML tags that's difficut you will need to learn all about regular expressions first for searching text.

If you downoad and install cygwin you can use linux tools like grep.

weppages are text, so yes you can just save it in a databse table. But you have to be carefull of length (how much text) so in MS SQL Server for instance you will want to use a column data type of ntext for example. Then you could set-up text indexing for searching, but again skipping all the tags would be difficult but not impossible.

Member Avatar for umshere

thanks dude for ur help,
but my problem is different... i dont want the simple searching ... i am deveoping a webbased application for an intranet webpage help... i want it to be like microsoft help that u r getting pressing f1 key ....so i will create a no of help pages ...and the user can search that pages with any keywords ....there i need help...i want to find the code for searching the keywords inside the pages which is stored inside some directory..... will u help me for that ...

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.