How can I insert text in special form in a textarea and as well as in the database......
e.g.

[B]H<sub>2</sub>O[/B]

with tags. You can replace [sub] and [/sub] with <sub> and </sub>, and then enter it in your DB. You should watch out for unclosed tags though, if you write [sub] and not close it all following text will be subscript. You should be able to figger it out.

actually I have a textbox where i've to enter texts like some chemical equations and when I will click submit button that text will enter in my database............give me some idea...........

I am still trying to figure out exact what you want but i'll try. You could use javascript to insert the tags into the textarea.

<input type="button" value="Subscript" onclick="document.getElementById('message').value+='[sub][/sub]'">

Then use php to find and replace "[sub]" & "[/sub]" with the proper tags and then insert them into the database. Is that what you want?