How can I embed streaming video on HTML page? Will it use lots of bandwidth?

Most streaming video (e.g. Youtube) are actually flash video, you would need to convert your avi, mpeg files into FLV file format first, then use a flash player to play it online. You can try this Flash video player software (http://video.a4desk.com) which automatically converts your video and allow you to choose a few video player designs. However, you need to keep in mind video uses a lot more bandwidth than images, so you need to ensure your hosting plan supports that and if you need to change your hosting plan to support that.

yes... upload it first then you can just cut and paste the url to your page.

You can use FLV as video format, and make a flash video player to play the FLV file, the flash video player is a SWF file that can be embedded into web page with HTML embed tag. (FLash OBJECT and EMBED tag)

<object classid="clasid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=7,0,19,0"
width="432" height="330" id="myMovieName" >
<param name="movie" value="myMovieName.swf" />
<param name="quality" value="high" />
<param name="bgcolor" value="#FFFFFF">
<embed src="myMovieName.swf" quality="high" bgcolor="#FFFFFF" width="432" height="330" name="myMovieName" align="" type="application/x-shockwave-flash" pluginspace="http://www.macromedia.com/go/getflashplayer">
</embed>
</object>

Making flash video player in Aodbe Flash CS3 will be technical, you can use this web player Pro that will generate a custom flash video player for website and also provide a free video to flv converter.

For the bandwidth, it depends. If your streaming video will be visited by a lot of people, this will take much bandwidth.