I need help with grabbing a complete web site.

[bsdixon21222]

Hello all,

I have a unique situation. I am currently trying to save a complete web page that requires you to login with a user name and password. To be more specific, here is the web page I want.

www.eventid.net

This site requires you to have a subscription in order to see all the event IDs. What I would like to do is have those event IDs stored locally on my compuet. Now I could do this by logging in, saving each of the 65,000 pages, but I think python may be able to complete the task faster. What I would need the script to do is open up internet explorer/mozilla firefox and navigate to the website. Upon getting to the website it will have to enter the credentials to gain access to the 65000 ID pages and then save them.

I know what I am asking is a little vague so if you have any questions or comments you can leave them here that way I can address them. Thanks in advance! Smile

[jrcagle]

Well, I would probably contact the company and just ask them if you could have an archive. I'm guessing "No", based on their use policy:

2.3 You may view or print the Content only for your own personal use, provided that you maintain and include all copyright and other notices contained in such Content. You shall not store electronically any portion of any Content. Except as expressly permitted by the copyright laws, no copying, storage, redistribution or publication of any Content is permitted without the express permission of Altair or the owners of such Content or their authorized persons if other than Altair. You agree that you will not use your access to the Site and/or any Content stored therein (including the compilation of links to externally hosted content) to develop or enhance a competitive website or product offering.

But if they don't care, then they'll probably happily package up the info on a CD-ROM or something for you, or e-mail it if it's small.

Jeff