1,105,427 Community Members

Using perl to activate webpage javascript post

Member Avatar
anglaissam
Newbie Poster
5 posts since Apr 2013
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

Hi, i'm trying to write a small script for the website "www.fanfiction.net." This small perl script is part of a much larger script, which is why it has to be in Perl.

When you are logged in and go to any of the stories on the site there is a button that is marked a follow/fav. This button brinks up a javascript based popup with check boxes that is then submitted to the server.

I am able to get the script to download the pages source code, and use regex to pull out what i need. The question is, is there a way to get perl to automatically submit the post command to the server only given the source code of the webpage.

Thanks for any help.

Member Avatar
2teez
Posting Whiz in Training
224 posts since Apr 2012
Reputation Points: 40 [?]
Q&As Helped to Solve: 38 [?]
Skill Endorsements: 0 [?]
 
0
 

Please check this Perl module WWW::Mechanize::Firefox from METACPAN, it should help, with this kind of stuff you want to do.

Member Avatar
doubi
Newbie Poster
5 posts since Jun 2009
Reputation Points: 0 [?]
Q&As Helped to Solve: 0 [?]
Skill Endorsements: 0 [?]
 
0
 

To offer a couple of alternative suggestions:

1) If the pop-up you describe already exists in the markup of the page when it's loaded and is simply made visible by the Javascript you could of course simply populate the fields by hand and submit the form without bothering making it 'visible'. Plain ol' WWW::Mechanize should be able to do that for you.

2) Having collected the information you want from the page, you could just use some useragent to submit the appropriate PUT request to the server? You're already automating things after all, why bother futzing about with forms made for Puny Humans? :) Again, WWW::Mechanize's ua has a put() method that could do the job for you. To find out the parameters you need to submit you could use Firebug or something to look at your outgoing requests, or just look at the form field names in the markup.

N.B., some forms use sneaky obfuscation methods to prevent people like you from doing things like what you are trying to do (i.e. automate interaction with a page designed for humans), so how possible this is might depend on how serious fanfiction.net are about stopping you :) If they're fairly serious then yes, 2teez's proposed method of WWW::Mechanize::Firefox, which I believe more or less spins up a full Firefox instance and uses that to make requests, would be the way to go.

Finally, I'd recommend using something like HTML::TreeBuilder::XPath to pull the info you need out of the markup, rather than using regexen :)

Good luck!

--doubi

You
This article has been dead for over three months: Start a new discussion instead
Post:
Start New Discussion
View similar articles that have also been tagged: