943,764 Members | Top Members by Rank

Ad:
  • PHP Discussion Thread
  • Unsolved
  • Views: 1244
  • PHP RSS
Apr 20th, 2009
0

Curl function to select options from a select box and auto submit

Expand Post »
Hi all

i am a newbie and try different things everyday and always come here when i am stuck with something.

I want to write a script using curl and php that goes to this link :http://tools.cisco.com/WWChannels/LO...BasicSearch.do and then goes through each page for each country capturing a list of every partner in every country and saving it to database.

i have no ideas how script will select countries one by one from select box and redirect page to country page...which is the very first thing to do, once we are on the page pattern matching comes in play for storing name and address in database which i can manage.

Problem is before we select any country url is::http://tools.cisco.com/WWChannels/LO...BasicSearch.do
and after we select country say 'india' url is:http://tools.cisco.com/WWChannels/LO...BasicSearch.do , there is no reference to any country selected.

The Idea that i had was to traverse the HTML page, and enter all countries in an array and then make a recursive function to call a page with specific country but for that we need something different in URL for each country in recursive function right?

Please help
Reputation Points: 10
Solved Threads: 0
Light Poster
jyotiu is offline Offline
33 posts
since Sep 2008
Apr 21st, 2009
0

Re: Curl function to select options from a select box and auto submit

I received your message and had a crack at the code but it seems there is some sort of anti-bot protection script on the website. The following is the script I used:
php Syntax (Toggle Plain Text)
  1. <?
  2. //array('AF','DZ','AS','AD','AO','AI','AQ','AG','AR','AM','AW','AT','AU','AZ','BS','BH','BD','BB','BY','BE','BZ','BJ','BM','BT','BO','BA','BW','BV','BA','IO','BN','BF','BG','BI','KH','CM','CA','CV','KY','CF','TD','CL','CN','CX','CC','CO','KM','CG','CD','CK','CR','CI','HR','CY','CZ','DK','DJ','DM','DO','EC','EG','SV','GQ','EE','ET','FK','FO','FJ','FI','FR','GF','PF','GA','GM','GE','DE','GH','GI','GR','GL','GD','GP','GU','GT','GN','GW','GY','HT','HM','VA','HN','HK','HU','IS','IN','ID','IQ','IE','IL','IT','JM','JP','JO','KZ','KE','KI','KR','KW','KG','LA','LV','LB','LS','LR','LY','LI','LT','LU','MO','MK','MG','MW','MY','MV','ML','MT','MH','MQ','MR','MU','YT','MX','FM','MD','MC','MN','MS','MA','MZ','MM','NA','NR','NP','NL','AN','NC','NZ','NI','NE','NG','NU','NF','MP','NO','OM','PK','PW','PS','PA','PG','PY','PE','PH','PN','PL','PT','PR','QA','RE','RO','RU','RW','SH','KN','LC','PM','VC','WS','SM','ST','SA','SN','CS','SC','SL','SG','SK','SI','SB','SO','ZA','ES','LK','SR','SJ','SZ','SE','CH','TW','TJ','TZ','TH','TL','TG','TK','TO','TT','TN','TR','TM','TC','TV','UG','UA','AE','GB','US','UM','UY','UZ','VU','VE','VN','VG','VI','WF','YE','ZM','ZW');
  3. $country=array('AF');
  4. for ($id=0;isset($country[$id]);$id++) {
  5. $ch = curl_init();
  6. // set the target url
  7. curl_setopt($ch, CURLOPT_URL,'http://tools.cisco.com/WWChannels/LOCATR/performBasicSearch.do');
  8. curl_setopt($ch, CURLOPT_HTTPHEADER, Array("User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.15) Gecko/20080623 Firefox/2.0.0.15") ); // request as if Firefox
  9. // howmany parameter to post
  10. curl_setopt($ch, CURLOPT_POST, true);
  11. curl_setopt($ch, CURLOPT_POSTFIELDS,'state=&latitude=&longitude=&city=&zip=&lonlatRequired=N&smbSort=true&address=&country='.$country[$id]);
  12. curl_setopt($ch, CURLOPT_NOBODY, false);
  13. curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
  14. curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
  15. $result= curl_exec ($ch);
  16. curl_close ($ch);
  17. echo $result;
  18. }
  19. ?>
And if you check what it displays the curl function doesn't redirect to the third page which contains the results. It is stuck in the second page and may may have something to do with javascript or ajax. So although the above script partially works at retrieving the data it just needs extending to lead to the beginning results instead of the bot trap.

Also just as a note, the commented array can replace the array on the line below it when it all works so that the script will check all of the countries.
Sponsor
Featured Poster
Reputation Points: 410
Solved Threads: 258
Occupation: Genius
cwarn23 is offline Offline
3,004 posts
since Sep 2007

This thread is more than three months old

No one has posted to this discussion for at least three months. Please let old threads die and do not reply to them unless you feel you have something new and valuable to contribute that absolutely must be added to make the discussion complete. Otherwise, please start a new thread in this forum instead.
Message:
Previous Thread in PHP Forum Timeline: l want to start a new on line web development business
Next Thread in PHP Forum Timeline: how to validate value in drop down menu in php





About Us | Contact Us | Advertise | Acceptable Use Policy
Forum Index | Build Custom RSS Feed


Follow us on Twitter


© 2011 DaniWeb® LLC