Hi guys,

I want to create a crawler to extract some infomation from a page.
The problem is that it is written with the Java Wicket Framework and I don't know how to scrape informations from it because I don't know how to submit some post parameters.

Is this possible to do ? :)
Thank you.

5 Years
Discussion Span
Last Post by Szabi Zsoldos

You can post to a page with cUrl.

pritaeas, thank you but how do I set the post methods for example to this link ?

The input field name is wmcCif:cif


There are the POST parameters viewed with Firebug.

cautare x
criteriu    filtru.cif
wmcCif:cif  1757980

And this is the form from the page

<form id="idTestWicket03__1__2" method="post" action="https://portal.onrc.ro:443/ONRCPortalWeb/appmanager/myONRC/wicket?_nfpb=true&_windowLabel=TestWicket03_1&_urlType=action&wlpTestWicket03_1__wu=%2FONRCPortalWeb%2Fappmanager%2FmyONRC%2Fwicket%2F%3Fwicket%3Ainterface%3D%3A4%3AformCautare%3A1%3AIFormSubmitListener%3A%3A"><div style="width:0px;height:0px;position:absolute;left:-100px;top:-100px;overflow:hidden"><input type="hidden" name="idTestWicket03__1__2_hf_0" id="idTestWicket03__1__2_hf_0" /></div>
                            <table cellpadding="5">
                                    <td width="150">
                                        Cautare dupa: 
                                        <select class="select_357" onchange="document.getElementById('idTestWicket03__1__2_hf_0').value='/ONRCPortalWeb/appmanager/myONRC/wicket/?wicket:interface=:4:formCautare:criteriu:1:IOnChangeListener::';document.getElementById('idTestWicket03__1__2').submit();" name="criteriu">
                                        <option value="filtru.buletin">Nr. de buletin</option>
                                        <option value="filtru.persoana">Persoana publicata în BPI</option>
                                        <option selected="selected" value="filtru.cif">CIF</option>
                                        <option value="filtru.reg">Nr. de ordine în Registru</option>
                                        <option value="filtru.dosar">Nr. dosar</option>
                                        <option value="filtru.interval">Interval de publicare</option>
                                <table cellpadding="5">
                                        <td width="150">
                                            <input class="input_142" type="text" value="1757980" name="wmcCif:cif"/>
                            <button class="submit" type="submit" onclick="var e=document.getElementById('idTestWicket03__1__2_hf_0'); e.name='cautare'; e.value='x';var f=document.getElementById('idTestWicket03__1__2');var ff=f;if (ff.onsubmit != undefined) { if (ff.onsubmit()==false) return false; }f.submit();e.value='';e.name='';return false;"><span>Cauta</span></button>

Edited by Szabi Zsoldos


The problem is way harder than this, I'm familiarized with CURL but this one is giving me a hard time :(
I am successfully logging in but when it comes to extract certain data, it is not working.


I've tried different methods to login, succeded with DOM but not with CURL for this particular page.

function Login($data = "") {
    $ch = curl_init();

    curl_setopt($ch, CURLOPT_URL,"https://portal.onrc.ro/ONRCPortalWeb/appmanager/myONRC/public?_nfpb=true&_pageLabel=login");
    curl_setopt($ch, CURLOPT_HEADER, FALSE);
    curl_setopt($ch, CURLOPT_POST, 1);
    curl_setopt($ch, CURLOPT_POSTFIELDS,http_build_query(array(
        "j_username"  => $data['j_username'],
        "j_password"  => $data['j_password']
    curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    $server_output = curl_exec ($ch);

    return $server_output;        

I dont know if it's not based on some ajax calls that are encrypted....

Edited by Szabi Zsoldos

This topic has been dead for over six months. Start a new discussion instead.
Have something to contribute to this discussion? Please be thoughtful, detailed and courteous, and be sure to adhere to our posting rules.