Java

Java RSS

write in FORMs with Java

•

Join Date: Jul 2007

Posts: 31

Reputation: claudiu_is is an unknown quantity at this point

Solved Threads: 1

claudiu_is claudiu_is is offline

Offline

Light Poster

write in FORMs with Java

Oct 7th, 2007

I`m trying to write a web crawler but i `don`t know how to handle the situation when i have to login to acces a site. How to type in that login form from my java program? I`ve searched on the net but the results points to JSP. I haven`t worked with JSP before, isn`t there a simplier method to complete those user and password HTML fields ??

•

Join Date: Sep 2007

Posts: 27

Reputation: schoolsoluction is an unknown quantity at this point

Solved Threads: 3

schoolsoluction schoolsoluction is offline

Offline

Light Poster

Re: write in FORMs with Java

Oct 7th, 2007

search google for 'curl', that should get you started...

Download java source code examples from http://java-assignment.com/
N Puzzle game, Magic squares, Huffman compression techniques, ...

•

Join Date: Jul 2007

Posts: 31

Reputation: claudiu_is is an unknown quantity at this point

Solved Threads: 1

claudiu_is claudiu_is is offline

Offline

Light Poster

Re: write in FORMs with Java

Oct 7th, 2007

You are talking about JavacURL right ? the cURL library for Java, well, I tried to use that but the documentation from there http://curl.haxx.se/libcurl/ is made for C. In the downloaded packets of JavacURL i found just some instructions for installing that library in the Eclipse editor and a javadoc with few classes and methods but with lots of fields;

I don`t know what fields I should use in my program ....Isn`t there another way to resolv my problem except this stupid JavacURL ? I`ve lost hours

tonight fiding a way to make use this.. If there isn`t another solution, I`ll try again in the morning, but I hope you guys will gave some usefull tips about JavacURL

•

Join Date: Sep 2007

Posts: 27

Reputation: schoolsoluction is an unknown quantity at this point

Solved Threads: 3

schoolsoluction schoolsoluction is offline

Offline

Light Poster

Re: write in FORMs with Java

Oct 8th, 2007

You can call the .exe or shell script with some arguments...

Thats the way I do it... Curls saves the results in text files and then I analyse those files... followed by a new call to curl...

Download java source code examples from http://java-assignment.com/
N Puzzle game, Magic squares, Huffman compression techniques, ...

•

Join Date: Feb 2006

Posts: 2,385

Reputation: masijade has much to be proud of

Solved Threads: 252

masijade

Offline

Nearly a Posting Maven

Re: write in FORMs with Java

Oct 8th, 2007

•

Originally Posted by claudiu_is

I`m trying to write a web crawler but i `don`t know how to handle the situation when i have to login to acces a site. How to type in that login form from my java program? I`ve searched on the net but the results points to JSP. I haven`t worked with JSP before, isn`t there a simplier method to complete those user and password HTML fields ??

HttpUrlConnection as a POST request. See http://coding.derkeiler.com/Archive/.../msg01212.html for an example (found in 20 seconds using Yahoo). It is for HttpsUrlConnection, rather than HttpUrlConnection, so strip out the SSL stuff, otherwise, it should be the same.

In summary for you, read the form, finding the different parameters (including the hidden ones), then, using those parameter names, create a POST response and connect again.

Hopefully, you are familiar with the forms already, as I would not count on being able to create some generic login form manipulator.

Edit: And, yes, the code is not that good, and he is complaining that it is not working over a proxy, but it is at least a starting point for you.

Last edited by masijade; Oct 8th, 2007 at 8:05 am.

Java Programmer and Sun Systems Administrator

----------------------------------------------

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.
--Brian Kernighan

•

Join Date: Nov 2004

Posts: 6,143

Reputation: jwenting is just really nice

Solved Threads: 212

jwenting

Offline

duckman

Re: write in FORMs with Java

Oct 8th, 2007

A nice handy help for doing network thingies from Java is Apache Commons Net (and/or Apache Commons HttpClient).

As people are clearly allowed to attack me but I'm not allowed to defend myself, I no longer post to this site.

•

Join Date: Jul 2007

Posts: 31

Reputation: claudiu_is is an unknown quantity at this point

Solved Threads: 1

claudiu_is claudiu_is is offline

Offline

Light Poster

Re: write in FORMs with Java

Oct 10th, 2007

i`ve just found out that i need session authentication(this is the tupe of authentication for trackers), does that mean that i have to work with cookies ? uhhh, cookies are new for me

Last edited by claudiu_is; Oct 10th, 2007 at 6:20 am.

•

Join Date: Sep 2007

Posts: 27

Reputation: schoolsoluction is an unknown quantity at this point

Solved Threads: 3

schoolsoluction schoolsoluction is offline

Offline

Light Poster

Re: write in FORMs with Java

Oct 10th, 2007

Yes indeed.

With curl for exemple, you login. Then you get a session coockie. That one you write to a file and pass it on to all your later curl calls...

Download java source code examples from http://java-assignment.com/
N Puzzle game, Magic squares, Huffman compression techniques, ...

•

Join Date: Apr 2008

Posts: 3

Reputation: dkhang is an unknown quantity at this point

Solved Threads: 0

dkhang

Offline

Newbie Poster

Re: write in FORMs with Java

Jul 1st, 2009

I am using URL Connection to login to the site http://sms.vn/send/login.jsp but it just returns the same login page again.
My program will do like this: it opens an url connection to get the login form from the site, fill in the user name and password, ask for user to type captcha and send the POST request to the site with those information. My code works for another site like:
http://www.1your.com/drupal/sample_login.php
I am so sure that my user name and password are right, you can try it, it is free and will not cause much harm for me

 Help with Code Tags 
 Java Syntax (Toggle Plain Text) 
import java.io.IOException;
import java.io.PrintStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLEncoder;
import java.util.Scanner;
import java.util.logging.Level;
import java.util.logging.Logger;
 
/**
 *
 * @author Duy Khang
 */
public class NewClass2 {
 
    private static String LOGIN_INPUT = "login";
    private static String LOGIN_VALUE = "LOGIN";
    private static String NICK_INPUT = "nick";
    private static String NICK_VALUE = "vodkhang";
    private static String PASS_INPUT = "pass";
    private static String PASS_VALUE = "vhnkozou";
    private static String CHECK_CODE_INPUT = "checkcode";
    private static String CHECK_CODE_VALUE = "";
 
    public static void main(String[] args) {
        try {
            URL url = new URL("http://sms.vn/send/login.jsp");
            URLConnection connection = url.openConnection();
            connection.setDoOutput(true);
 
            Scanner sc = new Scanner(connection.getInputStream());
            while (sc.hasNextLine()) {
                System.out.println(sc.nextLine());
            }
            Scanner sc2 = new Scanner(System.in);
            System.out.println("enter the check code value: ");
            CHECK_CODE_VALUE = sc2.nextLine();
 
            String encodedLoginUserName = URLEncoder.encode(NICK_VALUE, "UTF-8");
            String encodedLoginPassword = URLEncoder.encode(PASS_VALUE, "UTF-8");
 
 
            URLConnection connection1 = url.openConnection();
            connection1.setDoOutput(true);
            PrintStream output = new PrintStream(connection1.getOutputStream());
 
            StringBuilder request = new StringBuilder();
            request.append(LOGIN_INPUT);
            request.append("=");
            request.append(LOGIN_VALUE);
            request.append("&");
 
            request.append(NICK_INPUT);
            request.append("=");
            request.append(encodedLoginUserName);
            request.append("&");
 
            request.append(PASS_INPUT);
            request.append("=");
            request.append(encodedLoginPassword);
            request.append("&");
 
            request.append(CHECK_CODE_INPUT);
            request.append("=");
            request.append(CHECK_CODE_VALUE);
            System.out.println("request: " + request);
 
            output.println(request);
            System.out.println("----------------------------------------------------");
            Scanner sc3 = new Scanner(connection1.getInputStream());
            while (sc3.hasNextLine()) {
                System.out.println(sc3.nextLine());
            }
        } catch (MalformedURLException ex) {
            Logger.getLogger(NewClass1.class.getName()).log(Level.SEVERE, null, ex);
        } catch (IOException ex) {
            Logger.getLogger(NewClass1.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
}
import java.io.IOException;
import java.io.PrintStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLEncoder;
import java.util.Scanner;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author Duy Khang
 */
public class NewClass2 {

    private static String LOGIN_INPUT = "login";
    private static String LOGIN_VALUE = "LOGIN";
    private static String NICK_INPUT = "nick";
    private static String NICK_VALUE = "vodkhang";
    private static String PASS_INPUT = "pass";
    private static String PASS_VALUE = "vhnkozou";
    private static String CHECK_CODE_INPUT = "checkcode";
    private static String CHECK_CODE_VALUE = "";

    public static void main(String[] args) {
        try {
            URL url = new URL("http://sms.vn/send/login.jsp");
            URLConnection connection = url.openConnection();
            connection.setDoOutput(true);

            Scanner sc = new Scanner(connection.getInputStream());
            while (sc.hasNextLine()) {
                System.out.println(sc.nextLine());
            }
            Scanner sc2 = new Scanner(System.in);
            System.out.println("enter the check code value: ");
            CHECK_CODE_VALUE = sc2.nextLine();

            String encodedLoginUserName = URLEncoder.encode(NICK_VALUE, "UTF-8");
            String encodedLoginPassword = URLEncoder.encode(PASS_VALUE, "UTF-8");


            URLConnection connection1 = url.openConnection();
            connection1.setDoOutput(true);
            PrintStream output = new PrintStream(connection1.getOutputStream());

            StringBuilder request = new StringBuilder();
            request.append(LOGIN_INPUT);
            request.append("=");
            request.append(LOGIN_VALUE);
            request.append("&");

            request.append(NICK_INPUT);
            request.append("=");
            request.append(encodedLoginUserName);
            request.append("&");

            request.append(PASS_INPUT);
            request.append("=");
            request.append(encodedLoginPassword);
            request.append("&");

            request.append(CHECK_CODE_INPUT);
            request.append("=");
            request.append(CHECK_CODE_VALUE);
            System.out.println("request: " + request);

            output.println(request);
            System.out.println("----------------------------------------------------");
            Scanner sc3 = new Scanner(connection1.getInputStream());
            while (sc3.hasNextLine()) {
                System.out.println(sc3.nextLine());
            }
        } catch (MalformedURLException ex) {
            Logger.getLogger(NewClass1.class.getName()).log(Level.SEVERE, null, ex);
        } catch (IOException ex) {
            Logger.getLogger(NewClass1.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
}

•

Join Date: Apr 2008

Posts: 3

Reputation: dkhang is an unknown quantity at this point

Solved Threads: 0

dkhang

Offline

Newbie Poster

Re: write in FORMs with Java

#10

Jul 1st, 2009

 Help with Code Tags 
 Java Syntax (Toggle Plain Text) 
 
import java.io.IOException;
import java.io.PrintStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLEncoder;
import java.util.Scanner;
import java.util.logging.Level;
import java.util.logging.Logger;
 
/**
 *
 * @author Duy Khang
 */
public class NewClass2 {
 
    private static String LOGIN_INPUT = "login";
    private static String LOGIN_VALUE = "LOGIN";
    private static String NICK_INPUT = "nick";
    private static String NICK_VALUE = "vodkhang";
    private static String PASS_INPUT = "pass";
    private static String PASS_VALUE = "xxxxx";
    private static String CHECK_CODE_INPUT = "checkcode";
    private static String CHECK_CODE_VALUE = "";
 
    public static void main(String[] args) {
        try {
            URL url = new URL("http://sms.vn/send/login.jsp");
            URLConnection connection = url.openConnection();
            connection.setDoOutput(true);
 
            Scanner sc = new Scanner(connection.getInputStream());
            while (sc.hasNextLine()) {
                System.out.println(sc.nextLine());
            }
            Scanner sc2 = new Scanner(System.in);
            System.out.println("enter the check code value: ");
            CHECK_CODE_VALUE = sc2.nextLine();
 
            String encodedLoginUserName = URLEncoder.encode(NICK_VALUE, "UTF-8");
            String encodedLoginPassword = URLEncoder.encode(PASS_VALUE, "UTF-8");
 
 
            URLConnection connection1 = url.openConnection();
            connection1.setDoOutput(true);
            PrintStream output = new PrintStream(connection1.getOutputStream());
 
            StringBuilder request = new StringBuilder();
            request.append(LOGIN_INPUT);
            request.append("=");
            request.append(LOGIN_VALUE);
            request.append("&");
 
            request.append(NICK_INPUT);
            request.append("=");
            request.append(encodedLoginUserName);
            request.append("&");
 
            request.append(PASS_INPUT);
            request.append("=");
            request.append(encodedLoginPassword);
            request.append("&");
 
            request.append(CHECK_CODE_INPUT);
            request.append("=");
            request.append(CHECK_CODE_VALUE);
            System.out.println("request: " + request);
 
            output.println(request);
            System.out.println("----------------------------------------------------");
            Scanner sc3 = new Scanner(connection1.getInputStream());
            while (sc3.hasNextLine()) {
                System.out.println(sc3.nextLine());
            }
        } catch (MalformedURLException ex) {
            Logger.getLogger(NewClass1.class.getName()).log(Level.SEVERE, null, ex);
        } catch (IOException ex) {
            Logger.getLogger(NewClass1.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
}
import java.io.IOException;
import java.io.PrintStream;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
import java.net.URLEncoder;
import java.util.Scanner;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 *
 * @author Duy Khang
 */
public class NewClass2 {

    private static String LOGIN_INPUT = "login";
    private static String LOGIN_VALUE = "LOGIN";
    private static String NICK_INPUT = "nick";
    private static String NICK_VALUE = "vodkhang";
    private static String PASS_INPUT = "pass";
    private static String PASS_VALUE = "xxxxx";
    private static String CHECK_CODE_INPUT = "checkcode";
    private static String CHECK_CODE_VALUE = "";

    public static void main(String[] args) {
        try {
            URL url = new URL("http://sms.vn/send/login.jsp");
            URLConnection connection = url.openConnection();
            connection.setDoOutput(true);

            Scanner sc = new Scanner(connection.getInputStream());
            while (sc.hasNextLine()) {
                System.out.println(sc.nextLine());
            }
            Scanner sc2 = new Scanner(System.in);
            System.out.println("enter the check code value: ");
            CHECK_CODE_VALUE = sc2.nextLine();

            String encodedLoginUserName = URLEncoder.encode(NICK_VALUE, "UTF-8");
            String encodedLoginPassword = URLEncoder.encode(PASS_VALUE, "UTF-8");


            URLConnection connection1 = url.openConnection();
            connection1.setDoOutput(true);
            PrintStream output = new PrintStream(connection1.getOutputStream());

            StringBuilder request = new StringBuilder();
            request.append(LOGIN_INPUT);
            request.append("=");
            request.append(LOGIN_VALUE);
            request.append("&");

            request.append(NICK_INPUT);
            request.append("=");
            request.append(encodedLoginUserName);
            request.append("&");

            request.append(PASS_INPUT);
            request.append("=");
            request.append(encodedLoginPassword);
            request.append("&");

            request.append(CHECK_CODE_INPUT);
            request.append("=");
            request.append(CHECK_CODE_VALUE);
            System.out.println("request: " + request);

            output.println(request);
            System.out.println("----------------------------------------------------");
            Scanner sc3 = new Scanner(connection1.getInputStream());
            while (sc3.hasNextLine()) {
                System.out.println(sc3.nextLine());
            }
        } catch (MalformedURLException ex) {
            Logger.getLogger(NewClass1.class.getName()).log(Level.SEVERE, null, ex);
        } catch (IOException ex) {
            Logger.getLogger(NewClass1.class.getName()).log(Level.SEVERE, null, ex);
        }
    }
}