how to read data from a webpage

[murali2489]

Dear Team,

I am writing a Swing application for my project which gets the user information and saves it in a database and then displays in a JTable.

I thought like, instead of manually updating the data from our client website, why not read the data from the site and update required field in my database and then show in JTable.

our client website contains fileds like ticket no, date, description ...etc . I want to read the data from those fields.

FYI i am writing code in Java SE. Please advise, is there any methods available for doing this.

Thanks in advance

[JamesCherrill]

This sourceforge project may help, or this blog has a simple DIY example.

ps: Just because info is on a web page that does not mean that you are free to use it any way you like. Please check that your use does not violate anyone's intellectual property rights.

[JamesCherrill]

I was playing around with this problem for fun today, and it's interesting just how simple it can be. Here, just for your amusement is a 3 line method that, given a URL of a web page (supplied as a String), returns the HTML for that page, also as a single String...

    // reads and returns the HTML text from the specified URL 
    // eg "http://docs.oracle.com/javase/7/docs"

    static String readHTMLfrom(String urlString) throws IOException {
        URLConnection urlConnection = new URL(urlString).openConnection();
        InputStream in = urlConnection.getInputStream();
        return new Scanner(in,"UTF-8").useDelimiter("\\A").next();
    }

The use of a Scanner to copy a whole input stream to a String is a favourite little shortcut - the secret is in the definition of the "\A" Pattern used as the delimiter ;)

After that it's a metter of parsing the HTML for the data you need - maybe you can do this in an ad-hoc "indexOf" kind of way, or maybe you will need to use a full XML parser. It all depends on the exact design of the web page.

[murali2489]

Hi James,

Thanks for the Suggestions. Its really helpful !!!