http://www.daniweb.com/forums/forum9.html

Originally Posted by santiagozky

I don't get your question. are you asking how to get all the titles programaticly or how to store the title strings?

if you are asking how to ge tthe titles, I would suggest to read line by line and use a regular expression to get the string you like.

Take a look at the java Pattern class: http://java.sun.com/j2se/1.4.2/docs/...x/Pattern.html

You probably will be doing something like title=\"([a-zA-Z0-9])+\" and then get the string from the capturing group. (I haven't done this in a while so the example is probably wrong, but it is the idea.)

I did something like this but I have it at home. I'll rescue it at night to show you.

title1
title2
title3
title4
etc...

Originally Posted by KirkPatrick

Much appreciated response ezzaral, just one quick question when reading the text from the url, does that automatically read the source of the page?

Originally Posted by Ezzaral

(Yes, the HTML page source is just marked up text)

Originally Posted by santiagozky

this is some code I wrote to make a custom crawler to get images from a site. just change the pattern and the method that stores the string (Bajador is a class that downloads the image in my case) and put everything in a loop that goes through all the urls you wish to examine.

Help with Code Tags

Java Syntax (Toggle Plain Text)

 try {
                    URL url = new URL(baseURL+"/index.php?id="+par);
                  BufferedReader br= new BufferedReader( new InputStreamReader(url.openStream()));
                  String line;
                  while((line=br.readLine())!=null){
                      line=line.trim();
                      String pat="<img src=\"/(\\w*).jpg\" alt=\"Picture\"/></div>";
                       Pattern pattern = Pattern.compile(pat);
                       Matcher matcher = pattern.matcher(line);
                       boolean matchFound = matcher.find();
                        if (matchFound) {
                            String nombre=matcher.group(1);
                            String urlI=baseURL+"/file/"+nombre+".jpg";
                            Bajador baj= new Bajador(baseURL,nombre,par);
                            baj.start();
                          //  baja(baseURL,nombre);
 
                        }
                  }
                } catch (MalformedURLException ex) {
                    Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
                } catch (IOException e) {
                    Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, e);
                }

try { URL url = new URL(baseURL+"/index.php?id="+par); BufferedReader br= new BufferedReader( new InputStreamReader(url.openStream())); String line; while((line=br.readLine())!=null){ line=line.trim(); String pat="<img src=\"/(\\w*).jpg\" alt=\"Picture\"/></div>"; Pattern pattern = Pattern.compile(pat); Matcher matcher = pattern.matcher(line); boolean matchFound = matcher.find(); if (matchFound) { String nombre=matcher.group(1); String urlI=baseURL+"/file/"+nombre+".jpg"; Bajador baj= new Bajador(baseURL,nombre,par); baj.start(); // baja(baseURL,nombre); } } } catch (MalformedURLException ex) { Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex); } catch (IOException e) { Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, e); }

Hope this help you.

Originally Posted by adatapost

You could use HTML Parser, which a Java library used to parse HTML in either a linear or nested fashion. It is an open source tool and can be found on SourceForge.
you could also useThe Swing HTML Parser.