pulling plain text from a website, how?

[aanders5]

Ok, so basically i am just trying to pull plain text from a website. basically it will display a person's Runescape stat's in text form.

here is my code so far on how to get the url for the page.

private void statsButtonActionPerformed(java.awt.event.ActionEvent evt) {
       String oldrsn = rsnTextField.getText();
       String newrsn = null;
       String statPuller = null;
       for (int i = 0; i < oldrsn.length(); i++) {
           newrsn = oldrsn.replaceAll(" ", "_");
       }
       statPuller = "http://hiscore.runescape.com/index_lite.ws?player=" + newrsn;
       

    }

basically, that does is combine the person's gamer name along with the correct url to access their stats

So, for example, if I wanted to get the user "Elite Clues", this code would make the url http://hiscore.runescape.com/index_lite.ws?player=Elite_Clues I would then want his/her data in my Java program

And that url page for "Elite Clues" displays this:
(you can just pop that link into the url to see yourself)

254223,1730,41252224 181358,92,7018183 175673,90,5347709 241135,94,7962470 238717,92,6674838 617607,75,1264987 234511,70,757728 560087,76,1402988 595779,74,1101241 -1,-1,-1 1084258,60,295003 412654,78,1659647 456400,70,790204 421026,63,386483 337367,65,485600 967287,64,426256 294510,57,222206 427295,58,231120 329460,61,315285 121978,81,2303119 271763,57,204005 512900,50,110160 420812,62,336172 179464,67,552157 118258,74,1099187 319783,41,45061 -1,-1 -1,-1 -1,-1 211063,1383 -1,-1 56327,1467 -1,-1 -1,-1 -1,-1 -1,-1 -1,-1

So I am trying to find a way to pull that data from a url into my program.
I can parse it and sort it from there on my own.
Thank you,
-Austin

[Ezzaral]

Here's a basic example on getting text from a URL:
http://www.exampledepot.com/egs/java.net/ReadFromURL.html

[aanders5]

Ok, so do I get it to read that stuff into a string. like, I want that page to turn into

statPullerData = "254223,1730,41252224 181358,92,7018183 175673,90,5347709 241135,94,7962470 238717,92,6674838 617607,75,1264987 234511,70,757728 560087,76,1402988 595779,74,1101241 -1,-1,-1 1084258,60,295003 412654,78,1659647 456400,70,790204 421026,63,386483 337367,65,485600 967287,64,426256 294510,57,222206 427295,58,231120 329460,61,315285 121978,81,2303119 271763,57,204005 512900,50,110160 420812,62,336172 179464,67,552157 118258,74,1099187 319783,41,45061 -1,-1 -1,-1 -1,-1 211063,1383 -1,-1 56327,1467 -1,-1 -1,-1 -1,-1 -1,-1 -1,-1 ";

[Ezzaral]

Did you try the example? It really doesn't get much more straightforward.

You can append each line as it's read to a single string if you wish. readLine() is going to strip the newline from each line of text as it is read, so you'll have to put in your own separator between lines.

[aanders5]

Ok, here is my problem right now... This may be slightly hard to follow but I'm hoping you will understand.

I got this working for all but the end part of my code, due to some bug.
So, I found this bug using the gamertag "Gladz".
I grabbed the text and with that code you gave me, and I get this...

2804,2433,34665047041659,99,1764716926563,99,1730835316960,99,2517954825045,99,2603468532347,99,1570951010817,99,131508829254,99,1860395512809,99,15552148134140,97,1110038134832,99,1352335355285,97,1113800865478,98,1210832518417,94,84756442327,99,150679056763,99,137435655365,99,1408401016124,92,665851231643,90,566834234021,92,69675745355,99,148353828151,97,1157310039192,90,56704519604,95,914106412880,99,130953161923,105,24613288-1,-1-1,-132277,124762251,2912-1,-176064,119142073,124747695,129021911,2681-1,-1-1,-1

it grabs the skill levels nicely, however, the problem is with the Activities section,for some reason at the end, it combines the user's rank with their score when it is parsed into that code of yours.

(EDIT: I ADDED AN ATTACHED IMAGE BELOW TO HELP YOU SEE THE ISSUE BETTER, the "Barbarian Assault" section where the error's are at is labelled B.A. in the Runescape Highscore page)

If you look here at the official game's highscore board for user "Gladz": http://services.runescape.com/m=hiscore/hiscorepersonal.ws?user1=Gladz
we can see that the text above fused the scores and ranks together.
So it is combining these...
attackers score = attackers score + defenders rank
defenders score = defenders score + collectors rank
collectors score = collectors score + healers rank
The only unaffected value is the healer's score, I am assuming that is because 'gladz' has no castle war rank.

also, I am assuming that if the user was ranked in every activity, it would do this to all the activity scores.

Here is the raw web page that I am grabbing the data from with that code: http://hiscore.runescape.com/index_lite.ws?player=gladz
In there, we can see that the score's and rank's are separated.

So why is your code fusing them together when it parses? I only want the "scores" for each activity.

Again, I really hope you can follow what I am saying. I am using the Runescape highscores page to just double check my own program, and that is how I realized that the score's were way off.

Thank you,
-Austin

[Ezzaral]

Post the code that you used to aggregate the data string. It looks like you're not adding any separation between the lines that it reads. The actual file content looks like this

2804,2433,346650470
41659,99,17647169
26563,99,17308353
16960,99,25179548
25046,99,26034685
32347,99,15709510
...

Each score is on a separate line. The readLine() call will strip the line break, so if you're just adding each line read to a single string variable it will get output like you posted above 2804,2433,34665047041659,99,1764716926563 To remedy that, you could add a separator character between them such as "|", but really you'd save yourself the effort of re-parsing the string later if you created a small data object to hold each score entry. Add them to an array list as you read and create them.

[aanders5]

I am double posting so you can see a good example of how my code works.
Also, you will see how the game's highscores match up with my own programs.

This time I am is using the character "elite clues"
Runescape Highscores for "elite clues": http://services.runescape.com/m=hiscore/hiscorepersonal.ws?user1=Elite_Clues
Image Attached of program using "elite clues"

The reason why this one works I believe, is because they do not have any two activities ranked that are next to each other.

So, that is the end-all problem I guess. But I think it is due to that parse code you gave me for reading the website.

Thanks much,
-Austin

[aanders5]

Nope. here is my actual code for reading the page.

not formatted, sorry.

private void statsButtonActionPerformed(java.awt.event.ActionEvent evt) {                                            
       String oldrsn = rsnTextField.getText();
       String newrsn = "";
       String statPuller = "";
       String statPullerData = "";
       warningLabel.setText("");
       for (int i = 0; i < oldrsn.length(); i++) {
           newrsn = oldrsn.replaceAll(" ", "_");
       }

        statPuller = "http://hiscore.runescape.com/index_lite.ws?player=" + newrsn;

       try {
    // Create a URL for the desired page
    URL url = new URL(statPuller);

    // Read all the text returned by the server
    BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
    String str;
     int counter = 0;
    while ((str = in.readLine()) != null) {counter++;
        // str is one line of text; readLine() strips the newline character(s)

   

       
        statPullerData = statPullerData + str;
    }
     System.out.println(statPullerData);
     
    in.close();
} catch (MalformedURLException e) {
} catch (IOException e) {
}    

     for (int j = 0; j < statPullerData.length(); j++) {
           statPullerData = statPullerData.replaceAll(",", " ");
       }

        for (int j = 0; j < statPullerData.length(); j++) {
           statPullerData = statPullerData.replaceAll("-1", "-");
       }



        System.out.println(statPullerData);









ArrayList<String> al = new ArrayList<String>();
Scanner scan = new Scanner(statPullerData);
while (scan.hasNext()) {
    al.add(scan.next());

}


if (al.isEmpty()) {
    warningLabel.setForeground(Color.red);
    warningLabel.setText("INVALID USERNAME!");
}



totalLevelLabel.setText(al.get(1));
attackLevelLabel.setText(al.get(3));
defenceLevelLabel.setText(al.get(5));
strengthLevelLabel.setText(al.get(7));
constitutionLevelLabel.setText(al.get(9));
rangedLevelLabel.setText(al.get(11));
prayerLevelLabel.setText(al.get(13));
magicLevelLabel.setText(al.get(15));
cookingLevelLabel.setText(al.get(17));
woodcuttingLevelLabel.setText(al.get(19));
fletchingLevelLabel.setText(al.get(21));
fishingLevelLabel.setText(al.get(23));
firemakingLevelLabel.setText(al.get(25));
craftingLevelLabel.setText(al.get(27));
smithingLevelLabel.setText(al.get(29));
miningLevelLabel.setText(al.get(31));
herbloreLevelLabel.setText(al.get(33));
agilityLevelLabel.setText(al.get(35));
thievingLevelLabel.setText(al.get(37));
slayerLevelLabel.setText(al.get(39));
farmingLevelLabel.setText(al.get(41));
runecraftingLevelLabel.setText(al.get(43));
hunterLevelLabel.setText(al.get(45));
constructionLevelLabel.setText(al.get(47));
summoningLevelLabel.setText(al.get(49));
dungeoneeringLevelLabel.setText(al.get(51));

System.out.println(al.size());

for (int b = 53; b < al.size(); b++) {
    
    if (al.get(b).indexOf('-') == 0) {
        al.set(b, "-");
    }

    if (al.get(b).indexOf('-') == al.get(b).length()-1 && al.get(b).length() != 1) {
        al.set(b, al.get(b).substring(0,al.get(b).length()-1));
    }
 
    System.out.println(b + " " + al.get(b));
}



duelLevelLabel.setText(al.get(53));
huntersLevelLabel.setText(al.get(54));
roguesLevelLabel.setText(al.get(55));
fogLevelLabel.setText(al.get(56));
maLevelLabel.setText(al.get(57));
attackersLevelLabel.setText(al.get(58));
defendersLevelLabel.setText(al.get(59));
collectorsLevelLabel.setText(al.get(60));
healersLevelLabel.setText(al.get(61));
castleLevelLabel.setText(al.get(62));
conquestLevelLabel.setText(al.get(63));


double atk = Integer.parseInt(al.get(3));
double str = Integer.parseInt(al.get(7));
double def = Integer.parseInt(al.get(5));
double hp = Integer.parseInt(al.get(9));
double pray = Integer.parseInt(al.get(13));
double rng = Integer.parseInt(al.get(11));
double mage = Integer.parseInt(al.get(15));
double sum = Integer.parseInt(al.get(49));




double pbase = 0.25 * (def + hp + Math.floor(pray)/2 + Math.floor(sum)/2);
double base = 0.25 * (def + hp + Math.floor(pray)/2);
double melee = 0.325 * (atk + str);
double range = 0.325 * (Math.floor(rng*1.5));
double magic = 0.325 * (Math.floor(mage*1.5));

double top = 0.0;

if (melee > range) {
    if (melee > magic) {
        top = melee;
    }
}
if (range > melee) {
    if (range > magic) {
        top = range;
    }
}
if (magic > melee) {
    if (magic > range) {
        top = magic;
    }
}

System.out.println(melee);
System.out.println(magic);
System.out.println(range);
System.out.println(base);
System.out.println(pbase);

double level = Math.floor(base + top);
double p2pLevel = Math.floor(pbase + top);


      f2pCombatLevelLabel.setText(Double.toString((int)level));
      p2pCombatLevelLabel.setText(Double.toString((int)p2pLevel));

    }

[Ezzaral]

This is what I was talking about in your reading code

statPullerData = statPullerData + str;

It is glomming together each line, so the last number of one line is get concatenated to the number beginning the next. If you want a space to separate them, you would need to use

statPullerData = statPullerData + str + " ";

[aanders5]

Ok, so just add like a ","?

[aanders5]

Ah, I did that and am now reworking the code, I will let you know how it turns out.
prob in like 5 mins. :P

Thanks

[Ezzaral]

Using a small data structure for each entry would make it much easier

class ScoreEntry {
  int rank;
  int level;
  int xp;
}

and using something similar to this to read them

List<ScoreEntry> scoreList = new ArrayList<ScoreEntry>();
while ((str = in.readLine()) != null) {
   counter++;
   String[] values = str.split(",");
   ScoreEntry score = new ScoreEntry();
   score.rank = values[0];
   score.level = values[1];
   score.xp = values[2];
   scoreList.add(score);
}

You would need to make an allowance for the "Activities" portion that only has 2 values. Those may need to be a separate collection of different objects depending on how you intend to use them.

[aanders5]

Ok, thanks. I shall look into that.

1 last Q though before you go.
I just started this whenever I first posted, and it is my first time using netbeans EVER. So, how would I go about exporting this and showing my friends?

I have a domain with a site host, so I can toss it up on there, but...I'm not sure what to do.

Thanks!

[Ezzaral]

You can build an executable jar from Netbeans. Search the help content for "Building a Jar File".

[aanders5]

and how do I run jar's via html again? If that is what I do. lol

I find the jar stuff

right-click your project,
"set main project",
select your project,
"File",
"PROJECTNAME Properties",
"Run",
set your main class,
"Libraries",
"Add JAR/Folder",
select your wanted libs,
do a CLEAN build,
enter the build-path,
enter /dist and be amazed --> DONE!

However I get lost ad the add JAR/Folder. So, I hit that, and it makes a little folder icon in the area that shows my projects.
Then, it asks to me pic my libs, soooo i just picked the project I used "Expirement1". Now...idk what it means by "do a CLEAN build" or enter buld-path and the /dist thingy...

EDIT:I JUST ATTACHED AN IMAGE, IS THAT THE JAR THING I WANT??? If so, do I now just upload it to my website, stick it on a new page, and figure out how to run it? Will I need any other files with it?

[Ezzaral]

Applets can run from a web page. Jar files are standalone Java applications. If your application is not an applet, your friends will have to download the jar and run it themselves.

[aanders5]

Um....How come it can't be an applet itself?

[Ezzaral]

It can be. Did you create it as an applet? I have no idea what you built there based upon the fragments of code that you posted.

Edit: Looks like a standard JFrame swing form in your screen shot.

[aanders5]

also, I am going to sit on this topic since I can't do anything and I am not going anywhere until I figure this out. lol. :)

[aanders5]

here is what I made, A Java Application. (as shown in image)

EDIT: You want me to post whole code?

[Ezzaral]

If it is a standard Java Swing application, your friends will need to download the jar file from you and run it themselves.

You could convert it to an applet if you wanted to, but that would require a little bit of work converting the main JFrame to JApplet. Mostly it just involves moving a bit of code into specific methods required by the applet life cycle.

[aanders5]

Can I do it you think? I would need your help though most likely.

Also, I figured an application was the same thing as an applet for the most part.

[Ezzaral]

In many cases it can be as simple as putting the initialization code that you had in main() into the init() method for the applet. You can read about developing applets here: http://download.oracle.com/javase/tutorial/deployment/applet/subclass.html

[aanders5]

Muwhahahaha!!!! >:) I found a way to do it in 3 steps. lol. :)

Ok, So it is in an applet now and it runs. What's next bro'ski?

[Ezzaral]

See the section on Deploying the Applet

You need to Build the project to create the jar file, upload the file to your server, and create an HTML page that has your <applet> tag in it.

Edit: You'll find the jar file in your project's "dist" directory after you perform the Build.

[aanders5]

ok, I do a build, made the jar of expirement1.

I uploaded it to a folder with an index page as this

<html>
<head>
<title>
Test Apps - Austin Anderson
</title>
</head>


<body>

Testing Applications
<applet code =Expirement1 ='Expirement1.jar'>

</body>

</html>

nothing. :O http://dawnofvengeance.com/applications/

[Ezzaral]

Try changing your applet tag to

<applet code = 'NameOfYourClassHere' 
    archive = 'NameOfJarHere.jar', 
    width = 300, 
    height = 300 />

[aanders5]

whatcha mean by name of class?

[aanders5]

still nothing

<applet code = 'NewJApplet'
      archive = 'Expirement1.jar',
      width = 300,
      height = 300 />

[Ezzaral]

The name of your applet class.