Getting text from webpage that has embedded flash

[themush]

Hi,

Im trying to parse a bands myspace page and get the total number of plays for their songs.

The problem is that I cannot use urllib2 as the "Total plays" string
does not appear in the page source. e.g.
http://www.myspace.com/mybloodyvalentine

Any idea of ways around this?

Thanks,
O

[zachabesh]

This seems difficult/probably not feasible as the flash app would be server side, and not really available to the user.

[JasonHippy]

The short answer is there is no way around it.

The flashvars that get passed into the myspace media player .swf are merely the parameters the player needs to be able to rettrieve the artists details and playlist from a database. The flashvars do not contain the total play count or any other usable information!

The database is the only place that'll have the artists total play data and you won't be able to access that because all of the database connection details for the media player will be encrypted and abstracted away somewhere out of sight, either in the swf itself or in another server side script...So the bottom line is you're not going to be able to do it!

Cheers for now,
Jas

[themush]

Thanks for the replies. It seems like there must be some roundabout way to do it. The text is selectable in the browser when the page is rendered.

But alas, maybe not.

[JasonHippy]

Thanks for the replies. It seems like there must be some roundabout way to do it. The text is selectable in the browser when the page is rendered.

But alas, maybe not.

No, I'm afraid not. The selectable text in the usage statistics is a part of the .swf itself.
In other words, when the myspace media player .swf was published from flash, the dynamic text fields in the .swf had their selectable property set to true (which is the default setting for dynamic text in flash!). This is the only reason the statistic text is selectable in the browser. If the creators of the myspace media player had set the property to false, the numbers in the usage stats would be unselectable!

So when you select the total plays text and copy it to your clipboard it is done via flash (But I'm not sure of the exact mechanisms employed!).

Again, I don't think that there is any way that you could access the contents of the .swf as the statistics are dynamically loaded at runtime. So simply finding the .swf in your browser cache and analysing the .swf with python wouldn't yield any results for you.

The only way I can think of, would probably be (and I'm clutching at straws here) to load the myspace page into your browser and examine the contents of your pcs memory once the media player .swf has loaded fully and started running.. You'd have to find the block/blocks of memory being used by the .swf and try to track down where the usage information is kept...It'd be like looking for a needle in a haystack!

You'd probably have to search memory manually a few times with some kind of Hex memory dump utility and then once you've managed to isolate the information you're looking for you'd need to devise an algorithmic method of tracking the required information down before being able to program anything in Python.

And then you would have to have your browser open at the desired myspace page for your script to work....

So if this is even remotely feasible (which I don't think it is!), it's a hell of a lot of effort to go to to try and track down how many plays a band has on myspace...It'd be quicker to just load your browser and view the page! heh heh!

You can quite easily parse a myspace page to get the total page views though...This information is published directly in the HTML page itself...But I'm guessing you've already been there and done that!

Cheers for now,
Jas.

[themush]

never say never. but that sounds like a lot of work.

thanks for the detailed reply.