Thanks... work very well on LAN for cell number from the first example (you modified it later no?). On internet you can see the labels delay... and if you write something from server screen (about 3,4 modified cells right?) you get about 2 seconds delay. However the more changed cells, the mode delay, but I can say your way is very good to follow. I'll get back tomorrow... with new ideas I hope.
Best regards,
Clawsy

Thanks... work very well on LAN for cell number from the first example (you modified it later no?)

Earlier I was dividig the screen into 16X16 tiles (better word than "cells"). Later I found a better trade-off dividing into 8x8 tiles (ie about 160x128 pixels each). The tile size is definitely an interesting thing to play with, as more/smaller tiles increase the CPU usage on the server, but reduce the amount of data to send. In my case I'm more worried about server CPU load.

musthafa: You have my permission to use my code in your application, but you must place a comment
// uses code provided by James Cherrill 2010
in your program near where my code is used.
Cheers
James

definitely if i apps succeed i put your name in my code as courtesy ...

On internet you can see the labels delay... and if you write something from server screen (about 3,4 modified cells right?) you get about 2 seconds delay.

This is puzzling. I've added some more statistics to the server, and when nothing is happening much on the screen the updates are under 100kB each. That shouldn't take 2 seconds unless you're on a modem connection!
I've attached the version with these stats, no delays, and with the tiles back to 256 tiles/screen (80*64 pixels) so we can see
what that tells us on your internet connection.

Ok, now have 2 more ideas.
1. After we extract the pixel data, to determine the coordinates of the rectangle so that you will not send the whole tile anymore but only the changed rectangle. Here is the benefit:
-> The screen split the screen in fewer pieces or just leave 8... however lower CPU usage
-> The data sent will be smaller. For example, if 12x12 pixel image changed, only 12x12 pixel data will be send instead of 160x128.
I will work one that.
Of course we have also to send the x,y coordinates of the changed rectangle and draw it at that position in the cell i,j.

2. This is interesting but maybe not so realistic... I wanted to try if I can substract two pixel arrays and send the remaining pixels only. Other pixel would have a color value to be transparent cause when we draw the image at the client we don't want to "erase" the pixels on which we draw the pixels that have the unimportant value (that's why we need them transparent).
I don't know if you understood something from the second idea but first would be great.
If we solve this issue too I think it's enough for this thread... as it's not ment for network transfer but for image processing. Transfer maybe in another thread.
I start coding on it. Think, code and debug :).

Hi Clawsey. Interesting ideas!
1. Shouldn't be too hard to try this - there's a loop that looks for changes between the prevoius and latest pixels for each cell/tile, so that's a place where the lowest& highest x&y can be recorded to define the smallest rectangle containing all the changes. On the other hand, with 256 tiles each one is pretty small, and there may not be much to gain. I'm still wondering what speed internet link you are using, and why the data rate seems so low (is there a high latency? - what's the ping?).
2. Each pixel already has an alpha value in its first byte, set that to 0 and the pixel is totally transparent. So you could do as you say and set all the unchanged pixels to alpha=0. You can then set the RGB values to some constant value (because they don't matter for totally transparent!) and compress the whole image as PNG which is ultra-efficient at compressing runs of identical pixels. I think the is potentially a really great idea, and should be pretty easy/small to code.
I can give you the code for sending and receiving an Image with PNG compression if you like - it's really small.

Should I try the 1st idea first or the second? Maybe start with 1... As for the code to send PNG, that's why I used till now so I guess I have it: a BufferedImage written a BufferedOutputStream using ImageIO and PNG compression then mahe that buffer an array of bytes and send it over the connecsion. I did that.. but the mai part is to deal with the pixel data - I mean the second idea.
So I'll try the first idea first. I have ADSL connection and speed of 4Mbps = about 500KB/s maximum download speed. In LAN it's very fast but internet is something else. + I have a router with my port forwarded.

If I understand correctly - you dont need to do the stuff with a byte array to use png - the following should be simpler and faster:

OutputStream out = clientSocket.getOutputStream();
ImageIO.write(image, "png", out);

and, at the client end...

Image remoteImage = ImageIO.read(socket.getInputStream());

Hmm... nice, I didn't tried that yet. However I have some wuestion about my second idea... with the pixels... I don't fully undestand yet the RLE encpding algorith, even if I understand the general purpose (I read about it)... and I have 2 things I'm thinking about:
1. If I try the second idea I don't need to split the image anymore, do I? Just make those pixel transparent then compress the px data using RLE the send it.. or use PNG compression as you sai.d
2. If I split the image and modify your example with tiles, please help me to find out where and how can I check the pixel change (x,y) to make that pixel 0 and then give any RGB value to that pixel... I mean where to modify cause I dont undertand 100% all the variables yet, even if I understan overall the encoding and the 'cells'.

I would try the second idea. Don't use my RLE because that uses the alpha byte, and we need the alpha byte intact. Use PNG. And no, there's no reason to split the image with that. I think it's got great potential.
I'm offline now till tomorrow.

Ok, I modified here at the sender:

for (int k = 0; k < pixels.length; k++) {
							if (pixels[k] != prevPixels[k]) {
                                                          //  System.out.print(" "+pixels[k]);
								prevPixels[k] = pixels[k];

								cellChanged = true;
							}
                                                        else{
                                                            pixels[k]=0;
                                                        }
						}

and at the client:

void RLEdecode(int[] data, int[] target) {
		int index = 0;
		for (int i = 0; i < data.length; i++) {
			int value = data[i] | 0xFF000000; // reinstate Alpha value 255
			int count = data[i] >>> 24;
	  	   // System.out.println(index + " = " + count);
		   for (int k = 0; k< count; k++) {
                       if(value==2) target[index++] = 0;// i need to set RGB value to be 2 at sender... how to do this?
                       else
		   	target[index++] = value;
		   }
		}
	}

I tried directly to run this... and I can see modifying only pixels that change at server! but the rest are... black :)). However I can't wait to do this. Thanks for so much support James... it's so nice to work togheter at this issue.

- I still have about 190-200 ms delay at image PNG compression. (ImageIO.write(...)). The 'grabbing' is much faster... so I have no problem with it
- I use drawImage to draw the image at the client but the pixels set with 0 at the server are black.. not transparent. I might miss something.

If you are using my RLE compressor, that was written assuming that the Alpha value was not being used - so it uses that byte to hold repeat counts, and forces it back to Alpha = 255 (100%) when decompressing. Obviously that's not going to work if you want to send transparent pixels and keep their transparency!
I have an idea for incorporating the changed/unchanged info in my RLE compressor that may just work very well. I'll try to do some work on it, but I'm a bit busy at the moment. I'll get back to you when I get a chance.

I didn't used your RLE but I thinks I have too... if oyur idea will work it would be nice to try... cause the PNG compression doesn't help with timing even it a very good compression algorithm.
Good luck on the idea... I'll try to think about that too.

OK, my repution hangs on this one...
Server & viewer jars as before but with my latest non-tiled RLE compressed algorithm. This also compresses runs of consecuitive pixels that have not changed into a single int. Update volumes are down to 6kB when nothing much is happening, refresh rates are still up to 5 per sec without the delays.
Let me know, OK?

So you modified your RLE? for a single image? Without taking out alpha? to support transparency... cause this example works really well too but I can only test it when I'll make mine work with RLE :) because I can test in on internet. Have to get rid of that delay.
So my question is if you customized the RLE... as you said. I would appreciate it very much.. cause PNG compression means a lot of delay.
I added reputation for you :). You deserve it... but I don't undestand why "your reputation hangs on this"?
Thx.

... I don't undestand why "your reputation hangs on this"?
Thx.

I was just that if this didn't work I'd look like a fool - nothing to do with DaniWeb!
This version takes the current array of ints and the previous version, and compresses into a single stream. Consecutive runs of unchanged pixels are coded as a single int (ie number of pixels unchanged), changed pixels are compressed by ordinary RLE. At the client the previous int array buffer is kept, and only the changed values are overidden. I then re-create the image from that buffer each time. No PNG, no transparency, so its simple & fast.
I've just got it working, so I'll clean it up a bit and post the source tomorrow.
ps You should be able to use those jars over the internet, just give the viewer the IP address of the server when it asks. The server listens on port 1234, so you may need to open that thru the firewall and/or router.

Ok :) ... however my idea of pixel change and transparency still remains? I mean Is it possible to compress it wit your RLE or you need alpha for RLE and cannot transfer it too? I was getting PNG of 4,5K but the delay of compression was too big. If your RLE would work with my idea... would be so nice. But however you current compression algo is very nice and fast. I enjoy it.

Yes, this is based on your idea of pixel change, but instead of transparency I just send a code of how many pixels to leave unchanged. The (current) comments from the code may help...

// Output stream format is sequence of 1 integer records describing the
// data array in natural order (element 0 ... data.length-1)
//
// Unchanged record (sequence of >=1 unchanged pixels):
// bit 0 is 1,
// bits 8-31 are number of consecutive unchanged pixels.
//
// Changed record (sequence of >=1 identical changed pixels):
// bit 0 is 0,
// bits 1-7 are number of consecutive pixels the same,
// bits 8-31 are the 3 byte RGB values for these pixels.

commented: effort!!!! +0

Aha... nice. A real bit algorithm :). I can't test it on internet because I have some problems with forwarding with my router... but maybe you can post tomorrow, or you can share the code in my inbox :) (I mean there's your work and ideas... and... maybe some ideas from me too... if you agree :) it's your choice how you share it to me).

yse clawsy is correct is our ides and thoughts...

so you please post your reply to my inbox also or in daniweb...

because i also make my apps as per your suggestion...

but clawsy says remains slow ...

but i till change my apps with your ideas along with i now doing another apps..

that's why i can't reply what happened so far..

any how James post your ideas...and we will try and hopefully we can get good result...

my part i am doing now and ho[efully i will be back with good ideas and result...

OK Guys, here you are - actual source code.
As before, you need to establish a socket-socket connection and pass the sockets to the server & client classes.
For fastest possibel refresh, set the targetRefreshInterval to 0.
With nothing much happenening on the screen the updates are now as small as 1 - 2KB. I can't believe we can do much better than that!
Have fun, let me know how you get on.
James

Aaaarggggg! Sorry Guys, the previous post has the WRONG VIEWER code.
Here's the correct one.

Brilliant James. Very nice. I'll test it and tell you.
Have a question... close the thread?

Have a question... close the thread?

I think that depends on what you find. If you think it's as good as we're going to get then, yes, we're all done with ths topic. If not, let's keep going until we have a good answer :-)

how do split as per our suggestion and make it....

it little confuse to replace byte array as int[]..

if i made it it produce blank image...

public class robot {

    private Robot rt;

    private Toolkit tk = null;
    private Rectangle screenRect;
    private Rectangle ScreenRect = new Rectangle(0 , 0, 920,718);
    public robot() {
        tk = Toolkit.getDefaultToolkit();
        screenRect = new Rectangle(tk.getScreenSize()); 
        try {               
            rt = new Robot();
        }
        catch (AWTException awte) {
            awte.getStackTrace();
        }     
    }

    public BufferedImage captureScreen() {    
        screenRect = new Rectangle(tk.getScreenSize()); 
        return rt.createScreenCapture(screenRect); 
    }

    public byte[] CaptureScreenByteArray() {  
        return ImageUtility.toByteArray(captureScreen());
    }  

 public static byte[] toByteArray(BufferedImage image) {
        try {
            ByteArrayOutputStream out = new ByteArrayOutputStream();
        
            write(image, 0.4f, out);
            return out.toByteArray();
        } catch (IOException e) {
            throw new RuntimeException(e);
        }
    }

    
    @SuppressWarnings("unchecked")
	public static void write(BufferedImage image, 
            float quality, OutputStream out) throws IOException {
        Iterator writers = ImageIO.getImageWritersBySuffix("jpeg");
        if (!writers.hasNext()) {
            throw new IllegalStateException("No writers found");
        }
        ImageWriter writer = (ImageWriter) writers.next();
        ImageOutputStream ios = ImageIO.createImageOutputStream(out);
        writer.setOutput(ios);
        ImageWriteParam param = writer.getDefaultWriteParam();
        param.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
        param.setCompressionQuality(quality);
        
        if (quality >= 0) {
            param.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
            param.setCompressionQuality(quality);
            
        }
        writer.write(null, new IIOImage(image, null, null), param);
        ios.close();
        writer.dispose();
    }

}

@musthafa.aj: I don't really understand what are you trying to do.. send split images? Sorry I'm so busy...

@JamesCherrill: I think thats the best way till now... to send compressed pixel data and have a good image flow / performance. However I'll have to think more about this case... being uncompressed bitmap it's so big - I mean the first frame. I send a screenshot with a lot of colours... (my wallpaper) and it sent 3385kB image :). I think that's why applications like TeamViewer make your background wallpaper one colour before the transmision starts. I guess I'll do this trick with the background too :).

Yes, the first frame is always going to be a big hit, and a colourful background is going to kill the GIF/PNG type compressions that you need for text etc. A 1280*1024 24 bit colour image is 3840k of raw data, so 3380k isn't surprising. For certain, replacing the background picture with a solid colour will give a big improvement. You're right - that IS why commercial products like to grey out the background. Maybe, just for the very first screen, it would be better to revert to the PNG version, then switch to my differential RLE thereafter???
musthafa's code (which I don't really understand) uses JPEG, which will be good for the background picture, but horrible for text - I think JPEG is not appropriate.

commented: good effort!!! +1

yes... I thought about that... if the image size is so big I can send as PNG... at leas try it. I'll see and tell you. I send at 1ms every time (refresh)... The problem is not just the big size but the time of pizel processing is big when there are lots of colours. I'll give it a try when I have time. Thxs. Keep in touch :). Have nice day!

james..

i check the speed at internet ...

it is really what i expected...

i little bit speed than my own apps...good:)

but i fiddling with mouse event with your code can you ...

can you help me in your way how do send keyboard and mouse event from client side(screenwatcher) to server(screen sender..)

my apps have it own but when i implement your idea..i missed something ...

plz help me..

then if will come to conclusion we will close this thread with confirmation from clawsy...:)

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.