Streaming data without saving to HDD

[bleedi]

Hey,

I'm working on a project where we have to stream files from a server app to clients. I have it all working nicely, but we have a special client type that cannot save files to HDD, not even temporarily. I can get the byte stream from server to this special client, so only the data handling in the client side is the problem.

I was thinking, is there any way to open the streamed files in the client app without saving them first? For example, if I send a .JPG file from server to client, how can I open the file without saving it as .JPG with FileStream first? Can I do this with MemoryStreams or..? At the moment I have a partial system using MemoryStreams, but I haven't found a way to handle all kinds of file types I need. At the moment it can only handle .TXT and .JPG files, but the list goes on, and I really wouldn't like to write my own byte-parsers for all of them :P

Thanks for your answers in advance :)

[skatamatic]

Well, assuming you aren't sending something that takes up more memory than they have, you can just take the raw byte data, held in memory after a tcp receive, and convert it to whatever you choose.

Here's an example with a bmp:

//assume the byte[] is what was received via socket
private Bitmap MakeBMP(byte[] data)
{
     TypeConverter tc = TypeDescriptor.GetConverter(typeof(Bitmap));
     return (Bitmap)tc.ConvertFrom(byteArray);
}

Note that the more complex the resulting object is, the harder it is to cast a byte[] into it. BMP, fortunately, is very straight forward. I'm not too sure if the same is true for a JPEG.

[bleedi]

Well, assuming you aren't sending something that takes up more memory than they have, you can just take the raw byte data, held in memory after a tcp receive, and convert it to whatever you choose.

Here's an example with a bmp:

//assume the byte[] is what was received via socket
private Bitmap MakeBMP(byte[] data)
{
     TypeConverter tc = TypeDescriptor.GetConverter(typeof(Bitmap));
     return (Bitmap)tc.ConvertFrom(byteArray);
}

Note that the more complex the resulting object is, the harder it is to cast a byte[] into it. BMP, fortunately, is very straight forward. I'm not too sure if the same is true for a JPEG.

This is just about as far as I have got now, but I see a problem here. What if I have a complex file format, that isn't open-sourced? It can be serialized to byte array and streamed over the web... But how can I construct, for example, a Unity .prefab file or Autodesk's .FBX file with TypeConverter (or anything else for that matter)?

EDIT:
Also, another problem here is, I have a closed library I have load the item in. The loader needs a file path, so is there a way to map a virtual path to the memory file? I don't have any object classes to represent the file format, like there is in skamatic's example (Bitmap).

[skatamatic]

I'm sure there's an outrageously talented (or informed, rather) hacker on this forum that will prove me wrong, but I don't think that mapping a virtual path is possible in the windows api. Is the library in question looking for a physical path, or a stream (which can often be confused with a path)? Try using something like Dll Export Viewer to look at the library, maybe there's a hidden gem of a function that will do just what you are looking for.

Maybe a more thorough explanation may be in order. If you know the server's IP (or name on the DNS) you could potentially use that as the path:
path = @"\\192.168.5.100\MySharedData\FileInQuestion.ext";

If the file does, in fact, need to be downloaded you have quite the issue on your hands. I recently had to hack a closed library for a USB driver that wasn't properly deleting a structure. It was painful but certainly possible (Ollydbg is a pretty good tool for that kind of stuff). Basically, if you took this route, you would need to find the part of the dll that loads the file from the disk (probably filling a byte array in the process) and redirect that byte array's address to that of your downloaded byte[]. Sounds easy - don't expect it to be anywhere close to easy though haha.

Another option is to contact technical support with the companies that developed the file types in question. Maybe they can offer a different version of the library that supports memory streams/byte arrays or at least some guidance.

Last option is to figuire out exactly what the headers/data is in the file and write your own adapter. You might get lucky and find a reference online, or you might get unlucky and spend countless hours on something that will never work.

Sorry I can't be of anymore help.

[CsharpChico]

Haven't worked much with Server but when getting an image from a web url or byte[] the memstream will work good I prefer to use Bitmap so that I can set the size of the image. Heres a simple method to get image from a url.

Bitmap MyImage(string url, Size size)
        {
            return new Bitmap(Image.FromStream(new System.IO.MemoryStream(new System.Net.WebClient().DownloadData(url))),size);
        }

Note that the new System.Net.WebClient().DownloadData(url) is the byte array you should be able to replace that with your byte array to display the Image.

to use the method you can do something like this

this.BackgroundImage = MyImage("http://ts1.explicit.bing.net/images/thumbnail.aspx?q=892307250356&id=2715889ad14e9a5163ce86adbd8c6bcc&url=http%3a%2f%2fwww.femailfootfetishsex.com%2fgiantess-foot-worship-stories-t66.jpg", this.ClientSize);

hope it helps :)

[skatamatic]

Haven't worked much with Server but when getting an image from a web url or byte[] the memstream will work good I prefer to use Bitmap so that I can set the size of the image. Heres a simple method to get image from a url.

Bitmap MyImage(string url, Size size)
        {
            return new Bitmap(Image.FromStream(new System.IO.MemoryStream(new System.Net.WebClient().DownloadData(url))),size);
        }

Note that the new System.Net.WebClient().DownloadData(url) is the byte array you should be able to replace that with your byte array to display the Image.

to use the method you can do something like this

this.BackgroundImage = MyImage("http://ts1.explicit.bing.net/images/thumbnail.aspx?q=892307250356&id=2715889ad14e9a5163ce86adbd8c6bcc&url=http%3a%2f%2fwww.femailfootfetishsex.com%2fgiantess-foot-worship-stories-t66.jpg", this.ClientSize);

hope it helps :)

If you had read the thread, you would have noticed that he mentioned this is not for data structures in the public domain (ie bitmap or jpg). Instead he has to come up with a method of loading a closed source function that accepts a path to a file when there is in fact no hard drive to save the file to in order to build an object.

[bleedi]

Okay, I've managed to get a reply from the company that owns the file, and they will be implementing the byte array constructor in their next release. We we're just a bit early building our own software. :)

[CsharpChico]

If you had read the thread, you would have noticed that he mentioned this is not for data structures in the public domain (ie bitmap or jpg). Instead he has to come up with a method of loading a closed source function that accepts a path to a file when there is in fact no hard drive to save the file to in order to build an object.

Yes I am aware of that this is why I have mentioned that he can change the byte[] where i get from url to his byte array so it would look like this

Bitmap MyImage(byte[] data, Size size)
{
return new Bitmap(Image.FromStream(new System.IO.MemoryStream(data)),size);
}

Another goog way to use this is also reading from resource when the resource is a binary resource :)