Video recording and transcoding/compression in silverlight

[asifjavaid]

Hi All,

I want to develop an application in which I want to record a video in silverlight and send it to the server. I have the following questions in my mind.

1. In which format the video will be record in silverlight using Webcam ? (Raw format OR WMV OR anything else ?)
2. If it is a Raw format, then can we transcode it to WMV OR any other format?
3. Where the recorded video will be saved in silverlight? I think it will be on Isolated space. What will be the best way to send the recorded video to server to save it which will take less bandwidth?
4. How can we encrypt the video in silverlight if we need to save the bandwidth?

Any help will be highly appreciated. Waiting for the reply.

Many Thanks.

Asif Javaid.

[Ketsuekiame]

1. Can sometimes depend on the Camera implementation, but generally if you use Microsoft libraries it will be WMV.
2. Yeah, just find a library to do that for you.
3. It will be stored in Silverlight's local storage. You may not have enough room to store an entire video so it could be better to stream it to your server. WMV is already compressed, I wouldn't compress it again.
4. Encryption won't save you bandwidth, but will increase it. Only encrypt it if the data is sensitive. In this case, you could encrypt a certain amount of video frames and send it to the server in a burst. Decrypt and store until the video is complete, then re-encrypt it once you have the last key-frame. (Server side)

[asifjavaid]

Many thanks for the answer.

1. Can you give me any sample code or a link in which we can learn how to stream the video and save it to the server directly, both client and service code samples ?

Thanks.

[Ketsuekiame]

Sorry but that's a bit complex for me to write out 'sample' code without giving you the entire implementation (therefore teaching you nothing)

However look at the System.Net.Sockets.TcpClient and System.Net.Sockets.TcpListener classes. The MSDN pages should tell you all you need to know in order to get a basic implementation going.

[asifjavaid]

Hi Ketsuekiame,

I have got lot of understanding how to proceed with this further.
Yes you are right I will use streaming to the server.
For sending webcam stream to a specific IP Address and port, I have gone through the article http://www.silverlightshow.net/items/The-ABC-of-streaming-in-Silverlight.aspx

But in this article, it shows that the streaming is dependent on Expression Encoder OR VLC. But we want to make it browser application, so that in production environment there must be a way to send webcam stream to the server without using anything third part software.

Could you please guide me the best way OR an SDK for silverlight to send the webcam stream to the server in silverlight?

Thanks.

[Ketsuekiame]

That link is looking at things backwards to the way you actually want to operate.

What you need to do is grab the raw data. This MSDN link will help with respect to the Capture Device, but you want to use a FileSink to save the data. You can then upload chunks from this to the server.

[asifjavaid]

I am using Silverlight Application project, Visual Studio 2012.
In System.Windows.Media namespace
AudioSink is availble
VideoSink is available
FileSink is not available.

I think it is available only for Silverlight Windows Phone.

[Ketsuekiame]

Ah yes, seems you're right there. In that case use VideoSink

[asifjavaid]

Ah right, so I think here are the steps I have to follow, and please correct me if I am wrong anywhere.

1. In VideoSink there is a method which I need to override.

   protected override void OnSample(long sampleTime, long frameDuration, byte[] sampleData)
   {
   }

2. I have to write a Tcp Stream Socket listener SERVER, which running on a specific port. This server will create a new socket for each client. In this project the client will be the Silverlight Application.

3. In the OnSample method, we need to send the byte array to the Server which will then right the byte array to the file. And because the byte arrays are raw (uncompressed) data then we have to encode it on to specific format avi/wmv before writing to a file.

Any more suggestions and precautions which I need to take care of you are welcome to suggest:).

Many Thanks again.

[Ketsuekiame]

Everything is okay up to number 3. I would, to avoid issues, write all the data to file uncompressed. Only when you close the stream would you then encode to your specific format.

Additionally, I would use local storage to buffer data. The reason I say this is because your video may have a higher bitrate than your stream can handle. The downside to this, is that if the user closes their browser, potentially you will lose a lot of data.
If you don't do this, it could have the consequence of causing your video to stutter and application to slow down (if you block while uploading data).

There's no guaranteed way to ensure everything you capture from the webcam ends up at the server, especially where the user's bandwidth is limited. Best you can do is buffer what you can and send as much as possible, warning the user if the upload hasn't finished yet or you're running out of buffer space (say 50Mb)

[asifjavaid]

Right, I understand the things more better now. Now I have one confusion.
1. If we use local storage butter data, in silverlight i think that will be its isolated space. There must be a available space issue, because, I have created video from webcam can which is uncompressed AVI format. Its size is almost 5 seconds but its size is very huge 161MB. And that will might be a problem in case of long duration videos.

1. What you suggest If I write all bytes in a text file which will be stored in isolated space, and then in onStop event then sent that file to the server but issuing a warning to the user ? Then at the server by recieving the text file, read it and create the video file at the server by using AVI/WMV wrapper.

[Ketsuekiame]

No, I suggest you write data to isolated space, but at the same time, take as much as is possible from the beginning of that file and upload it to your server. This will give you a buffered stream.

So your capture class places data into the file buffer and your server class takes data out of the file buffer.

Also, yes that's a lot of data and very large. Looks like you're going to have to compress on the fly anyway (which will make things fun...). Take a look at FFLib.Net and the Windows Media Format SDK

Beyond this there's not much else I can offer.