Member Avatar for world_weapon

Hi, just wondering if anyone had any new ideas on a plugin for Pidgin to enable video and voice. One of the problems is the way the data is used by each protocol. I figured that mplayer is a capable backend to display sounds and graphics from various formats. This could allow the plugin to at least receive voice and video. Another thing was to enable a v4l2 device like a webcam with built in mic. If the users had a registered device, they could have the plugin enable it for the program. Each protocol may handle the audio and video differently, so I guess to handle that the plugin would have to create a new v4l2 soft device stream from the main inputs ran through mencoder to produce the proper video and audio objects. That way a single video device (webcam) could be used to broadcast on multiple networks by having each protocol get the software device from the plugin. I have no idea if this is a good start, but after thinking about it, this could be a useful implementation at least to the plugin aspect. If anyone has any thoughts on this, like where to begin, or similair projects, or if you know about some of this stuff, drop a line. I have no idea how to program with libpurple or v4l2. I know how to call mplayer and mencoder from bash scripts buts thats about it. Any ideas are welcome.

  • 1 Contributor
  • 2 Replies
  • 187 Views
  • 2 Months Discussion Span
  • Latest Post Latest Post by world_weapon

Dani AI

Generated

Good start, — your instinct to decouple “protocol handling” from “media presentation” is exactly right. A practical pattern that scales and simplifies per-protocol quirks is to present each stream as a plain system device (virtual webcam / virtual soundcard) and run a small mediator that decodes/transcodes protocol payloads into those devices. That keeps libpurple code focused on signaling and device selection, while a dedicated media pipeline handles format negotiation and timing. (github.com)

Concrete, minimal options you can prototype quickly:

  • Use a virtual video device (v4l2loopback) and a loopback audio device, then have a single feeder process per remote stream convert protocol packets into raw frames/samples and write to those devices. Example ffmpeg-style output to a v4l2 device:

    ffmpeg -i INPUT -vf scale=640:480 -pix_fmt yuv420p -r 25 -f v4l2 /dev/video2

    Build/runtime notes: FFmpeg supports a v4l2 device and can list supported formats; you may need libv4l2 support enabled. (ffmpeg.org)

  • If you want embedding instead of external feeders, implement the pipeline with GStreamer or libav inside your helper process and either push to v4l2loopback (so other apps see a webcam) or render directly into your UI. GStreamer provides v4l2src/v4l2sink and appsrc/appsink elements that suit this pattern. (gstreamer.freedesktop.org)

Practical tips and gotchas: manage permissions to /dev/video* (modprobe/load as needed), pick a negotiated pixel format/frame size and sample rate so consumers don’t block, and use per-stream buffering with timestamps to avoid jitter. v4l2loopback exposes useful runtime controls (exclusive_caps, keep_format, timeout) to help compatibility. For control messages between libpurple and your mediator use a small control socket (Unix domain socket or named FIFO) rather than ad-hoc stdin piping — that makes lifecycle and multi-instance management much easier. (github.com)

If you want, describe the protocol(s) you plan to support first (so the mediator can select codecs/containers); that will let you pick whether to transcode on the fly or pass through native frames to the virtual devices.

Member Avatar for world_weapon

Does anyone have any experience with coding for mplayer frontends? I think that information would be extremely useful in what I am trying to do. I am looking into the the libpurple core library and I have a feeling that is where the plugin should go for integrating mplayer as a media handler. If a libpurple plugin capable of opening and closing instances of mplayer can be created, then the core of the software will have the mechanism to at least receive signals and do SOMETHING with it. So far I am not so much interested in capturing video and audio, because that to me seems the easiest to research how to do for v4l,v4l2 and alsa/oss APIs. I'm sure there is copy and paste code lying around for that. But what I really want to work on is creating a libpurple plugin to create an Mplayer instance. I'm thinking exactly along the lines of how an Mplayer frontend would do it. Difference would include the fact that the plugin should handle multiple instances of mplayer so that it can handle receiving multiple streams of audio/video. This is necessary for the program to be able to display more than one incoming webcam/voice stream from the multiple clients libpurple can handle.
I really think that that once the protocols video/audio stream format is identified, mplayer more than likely can handle it. I have seen Mplayer play some corrupt files. I have no doubt that Mplayer is more than capable of handling the diversity of multiple messaging protocols video/audio streams. As soon as I successfully implement this, or find similar code and modify it to comply with libpurple plugin scheme, I will start a sourceforge software entry. I think this could really fill the gap for Video and Voice on libpurple(Pidgin). I understand from the developer trac at pidgin.im that this would be difficult to implement directly into libpurple, but in the meantime, I feel this type of plugin would do just fine. Just trying to get started anyways. Any help, knowledge, insight, or even code would be very much appreciated. Hopefully there are other open source enthusiast here who would like to see Pidgin handle webcam sessions and what have you. Thanks for any input that will help organize or develop this concept.

Member Avatar for world_weapon

Ok, so far I have figured out that the best way for me to go about this, is to make a program that creates child processes of mplayer. Basically the plugin will call this program with pipes handling the communication between the libpurple and the instance of the program. The program will then create pipes and fork itself according to the arguments coming in from the plugin pipe. Based on the input, will fork() and execlp() mplayer to play a stream. I just haven't figured out how to get the data from the plugin, to the instance of mplayer. Anyone got a clue? Thanks in advance.

Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.