If you want to build an API, then you need to have the skills and experience to do it. This isn't a job for a novice. It is doubtful that anyone here has the specific experience that you are looking for. Even less likely that if they do that they are going to try to 'guide" you through an advanced development project like this.
PHP is a server side language. It does not have access to the client machine. If you wanted to do something like this, you would need some kind of client based code (flash or java probably) which could possibly use php to drive some of the underlying functionality, chat, logging, authentication etc.
To retrieve the video/audio stream from the browser you need a browser plugin that has access to the webcam. The most common way to do this right now is with Flash. (I'm guessing you want this to run in the browser).
You'll need to write some ActionScript to get the video and audio from the webcam and stream it to a server. Then the server needs to be able to send that video/audio stream to the receiver of the video.
For the actionscript portion there are many examples online, or you may be able to find some open source code that does it.
You can write a RTMP server in PHP, but it would be re-inventing something in a language not meant for the task. RTMP streaming requires good IO, Memory utilization and networking which PHP is not meant for.
You may also want to look into XMPP, which is an Open Source Instant Messaging Protocol. There are also many existing examples and open source code around XMPP, even video and audio streaming such as with the OpenFire Server and RedFire Server.
Note that RedFire is the OpenFire XMPP server using Red5 to stream the video and audio wrapped in RTMP. It includes the whole sources for java based server and flash client.
Can I implement this OpenFire Server with Apache, Mysql, and PHP?
Is there any tutorial for this? I am a beginner in developing field .So please guide me.
OpenFire server primarily sends XML over TCP conforming to the XMPP specification or other media such as video and audio streams under specifications such as RTMP.
Apache is a webserver and primarily sends HTML over HTTP conforming to the HTML specifications.
So the two are separate and for different purposes. There are specifications such as BOSH which allows XMPP over HTTP, but I'm not sure if Apache is the best tool.
In other words, OpenFire runs by itseslf, it does not need Apache or MySQL or PHP. You can however integrate them for your needs, such as have Openfire write messages to MySQL, or talk to Apache and PHP.
I would not be able to show you through this forum how to do all this, neither can anyone else. You'll have to start researching XMPP, TCP, XML, OpenFire etc. over the web and asking specific questions in your progress.