xuggler rtmp advanced api tutorial

I posted a short tutorial on transcoding mpeg4/amr incoming streams to
rtmp using xuggler advanced api.

http://www.brokenmill.com/2010/08/xuggler-rtmp-howto/

Hope it helps
Stefan

Thanks a lot.

Thanks Stefan!  That's awesome.




--
http://www.xuggle.com/
xu‧ggle (zŭ' gl) v. To freely encode, decode, and experience audio and video.

Use Xuggle to get the power of FFmpeg in Java.

Thank you for that tutorial !

> rtmp using xuggler adva...

--
You received this message because you are subscribed to the Google Groups "xuggler-users" grou...

Yes, it may be used with any of Red5, Wowza and Adobe server.
Will post a short update as soon as I get some time - in a couple of
days about:
- publishing to a server
- adding video overlay - text/graphics - direct on the frames

Meantiome please read this xuggle presentation, it might answer some
of your questions

http://www.brokenmill.com/2010/03/xuggler/

Hope it helps
Stefan

The only thing you need to do based on the tutorial I posted is to
define
urlOut = "rtmp://localhost/<red5 app name>/<stream name>";

and to call
createOutput(urlOut);

The procedure is defined here:

Updated the tutorial... for red5 pushing.

Hope it helps.
Stefan

This was indeed extremely helpful. I have this live streaming a
desktop capture in H.264 to Wowza based on the bits and pieces I've
picked up here.

There is one thing I can't seem to solve, and perhaps someone can
help. Xuggle seems to hold or buffer the first packet for quite a long
time, perhaps 8-12 seconds. This effectively seems to add a 8-12
second delay to my live stream, which isn't good at all. I think it
must be a setting somewhere to make the buffer 0, but I can't find it.

It's supposed to be writing 5 frames per second, which it does after
the first packet. But the first packet isn't complete for many, many
frames. After that I get a packet almost every frame.

if (packet.isComplete()) {
container.writePacket(packet);
}

I tried

container.setInputBufferLength(0)

but this didn't seem to help.

Can someone point me in the right direction?

Hi Stas,

Can you give a quick code-snipper for folks to show them how to force a keyframe.  It's not actually obvious (and the idiot who wrote the Javadoc didn't make it super clear*),

- Art

* Yes, I was that idiot.

Thanks for all the replies. I really appreciate it. Reducing the GOP
did help somewhat. I've got about 1 keyframe/second. This helps people
who join mid broadcast. But I still have the initial 9 second or so
delay lingering.

From my logging I see:

...
[ENCODER] encoded image 37 in 31ms
[ENCODER] encoded image 38 in 47ms
[ENCODER] encoded image 39 in 32ms
[ENCODER] writing packet of size 112468 for elapsed time 7969ms
[ENCODER] encoded image 40 in 2828ms
[ENCODER] writing packet of size 169 for elapsed time 219
[ENCODER] encoded image 42 in 172ms
[ENCODER] writing packet of size 5237 for elapsed time 187
[ENCODER] encoded image 43 in 203ms
...

So basically it doesn't encode a packet for the first ~8 seconds. When
it does, it takes a full 2.8 seconds to encode. After that, I get one
packet every ~200 ms as I would expect.

I'm really close. But that startup delay is killing me.

I tried setting the first frame to a keyframe as follows:

IPacket packet = IPacket.make();
IConverter converter =
ConverterFactory.createConverter(currentScreenshot,
IPixelFormat.Type.YUV420P);
IVideoPicture outFrame = converter.toPicture(currentScreenshot,
timeStamp);
outFrame.setKeyFrame(true);
outFrame.setQuality(0);
coder.encodeVideo(packet, outFrame, 0);

This didn't seem to make a difference.

Regards,
Daniel

On Aug 26, 12:22 pm, "Sivakiran Yellamraju" <[email protected]>
wrote:
> To unsubscribe from this group, send email to [email protected] <mailto:xuggler-users%[email protected]> .
> For more options, visit this group athttp://groups.google.com/group/xuggler-users?hl=en.
> To unsubscribe from this group, send email to [email protected] <mailto:xuggler-users%[email protected]> .
> --http://www.xuggle.com/

You shouldn't be stuck if you follow the tutorial. Use the oflaDemo
for testing, before implementing your own red5 application. (test if
pushing works first by using the ffmpeg binary like here:
http://blog.xuggle.com/2010/06/18/xuggle-now-has-librtmp-support/)

If this works, them implementing a xuggler app for publishing rtmp
like in the tutorial should be straightforward.

Stefan

Here's a full, working Xuggler rtmp H.264 screen capture application.
I think a lot of people have requested this. It's simplified to the
absolute minimum working code, at least as far as I can tell. It very
reliably produces the buffer delay. It took me about a week of hacking
to get this working, but I'm putting this out for everyone in hopes
that someone can figure out what I'm overlooking. I really need to get
rid of the delay!

Note that this is just a reference implementation. My actual
implementation has a ton of performance optimizations in it, but this
does work and illustrates the delay problem nicely.

The receiving server is currently Wowza 2 in live-record mode.

import com.xuggle.xuggler.Configuration;
import com.xuggle.xuggler.ICodec;
import com.xuggle.xuggler.IContainer;
import com.xuggle.xuggler.IContainerFormat;
import com.xuggle.xuggler.IPacket;
import com.xuggle.xuggler.IPixelFormat;
import com.xuggle.xuggler.IRational;
import com.xuggle.xuggler.IStream;
import com.xuggle.xuggler.IStreamCoder;
import com.xuggle.xuggler.IVideoPicture;
import com.xuggle.xuggler.video.ConverterFactory;
import com.xuggle.xuggler.video.IConverter;
import java.awt.AWTException;
import java.awt.Rectangle;
import java.awt.Robot;
import java.awt.image.BufferedImage;
import java.io.IOException;
import java.io.InputStream;
import java.util.Properties;

public class XugglerRtmpReferenceImpl {

private static String url = "rtmp://your.test.server/screen/";
private static String fileName = "test/teststream";
private static int framesToEncode = 60;
//screen capture size/pos
private static int x = 0;
private static int y = 0;
private static int height = 480;
private static int width = 640;

public static void main(String[] args) {
IContainer container = IContainer.make();
IContainerFormat containerFormat_live =
IContainerFormat.make();
containerFormat_live.setOutputFormat("flv", url + fileName,
null);
container.setInputBufferLength(0);
int retVal = container.open(url + fileName,
IContainer.Type.WRITE, containerFormat_live);
if (retVal < 0) {
System.err.println("Could not open output container for
live stream");
System.exit(1);
}

IStream stream = container.addNewStream(0);
IStreamCoder coder = stream.getStreamCoder();

ICodec codec =
ICodec.findEncodingCodec(ICodec.ID.CODEC_ID_H264);

coder.setNumPicturesInGroupOfPictures(5);
coder.setCodec(codec);

coder.setBitRate(200000);
coder.setPixelType(IPixelFormat.Type.YUV420P);
coder.setHeight(height);
coder.setWidth(width);
System.out.println("[ENCODER] video size is " + width + "x" +
height);
coder.setFlag(IStreamCoder.Flags.FLAG_QSCALE, true);
coder.setGlobalQuality(0);

IRational frameRate = IRational.make(5, 1);
coder.setFrameRate(frameRate);
coder.setTimeBase(IRational.make(frameRate.getDenominator(),
frameRate.getNumerator()));

Properties props = new Properties();
InputStream is =
XugglerRtmpReferenceImpl.class.getResourceAsStream("/libx264-
normal.ffpreset");
try {
props.load(is);
} catch (IOException e) {
System.err.println("You need the libx264-normal.ffpreset
file from the Xuggle distribution in your classpath.");
System.exit(1);
}

Configuration.configure(props, coder);
coder.open();
container.writeHeader();

long firstTimeStamp = System.currentTimeMillis();
long lastTimeStamp = -1;
int i = 0;

try {
Robot robot = new Robot();
while (i < framesToEncode) {
//long iterationStartTime =
System.currentTimeMillis();
long now = System.currentTimeMillis();

//grab the screenshot
BufferedImage image = robot.createScreenCapture(new
Rectangle(x, y, width, height));

//convert it for Xuggler
BufferedImage currentScreenshot = new
BufferedImage(image.getWidth(), image.getHeight(),
BufferedImage.TYPE_3BYTE_BGR);
currentScreenshot.getGraphics().drawImage(image, 0, 0,
null);

//start the encoding process
long timeStamp = (now - firstTimeStamp) * 1000; //
convert to microseconds
if (i == 0) {
//make first frame keyframe
outFrame.delete();

if (packet.isComplete()) {
container.writePacket(packet);

System.out.println("[ENCODER] writing packet of
size " + packet.getSize() + " for elapsed time " + ((timeStamp -
lastTimeStamp) / 1000));
lastTimeStamp = timeStamp;
}

System.out.println("[ENCODER] encoded image " + i + "
in " + (System.currentTimeMillis() - now));
i++;

try {
// sleep for framerate milliseconds
Thread.sleep(Math.max((long) (1000 /
frameRate.getDouble()) - (System.currentTimeMillis() - now), 0));
} catch (InterruptedException e) {
e.printStackTrace();
}
//Thread.sleep((long) (1000 / frameRate.getDouble()));
}
} catch (AWTException e) {
e.printStackTrace();
}
container.writeTrailer();

So it looks like...

packet.isComplete()

...is never 'true' until image 37?

for what its worth.... WHat is the purpose, and can you 'wait' for it to be
complete rather than blasting past and dropping it?

It's a live stream of desktop screen capture. So I can't let it get
that far out of sync. In other words, the video that is playing is
8-10 seconds behind user's actions. And that just won't do for a live
stream.

It's pretty clearly an H264 issue at this point. If I just change the
codec:

ICodec codec = ICodec.guessEncodingCodec(containerFormat_live, null,
null, null, ICodec.Type.CODEC_TYPE_VIDEO);

and comment out the configuration code everything works perfectly.
Except that I get huge files. This is making me depressed :( So close
and yet so far...

Don't use B-Frames; If you set your B-frame threshold to a non-zero number then the H264 encoder has to buffer frames. For a full definition of the types of settings x264 can use, well, that's beyond the scope of this list, but check here for how x264 thinks of low-latency encoding:
http://x264dev.multimedia.cx/?p=249

And then see libx264.c in the captive/ffmpeg/csrc/libavcodec directory for how FFmpeg command line options map to x264 command line options.  Then in Xuggler, set the relevant settings through the setProperty command.

Finally, document what you learn here so others can learn :)

- Art




--

Hi again

Actually I made an application to stream FROM iphone/blackberry to a
server which transcodes and publishes to red5 in realtime using
xuggler, something like qik if you now. And the results were quite
amazing. But I defined my own media sending protocol.

There are some ffmpeg additions on my blog for depacketization of rtp
streams (amr and h263+), I do not know if h264 is supported out of the
box in the new ffmpeg.
http://www.brokenmill.com/2009/11/ffmpeg-amr-rtp-depacketization/
http://www.brokenmill.com/2010/01/ffmpeg-h263-rtp-depacketization/

Hope it helps
Stefan

Hi Daniel,
I need to strem live using H264 as well and stumble against the same
latency issue.
I have been trying for a while to solve this problem as well.

It is definitely a CODEC problem that is more severe for smaller fps
rates.
I tested with "adobe live media encoder" They have H.264 with many
presets and FPS adjustable.
So I came with the same delays as with xuggle.

I did set the following options:


videoStreamCoder.setNumPicturesInGroupOfPictures(grabber.frameRate.getNumerator() );

videoStreamCoder.setProperty("b_adapt", false);
videoStreamCoder.setProperty("bf", 0);

videoStreamCoder.setProperty("threads", 0);

This helped a little but still not sufficiently.


do you have any progress with this issue?
I know that the only steady solution is to call from Xuggle the
options for x264 --intra-refresh and --tune-zerolatency
However these seem not to be exposed in Xuggle. Or am I wrong ?

Thank you for the tutorial.But I not know how to write the getNextFrame, can I use IMediaReader for decode end encoding to outContainer?