Marco Ronchetti: May 2008

QuickTime for Java is a set of cross-platform APIs which allows Java developers to build multimedia, including streaming audio and video, into applications and applets.
That's quite nice, and to learn using it there is a book written by Chris Adamson and published by O'Reilly, available also in electronic form though O'reilly commons. The book goes through a series of examples. A basic example in chapter 6 shows how to record a video on disk. There is a problem though: while the machine records, the video stream is NOT shown on the PC screen. Obviously one would like instead to get a preview of what is being recorded while recording (so that e.g. you know how to move the camera).
It looks like a very basic requirement, so I started hunting for solutions on the net. Several hours of googling with various keywords produced more or less an empty set...

The best I could find was some sort of hint on an O'Reilly's Mac Dev Center - but still it was a bit vague - I wanted a fully developed example. At the bottom of the page I noticed an unanswered question by Amit Zohar: "So how do I capture video and audio in Java and save it into a movie file while allowing for a preview as well?" - Yes, this is what I also wanted to know.
The question was more that two years old... I decided to write to Amit to see if in the meantime he had been able to solve the problem - and yes he did! He was so kind to send me his OpenGL based code. THANKS AMIT!

Unfortunately over the last two years OpenGL has undergone some radical transformation - repackaging the classes, changing some methods' signatures etc. - so I had to update the code a bit - but it wasn't too much work. So in case someone has the same problem, here I publish here the solution. To run the code (on a Mac) you need to make sure that:

you installed QuickTime - this will also install the qtjava library as QTjava.zip in /System/Library/Java/Extensions;
you download the current release build of the Java OpenGL library - you must unzip the downloaded file;
your compile-time libraries must include QTJava.zip, the two jars of jogl: jogl.jar and gluegen-rt.jar
you put the directory containing the jnilib files that were downloaded with jogl in the runtime library path (e.g. by specifying the switch -Djava.library.path=/path/of/your/jnilib/files in your java command)

I think you need QuickTime Pro to be able to record - QuickTimeViewer is not enough - but I'm not 100% sure.

In principle it should work also on Windows - but I did not check.

The program "MiniRecorder" will first show you a window where you can play with various params (you can leave them as they are or change some of the options - e.g. change the default compressor to MPEG-4 and adapt its video quality to the level you like) - when you click ok you'll have an empty window with some buttons.
Video recording will begin when you click on "start" - you'll have a preview of what is being recorded. Click on "stop" to interrupt capturing, then "preview" to review the captured video, and "accept" or "discard" to keep/delete the file containing the saved video. Closing the window to quit.

The video is saved in a file named as you specify in the code. In the code you can also choose the directory where it will be located.

The code is composed by two classes: QTSessionFactory for initialization (adapted from Adamson's book) and MiniRecorded (essentially the code that Amit sent me with some modifications).

Here is the code:

//------ Class QTSessionFactory
package QT;
import quicktime.*;
public class QTSessionFactory {
private Thread shutdownHook;
private static QTSessionFactory instance;
private QTSessionFactory( ) throws QTException {
super( );
// init
QTSession.open( );
// create shutdown handler
shutdownHook = new Thread( ) {
public void run( ) {
QTSession.close( );
}
};
Runtime.getRuntime( ).addShutdownHook(shutdownHook);
}
private static QTSessionFactory getInstance( ) throws QTException {
if (instance == null)
instance = new QTSessionFactory( );
return instance;
}

public static void setupQTSsession( ) throws QTException {
// gets instance. if a new one needs to be created,
// it calls QTSession.open( ) and creates a shutdown hook
// to call QTSession.close( )
getInstance( );
}
}

//---- Class MiniRecorder

package QT;
import javax.swing.JComponent;
import javax.swing.JPanel;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JLabel;
/* ----------------- */
import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Component;
import java.awt.Dimension;
import java.awt.Toolkit;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
/* ----------------- */
import java.nio.IntBuffer;
/* ----------------- */
import com.sun.opengl.util.Animator;
/* ----------------- */
import javax.media.opengl.GL;
import javax.media.opengl.GLCanvas;
import javax.media.opengl.GLCapabilities;
import javax.media.opengl.GLEventListener;
import javax.media.opengl.GLAutoDrawable;
/* ----------------- */
import quicktime.QTException;
import quicktime.QTNullPointerException;
import quicktime.QTSession;
import quicktime.app.view.MoviePlayer;
import quicktime.app.view.QTFactory;
import quicktime.app.view.QTJComponent;
import quicktime.io.OpenMovieFile;
import quicktime.io.QTFile;
import quicktime.qd.QDGraphics;
import quicktime.qd.QDRect;
import quicktime.std.StdQTConstants;
import quicktime.std.StdQTException;
import quicktime.std.movies.Movie;
import quicktime.std.sg.SGSoundChannel;
import quicktime.std.sg.SGVideoChannel;
import quicktime.std.sg.SequenceGrabber;
import quicktime.util.QTBuild;

public class MiniRecorder implements StdQTConstants {
// The directory where files are saved
String activeDirectory = "/Users/ronchet/tmp/";
String fileName="movie";
// quicktime
SequenceGrabber sg;
QDGraphics gWorld;
QTFile qtFile;
Movie movie;
MoviePlayer moviePlayer;
JComponent qtc;
GLCanvas canvas;
int taskingDelay = 20;
int maxFrameRate = 30; // increasing may degrade preview speed
int compressorType = StdQTConstants.kComponentVideoCodecType;
int IMAGEWIDTH=640;
int IMAGEHEIGHT=480;
// camera flags
boolean cameraReady = false;
boolean isRecording = false;
boolean isPreviewing = true;
// image buffers
//MR int pixelData, newPixelData;
IntBuffer pixelData, newPixelData;
int WIDTH, HEIGHT;
// stats
int paintCount = 0;
long startMilli, endMilli;
// ui
JFrame frame;
Component imagePanel;
JPanel centerPanel, emptyPanel,buttonsPanel;
JButton startButton, stopButton, previewButton, acceptButton, discardButton;
final String START_RECORDING = "Start";
final String STOP_RECORDING = "Stop";
final String PREVIEW_RECORDING = "Preview Recorded Video";
final String ACCEPT_RECORDING = "Accept Recorded Video";
final String DISCARD_RECORDING = "Discard Recorded Video";
final String TITLE = "miniRecorder";
final Color BACKGROUND = Color.WHITE;
/**
* constructor.
*/
public MiniRecorder() {
try {
QTSessionFactory.setupQTSsession();
getQTinfo();
initSequenceGrabber();
} catch (Exception ex) {
log("Unable to initialize camera");
QTSession.close();
}
initUI();
}

private void getQTinfo() {
log("java.library.path: " + System.getProperty("java.library.path"));
log ("VERSIONS:");
log("OpenGL : " + javax.media.opengl.glu.GLU.versionString);
log("QT : " + QTSession.getMajorVersion( ) + "." + QTSession.getMinorVersion( ));
log("QTJ : " +QTBuild.getVersion( )+"." +QTBuild.getSubVersion( ));
}

private void initSequenceGrabber() throws Exception {
sg = new SequenceGrabber();
SGVideoChannel vc = new SGVideoChannel(sg);
// init pixelData
QDRect cameraImageSize = new QDRect(IMAGEWIDTH ,IMAGEHEIGHT);
gWorld = new QDGraphics(cameraImageSize);
WIDTH = gWorld.getPixMap().getPixelData().getRowBytes() / 4;
HEIGHT = cameraImageSize.getHeight();
pixelData=IntBuffer.allocate(WIDTH * HEIGHT);
newPixelData=IntBuffer.allocate(WIDTH * HEIGHT);

sg.setGWorld(gWorld, null);

vc.setBounds(cameraImageSize);
vc.setUsage(quicktime.std.StdQTConstants.seqGrabRecord
| quicktime.std.StdQTConstants.seqGrabPlayDuringRecord);
vc.setFrameRate(maxFrameRate);
vc.setCompressorType(compressorType);
vc.settingsDialog( );
SGSoundChannel sc = new SGSoundChannel (sg);
sc.setUsage(StdQTConstants.seqGrabRecord);

// init bufferedImage
int intsPerRow = gWorld.getPixMap().getPixelData().getRowBytes() / 4;
pixelData = IntBuffer.allocate(intsPerRow * cameraImageSize.getHeight());

cameraReady = true;
}

private void initUI() {

frame = new JFrame(TITLE);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
frame.setBackground(BACKGROUND);

// buttons panel
buttonsPanel = new JPanel();
buttonsPanel.setBackground(BACKGROUND);
startButton = new JButton(START_RECORDING);
buttonsPanel.add(startButton);
startButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent event) {
try {
startRecording();
} catch (Exception e) {
e.printStackTrace();
}
}
});

stopButton = new JButton(STOP_RECORDING);
stopButton.setEnabled(false);
buttonsPanel.add(stopButton);
stopButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent event) {
stopRecording();
}
});

previewButton = new JButton(PREVIEW_RECORDING);
previewButton.setEnabled(false);
buttonsPanel.add(previewButton);
previewButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent event) {
previewRecording();
}
});

acceptButton = new JButton(ACCEPT_RECORDING);
acceptButton.setEnabled(false);
buttonsPanel.add(acceptButton);
acceptButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent event) {
acceptRecording();
}
});

discardButton = new JButton(DISCARD_RECORDING);
discardButton.setEnabled(false);
buttonsPanel.add(discardButton);
discardButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent event) {
discardRecording();
}
});

// bottom panel - for buttons
JLabel space = new JLabel(" ");
buttonsPanel.add(space);
frame.add(BorderLayout.SOUTH, buttonsPanel);

// image panel
centerPanel = new JPanel();
centerPanel.setBackground(BACKGROUND);

emptyPanel = new JPanel();
emptyPanel.setPreferredSize(new Dimension(IMAGEWIDTH, IMAGEHEIGHT));
emptyPanel.setBackground(Color.ORANGE);

if (cameraReady) {
imagePanel = buildOpenGLCameraView();
centerPanel.add(imagePanel);
} else {
centerPanel.add(emptyPanel);
}

frame.add(BorderLayout.CENTER, centerPanel);
Toolkit toolkit = java.awt.Toolkit.getDefaultToolkit ();
Dimension screensize = toolkit.getScreenSize ();
frame.setBounds(0, 0, screensize.width, screensize.height-250);
frame.setVisible(true);

}

private void startRecording() {
log("start recording");
isRecording = true;
startButton.setEnabled(false);
stopButton.setEnabled(true);
buttonsPanel.validate();
startMilli = System.currentTimeMillis();

try {
prepareAndStartRecord();
} catch (QTException e) {
log("Unable to start recording");
} catch (QTNullPointerException e) {
log("Unable to start recording");
}

}

public void stopRecording() {
log ("stop recording");
try {
endMilli = System.currentTimeMillis();
sg.stop();
log("recording stopped");
double seconds = (endMilli - startMilli) / 1000;
double previewFps = paintCount / seconds;
log("preview stats: seconds=" + seconds + " fps=" + previewFps);

} catch (StdQTException e) {
log("Unable to stop recording");
}
isRecording = false;
stopButton.setEnabled(false);
previewButton.setEnabled(true);
frame.validate();
}

public void previewRecording() {
log("preview recording");
previewButton.setEnabled(false);
acceptButton.setEnabled(true);
discardButton.setEnabled(true);

// replace previewPanel with movie player
qtc = getQuicktimeMovieComponent(qtFile);
qtc.setPreferredSize(new Dimension(IMAGEWIDTH ,IMAGEHEIGHT));
setCenterComponent(qtc);

// Start playing the movie
try {
movie.start();
log("movie playing");
} catch (Exception e) {
e.printStackTrace();
}
}

public void acceptRecording() {
log("accept recording " + qtFile.getName());
acceptButton.setEnabled(false);
discardButton.setEnabled(false);
startButton.setEnabled(true);
setCenterComponent(imagePanel);
try {
movie.stop();
log("movie stopped");
} catch (StdQTException e) {
e.printStackTrace();
}
}

public void discardRecording() {
log("discard recording " + qtFile.getName());
acceptButton.setEnabled(false);
discardButton.setEnabled(false);
startButton.setEnabled(true);
setCenterComponent(imagePanel);
try {
movie.stop();
log("movie stopped");
} catch (StdQTException e) {
e.printStackTrace();
}
discardQTFile();
}

private void setCenterComponent(Component component) {
centerPanel.removeAll();
centerPanel.add("Center", component);
frame.validate();
}

/**
* Initializes the SequenceGrabber. Gets it's source video bounds, creates a
* gWorld with that size. Configures the video channel for grabbing,
* previewing and playing during recording.
*/

private void prepareAndStartRecord() throws QTException {
QTFile movieFile = getQTFile();
sg.setDataOutput(movieFile,
quicktime.std.StdQTConstants.seqGrabToDisk);
sg.prepare(false, true);
sg.startRecord();

// setting up a thread, to idle the sequence grabber
Runnable idleCamera = new Runnable() {

public void run() {
try {
while (sg != null && isRecording) {
Thread.sleep(taskingDelay);
synchronized (sg) {
sg.idleMore();
sg.update(null);
}
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
};
(new Thread(idleCamera)).start();
}

/**
* This creates a Panel, which displays the camera image using OpenGL
*/
public Component buildOpenGLCameraView() {
GLEventListener glEventListener = new GLEventListener() {

// Called by the drawable immediately after the OpenGL context is initialized.
public void init(GLAutoDrawable drawable) {
log("init OpenGL");
GL gl = drawable.getGL();
gl.glClearColor(1.0f, 1.0f, 1.0f, 0.0f);
gl.glShadeModel(GL.GL_FLAT);
gl.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT);
}

// Called by the drawable to initiate OpenGL rendering by the client.
public void display(GLAutoDrawable drawable) {
if (!isRecording) {
return;
}
GL gl = drawable.getGL();
gl.glClear(GL.GL_COLOR_BUFFER_BIT | GL.GL_DEPTH_BUFFER_BIT);
gl.glPixelStorei(GL.GL_UNPACK_ALIGNMENT, 1);
gWorld.getPixMap().getPixelData().copyToArray(0, pixelData.array(), 0,
WIDTH * HEIGHT);
flipVertically(pixelData);
gl.glDrawPixels(WIDTH, HEIGHT, gl.GL_BGRA,
gl.GL_UNSIGNED_INT_8_8_8_8_REV, newPixelData);
paintCount++;
}

public void flipVertically( IntBuffer pixelData ) {
for ( int row=0; row<HEIGHT; row++ ) {
System.arraycopy(pixelData.array(), row*WIDTH, newPixelData.array(), (HEIGHT-row-1)*WIDTH, WIDTH) ;
}
}
// Called by the drawable during the first repaint after the
// component has been resized.
public void reshape(GLAutoDrawable drawable, int i, int x, int width,
int height) {

GL gl = drawable.getGL();
// MR GLU glu = drawable.getGLU();
gl.glViewport(0, 0, WIDTH, HEIGHT);
gl.glMatrixMode(GL.GL_PROJECTION);
gl.glLoadIdentity();
// MR glu.gluOrtho2D(0.0, (double) WIDTH, 0.0, (double) HEIGHT);
gl.glMatrixMode(GL.GL_MODELVIEW);
gl.glLoadIdentity();

}

// Called by the drawable when the display mode or the display device
// associated with the GLDrawable has changed.
public void displayChanged(GLAutoDrawable drawable,
boolean modeChanged, boolean deviceChanged) {
}

};

GLCapabilities caps = new GLCapabilities();
canvas=new GLCanvas(caps);
canvas.addGLEventListener(glEventListener);
canvas.setBounds(0, 0, IMAGEWIDTH ,IMAGEHEIGHT);
Animator animator = new Animator(canvas);
animator.start();
return canvas;
}

public QTFile getQTFile() {
String path = activeDirectory + fileName;
int count = 0;
qtFile = new QTFile(path + count);
while (qtFile.exists()) {
count++;
qtFile = new QTFile(path + count);
}
log("getQTFile: " + path + count);
return qtFile;
}

public void discardQTFile() {
log("discardQTFile: " + qtFile.getName());
qtFile.deleteOnExit();
}

public void log(String text) {
System.out.println(text);
}

/**
* Gets a Movie component for the specified file
*/
protected JComponent getQuicktimeMovieComponent(QTFile qtFile) {
QTJComponent qtcmp = null;

try {
// Create the movie
movie = Movie.fromFile(OpenMovieFile.asRead(qtFile));
movie.setBounds(new QDRect(IMAGEWIDTH ,IMAGEHEIGHT));
moviePlayer = new MoviePlayer(movie);

// Create the QuickTime Movie Component
qtcmp = QTFactory.makeQTJComponent(moviePlayer);
return qtcmp.asJComponent();
} catch (QTException err) {
err.printStackTrace();
return null;
}
}

public static void main(String args[]) {
new MiniRecorder();
}
}

Marco Ronchetti

Wednesday, May 28, 2008

Converting ppt to pdf (and many others) from Java

Tuesday, May 27, 2008

Office on Mac - without M$

Thursday, May 22, 2008

Bugged setting Dialog in quicktime java!

Friday, May 16, 2008

Building with qtjava a video recorder that allows previewing while recording

About Me

Blog Archive

Labels