Introduction

The goal of v4l4j is to provide a simple-to-use Java API allowing access to video streaming hardware under Linux. Besides capturing frames, I also wanted v4l4j to make it possible to discover available controls and use them. Both of these should require as little knowledge as possible about the underlying hardware. All the required low-level information should be provided dynamically at run time so applications could adjust accordingly. I quickly realised that the first step in writing v4l4j would involved designing and writing some form of abstraction layer first, due to the many differences involved in access video hardware.

Providing seamless access to ANY V4L devices from user space isn't an easy task.

There are several hurdles to overcome in order to create a video stream, or access video controls:

First, video capture in Linux is done through the V4L API of the Linux kernel, which comes in two (incompatible) versions 1 (now deprecated) and 2. Which version of V4L is used depends on the hardware device itself and its driver. To this date, there are still a few V4L1-only drivers out there. The problem is that the semantics and usage of the two APIs is very different. Usual tasks involved in capturing frames (setting up the video device, intialising capture buffers, and so on) are done in different ways, using different data structures and ioctl calls.

Second, there are several ways to control a video device, which again, depends on which version of V4L is used:

• through specific ioctls (VIDIOCGPICT and VIDIOCSPICT) for V4L1,
• through user and extended V4L2 controls (for V4L2),
• through private ioctls (for both versions)

Usage of V4L1 VIDIOCGPICT ioctl is fairly straight forward, but it is very limited in what it can control (brightnes, hue, color, contrast and whiteness). Normal V4L2 controls are also easy to enumerate and use. Both are well documented in the V4L1 and V4L2 specifications. However, private ioctls are hard to deal with. First, because it is not possible to discover or enumerate them at run-time. Second, because they are driver-dependent, meaning that one driver may implement a certain set of private iotcls, while another may implement a totally different one. The only way to make use of private ioctls is by:

• hardcoding in the application ALL existing private ioctls for ALL drivers, and
• have the application to try and detect the underlying driver.

The former requires checking every single existing driver to find out whether or not it defines private ioctls, while the latter can not easily be done and again depends on which version of V4L is used.

Last, different video devices support different image formats. An image format defines how bytes are organised in an image buffer returned by V4L. It specifies the layout of the image in the buffer, the number of components, the color depth ... . There are *MANY* different image formats out there and a few color spaces also. Creating a video stream implies negotiating a video format supported by hardware and suitable to the application.

Which brings me back to my first point: Providing seamless access to V4L devices from user space isn't an easy task. This is why instead of creating java code to directly interact with V4L, I have created an intermediate layer: libvideo. v4l4j relies on libvideo to "smooth out" the rough edges of dealing with V4L.

Features

libvideo is a C library of functions which facilitates interactions with the capture interface of the V4L API. libvideo masks the differences between V4L1 and V4L2 and provides a unique API regardless of which version of V4L is used underneath.

• It automatically detects the V4L version and adjusts itself consequently.
• It offers image format negotiation by allowing the application to provide a set of acceptable image formats.
• It automatically detects all controls (V4L1, user and extended V4L2 controls) and reports them as V4L2 controls
• It holds a list of drivers with private ioctls. It will automatically detects the underlying driver and reports any private iotcls also as (fake) V4L2 controls.
• It offers transparent conversion from exotic and hardware-specific image formats to well-known ones (RGB24, BGR24, YUV420 and YVU420). libvideo relies on libv4l_convert to perform these conversions. libv4l_convert is part of the libv4l project.

All in all, you can think of libvideo as an adapter for both versions of V4L, which offer a uniform API to capture frames using V4L, and provides a few simplifications of intricate details you (most of the time) dont want to deal with.

Contents

Information on libvideo is split across multiple pages:

• The libvideo page (this page) gives a short introduction to libvideo and how it came about.
• The DownloadBuildUsage page explains how to download it, compile it and use it.
• The API pages explain how to write a program with libvideo.

Links to these pages and sections within them are available in the side bar at the top of the page.