Building Linux Device Drivers – Part 1

Hey there, awesome readers! Welcome to my second post, where I’m going to chat about my month-long experience of building Linux device drivers. My goal is to make things simpler and share my experiences with you. If I can wrap my head around it, you’ll be right there with me (fingers crossed!).

To make this simpler, I’ve divided the blog into three parts, each layering on the other to craft a comprehensive tale of my escapades in the Linux Kernel:

What exactly is a device driver?
The V4L2 and vivid test device driver
What am I doing with HDMI and vivid?

Let’s get started as we unravel the mysteries of Linux device drivers, dive into the realms of V4L2 and vivid test drivers, and uncover the fascinating connection between HDMI and vivid. Buckle up for the tech ride of the month!

What exactly is a device driver?

The term “device driver” inherently reveals its purpose – a ‘device’ encompasses hardware components like Hard Disks, Monitors, Keyboards, and WiFi Cards. The term ‘drive’ in ‘device driver’ signifies its role in making these devices work seamlessly within the computer system.Therefore, a device driver is essentially the software that enables the operation of hardware devices connected to your computer such as a camera or a printer with an operating system.

Without the presence of a device driver, the operating system lacks the necessary instructions and protocols to effectively handle and utilise the associated hardware. In essence, device drivers play a pivotal role in bridging the gap between the software and hardware layers of a computer system, ensuring that diverse hardware components can function harmoniously within the operating environment. Most modern operating systems will automatically install drivers for common devices.

Let me wrap up the section with a simple example to solidify this:

When you go to a website using a web browser like Firefox, the page requested reaches your browser through a hardware component called a Wireless LAN card (or Wired LAN card). Your browser doesn’t directly talk to this hardware. Instead, it talks to the Operating System (like Windows). Then, the Operating System talks to a device driver. Finally, the device driver controls the hardware (Wireless LAN card) to manage the sending and receiving of the data you want.

The V4L2 and vivid test device driver

V4L2, or Video for Linux 2, is a set of tools in Linux for handling video devices. It deals with the complexities of devices like webcams and TV tuners, making sure they work smoothly with the computer. V4L2 provides a common set of rules, or a framework, so these devices can understand each other better.

Before the V4L2 framework, developers creating drivers for video devices, like webcams, had to handle individual setup and connections to sub-devices—smaller units with specific roles. For example, in a webcam, the camera sensor and image processor are sub-devices that work together to enable the overall functionality of the video device. This approach led to complexity and potential errors. The V4L2 framework simplifies this process by providing standardised tools and a framework, making it easier for developers to create drivers and reducing the likelihood of errors in configuring both the main video device and its internal sub-devices.

The Virtual Video Test Driver(vivid), on the other hand, creates a virtual environment that behaves like different types of video-related hardware, while implementing v4l2 specifications. Allowing developers to test and simulate interactions between hardware and software applications they’ve built without needing actual physical devices.

Consider vivid as a crucial tool for video software developers, especially when working on applications like Zoom or Skype. Before the era of vivid, developers had to rely on real cameras and physical video devices for testing their applications, which could be cumbersome and resource-intensive. However, with vivid, developers can now create a virtual environment that simulates the behaviour of actual video hardware, allowing them to test and refine their software without the need for physical devices.

Furthermore, vivid integrates with V4L2 by providing a virtual video device that adheres to the V4L2 framework’s standards and interfaces. Therefore, application developers don’t have to worry whether their apps will work on actual hardware. If the application works on vivid, it should probably work on your laptop’s Realtek webcam, hopefully ;-).

What am I doing with HDMI and vivid?

Drumroll, please! It’s time for the big reveal of the project that inspired this blog! I’m currently focused on enhancing HDMI support on the vivid driver. But what does this mean, exactly?

Let’s consider a device that can receive multiple HDMI inputs and produce multiple HDMI outputs. This isn’t your everyday user device like a TV – it’s a more specialised device, such as the Cisco Room Kit Pro, illustrated below.

The Cisco Room Kit Pro connects multiple HDMI inputs through composing images of one or more inputs to one or more outputs, enabling various devices to be linked, like laptops or cameras. This device accommodates multiple inputs and facilitates smooth switching between different outputs. This means users can effortlessly display content from various sources on the screen during video conferences or presentations.

So, in this part of my project, I want to make vivid better at imitating connections between multiple HDMI inputs and outputs across vivid instances. When you create a “vivid instance,” you are basically making a virtual version of a video device. Each vivid instance acts like a real video capture or output device. Right now, you can loop video from an HDMI X output to an HDMI X input within a single vivid instance. So video from the HDMI 1 output can only be received on the HDMI 1 input

But what I want to do is to make it so that any HDMI input in any vivid instance can connect to any HDMI output in any other instance. This means that an HDMI input in a vivid instance can now connect to any HDMI output in any vivid instance, just like you can connect any video source to any HDMI input on the Cisco Room Kit Pro. This expansion would make vivid much more flexible and useful, allowing for more dynamic and interconnected HDMI setups across multiple instances.

I hope this explains the technical details in a clear and simple way. I also hope you’ve learned a bit about drivers and something interesting about the project I’m working on. If not, I’ll try to explain it better as I keep learning and take you along with me. If you’ve read this far, you’re awesome, and you might want to explore Linux Device Drivers – here is a good book to get you started. Until the next post, Adios!!!