Installation of a custom Linux SDK in a USB camera design

USB cameras are usually interfaced with desktop and embedded systems for video applications like surveillance and monitoring. However, industrial applications demand the imaging of fast-moving subjects. Here, SuperSpeed USB specifications enable a much higher data transfer speeds of up to 5 Gbps. Support for the SuperSpeed USB interface is available in embedded systems such as Raspberry Pi. Raspberry Pi 4 has an upgraded USB capacity by including two USB 3.0 ports, which can transfer data up to 10 times faster than USB 2.0 ports.

In embedded systems, built-in driver support is provided for interfacing USB cameras. Camera vendors do not write custom drivers for USB video class (UVC) compatible USB cameras. Instead, vendors implement video streaming hardware according to the video devices specification guidelines.

In an embedded system, the UVC driver module can be enabled in the kernel of the Linux OS. When the USB camera uses UVC driver support, it works as a plug-and-play device. A plug-and-play USB camera, See3CAM CU30, is interfaced in the UltraScale+ MPSoC platform. However, control of camera features such as precision iris and region of interest is required for industrial and scientific imaging applications. For such applications, custom Linux SDK for a camera’s interface and control is provided by the camera manufacturers.

This SDK includes the camera driver, library, and API support for camera interfacing and operations. Support for USB cameras with the custom Linux SDK in the kernel of Linux OS is emerging. So, we need to separately install the camera Linux SDK in the embedded system.

Teledyne Lumenera SuperSpeed USB cameras are built with leading sensor technology. The Linux SDK of the Lumenera camera contains lightweight libraries for both camera control and image acquisition. This work is performed a procedure for installing the Lumenera camera Linux SDK on the UltraScale+ MPSoC platform. In our hardware, the ZCU104 evaluation kit, which hosts Zynq UltraScale+ XCZU7EV MPSoC, is used for interfacing USB camera Lt545 (Figure 1). For the UltraScale+ MPSoC platform, there are several third-party supported Linux and Android OS offerings.

Figure 1 Here is a snapshot of the Lt545 camera with ZCU104 evaluation kit.

In this article, we demonstrate installing the Linux SDK on the Lumenera camera using PetaLinux and Ubuntu Root File System (RootFS). The 5.1MP USB camera with the custom Linux SDK is interfaced with ZCU104 evaluation kit.

In this approach, hardware is developed in Vivado. PetaLinux project creation and configuration were performed using the Vivado generated hardware description file (.xsa file). Two approaches were used for the RootFS of Linux kernel: PetaLinux and Ubuntu RootFS. Bootable images (image.ub, boot.src, and BOOT.BIN) created with PetaLinux project are copied to the boot partition of the Secure Digital (SD) card.

PetaLinux and Ubuntu RootFS are copied separately to the rootFS partition of the SD card. In the process of installing the custom Linux SDK, we observed that the build option is not available in the kernel of PetaLinux. So, we need to individually install camera driver, library and API support for interfacing each camera from the camera Linux SDK. Furthermore, using the Ubuntu RootFS approach, installing the camera SDK is similar to desktop systems. However, the system boot-time is significantly large compared to the PetaLinux approach.

Cross-compilation can be performed using both RootFS approaches. Using the PetaLinux approach, PetaLinux SDK is exported. Using the Ubuntu RootFS approach, sysroot is provided by the vendor Ubuntu. The developed applications are downloaded and copied in the ZCU104 evaluation kit for real-time testing of the applications.

System description

Our hardware comprises ZCU104 evaluation kit that hosts Zynq UltraScale+ XCZU7EV MPSoC, which includes Arm Cortex-A53-based application processing unit (APU) and dual-core Arm Cortex-R5-based real-time processing unit (RPU). Dedicated interfaces include two SuperSpeed USB3.0 links. We interfaced two USB3.0 cameras with the ZCU104 kit. As shown in Figure 2, camera See3CAM CU30 is UVC-compatible, whereas camera Lt545 uses custom Linux SDK for its interfacing/operation.

Figure 2 For See3CAM CU30, built-in UVC driver support is enabled in the kernel of PetaLinux.

As shown in Figure 3, See3CAM CU30 is a plug-and-play USB camera for interfacing it in the embedded system. However, camera Lt545 is not compatible with the UVC model. Moreover, there is no driver support in the kernel of PetaLinux.

Figure 3 See3CAM CU30 is a plug-and-play USB camera for interfacing with the embedded system.

For camera interfacing and operation, a custom Linux SDK is provided by the camera manufacturer, as shown in Figure 4. This SDK includes the camera API, driver, and examples for camera interfacing and operations.

Figure 4 Camera manufacturers usually provide Linux SDK.

The SDK installation process, as provided in the README.md, is as follows:

Disconnect any Teledyne Lumenera cameras
Install LuCam SDK by opening up a bash shell: tar xvfz teledyne-lumenera_lucam-sdk_<revision> . <arch>.tar.gz cd lucam-sdk_ < revision> sudo make install
Build example apps
Plug a supported Teledyne Lumenera camera into an appropriate USB port
Confirm installation: ./examples/helloCamera/helloCamera

Installation of custom Linux SDK of USB camera

Here, we describe the process of installing camera Linux SDK using PetaLinux and Ubuntu RootFS. Below are explained the installation processes using PetaLinux and Ubuntu, respectively.

PetaLinux Approach

When installing application/package/SDK in a Linux OS, Linux kernel and kernel header (Linux-headers) are used. It’s clear that there is no build option for building an SDK in PetaLinux. It means the custom Linux SDK given in II-0a cannot be installed using a sudo make install.

For installing camera SDK in PetaLinux, we need to reference the PetaLinux Tools Documentation Reference Guide provided by Xilinx. Chapter 8, titled “Customizing the Root File System” discusses the option for including prebuild libraries/applications and custom libraries/applications/modules in PetaLinux RootFS.

Figure 5 All the relevant files from camera Linux SDK have been copied to ”liblucamapi” and ”lucam” in the PetaLinux project.

Ubutnu RootFS Approach

Bootable images (image.ub, boot.src, and BOOT.BIN) were created with PetaLinux project and were copied to the boot partition of the SD card. The Ubuntu RootFS was copied to the rootFS partition of the SD card. With the Ubuntu RootFS approach, Linux kernel (5.4.0-1015-xilinx-zynqmp) with build option is shown in Figure 6.

Figure 6 Here is a view of the Linux kernel using the Ubuntu RootFS approach.

Thus, the custom Linux SDK given in II-0a can be installed using sudo make install. The snapshots for installing the camera Linux SDK on UltraScale+ MPSoC platform are shown in Figure 7 and Figure 8. With the Ubuntu RootFS approach, the process of installing the Linux SDK in the embedded system is quite similar to installing it in a desktop system. However, the limitation of this approach is that the system boot-time is significantly prolonged.

Figure 7 The snapshot shows the installation of camera SDK using the Ubuntu RootFS approach.

Figure 8 Another snapshot shows the installation of camera SDK using the Ubuntu RootFS approach.

Installation process recap

This work describes the procedure for installing the custom Linux SDK of a USB camera in an embedded system. The embedded system has built-in driver support for USB cameras that support the UVC model. However, for industrial and scientific imaging applications, a custom Linux SDK of the USB camera is provided by the camera manufacturers for camera interfacing and control. This camera SDK includes the camera driver, library, and API support.

Usually, we need to install camera the Linux SDK in the embedded systems. For the UltraScale+ MPSoC platform, there several third-party supported Linux/Android OS offerings. We demonstrated the procedure of installing a custom Linux SDK of a 5.1MP camera using the PetaLinux and Ubuntu RootFS approaches.

Nasir Ahmed is a senior researcher at the Department of Computer and Information Sciences (DCIS) in Pakistan Institute of Engineering and Applied Sciences (PIEAS) in Islamabad, Pakistan.

Related Content

Tutorial: Designing an FPGA from scratch – It’s like déjà vu all over again!
Open-source hardware for embedded security
Generate FPGA accelerators from C
Use FPGAs for stunning surveillance camera images

<!–
googletag.cmd.push(function() { googletag.display(‘div-gpt-ad-native’); });
–>

The post Installation of a custom Linux SDK in a USB camera design appeared first on EDN.

0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *