revyos SDK
Update history
Date | Version | Author | Update content |
---|---|---|---|
2023-07-21 | v1.2 | ztd |
|
2023-07-03 | v1.1 | ztd |
|
The SDK used by Sipeed is the one in this document.
The SDK is built locally using make
to configure the build environment. The following build process runs on ubuntu-22.04, so please reserve about 20 gigabytes of space.
1. Build environment configuration#
First, install the required packages and set the environment variables.
Note: When clone the following repo, please check whether it is the corresponding branch:
kernel branch is lpi4a
uboot branch is lpi4a.
opensbi branch is lpi4a
2. Build kernel#
First, please clone the used repo and create the corresponding folder (the following paths assume that the root directory is under the user directory).
Configuring the compilation toolchain
Create the installation target directory
Build the kernel
Build perf (build as needed)
Record commit-id
Install kernel, device tree to target directory
After that, you only need to copy or overwrite the contents of the rootfs to the corresponding directory. Note that the kernel image and kernel module directories must correspond to each other, or else the peripheral functions will be disabled due to the missing kernel module.
Starting from commit c56347a43e850de287a2249d3d9118910718527b
, the kernel defaults to include a 16GB memory device tree, so 8G/16G share one kernel, with only uboot being different.
3. Building uboot#
Note that at this point, you are still in the th1520_build directory and have already configured the environment variables and toolchain, refer to building the kernel for the steps.
Note that at this point, you are still in the th1520_build directory and have already configured the environment variables and toolchain, refer to building the kernel for the steps.
When flashing, pay attention to flash the uboot corresponding to the development board you are using. When flashing, please pay attention to the command you use. If the image version you are using is 0912
or above, you only need to run the following command to upgrade uboot:
Check the output files
4. Build opensbi#
Note that at this point, you are still in the th1520_build directory and have already configured the environment variables and toolchain, refer to building the kernel for the steps.
Then start executing the compile command
Checking the output files
Pack the kernel, uboot, opensbi related files from the current build into a zip archive.
To use the build files, just replace the files in the zip package to the appropriate locations.
Delete the files to be replaced in boot.ext4, then the files under rootfs/boot/ are put into boot.ext4;
Replace rootfs/lib/modules/ with the /lib/modules/ directory in rootfs.ext4;
If you have built perf, replace the files under rootfs/sbin with the files under /sbin in rootfs.ext4;
Just burn uboot directly.
5. GStreamer Player Adaptation Documentation with PTG omxil Library Support#
5.1. Overview#
PTG's OpenMAX IL library (hereinafter referred to as vpu-omxil) enables the LicheePi 4A to smoothly hard-decode 4k 60fps video, so how to use this library? This article will mainly introduce the integration and use of the Parole player on the LicheePi 4A development board, users can follow this article to understand the process of adapting to the LicheePi 4A Take the hard decoding of h264 as an example, the workflow of the hard decoding of the video is shown in the following figure
- the video stream is read in by the GStreamer and then goes through a series of pre-processing before being sent to the GStreamer's decoder
omxh264dec
. - omxh264dec calls a dynamic library, the vpu-omxil library provided by PTG, which accesses the hardware (kernel module) through the driver to perform hard decoding.
- the decoded stream is transferred to the video-sink of GStreamer and rendered by the player
5.2. GStreamer omxh264dec decoding test#
The omxh264 decoding section is separated out and has the following general structure
We build the chain shown in bottom-up order. The main purpose of this section is to get the omxh264dec decoder up and running, and does not deal with outputting screens, etc. The main purpose of this section is to get the omxh264dec decoder running.
5.2.1. 1. Driver Compilation, Installation, and Hardware Access Setup#
Hard decoding requires access to the hardware, and access to the hardware requires a driver, so you need to compile and install the driver.
1.1 Compile driver#
PTG provides the driver source:
https://gitee.com/thead-yocto/vpu-vc8000e-kernel
https://gitee.com/thead-yocto/vpu-vc8000d-kernel
https://gitee.com/thead-yocto/video_memory
1.2 Install the driver#
1.3 Setting Hardware Access Privileges#
After installing the kernel module, three device files, hantrodec vidmem vc8000, appear in the /dev directory. By default, users do not have access rights to these files. If you do not change the permissions, non-root users will get an error when opening the omxil library.
RevyOS Adaptation Records#
To get the kernel module for a specific version of RevyOS, go to [revyos/thead-kernel](https://github.com/revyos/revyos/wiki/%E6%94%AF%E6%8C%81-PTG-omxil-%E5%BA%93%E7%9A% 84-GStreamer-%E6%92%AD%E6%94%BE%E5%99%A8%E9%80%82%E9%85%8D%E6%96%87%E6%A1%A3), and download the artifacts from the GitHub CI
5.2.2. 2. Install vpu-omxil and adjust the configuration#
First, download and extract vpu-omxil into /usr/lib/omxil/: [vpu-omxil_1.2.1.tar.gz](https://drive.google.com/file/d/1pYgCVI7WltfpskltJ-RqzVUCEC21FS56/ edit?pli=1). As shown below, it is required:
- register the OpenMax component from vpu-omxil to libomxil-bellagio
- gst-omx (this package provides the omxh264dec decoder) You also need to know the name of the component you are calling when you call libomxil-bellagio.
2.1 Registering components from vpu-omxil to libomxil-bellagio#
2.2 Adjusting gstomx.conf settings#
Adjust the settings of gstomx.conf so that the decoder omxh264dec calls the correct components, see the patch for gst-omx:
[gst-omx-01-add-libomxil-config.patch](https://gist.github.com/Sakura286/26777ea8204c1819885e093806a4f7ca#file-gst-omx-01-add- libomxil-config-patch)
5.2.3. 3. Add dmabuf patch#
Please check the dmabuf patch for gst-omx provided by PTG:
[gst-omx-02-set-dec-out-port-dmabuf.patch](https://gist.github.com/Sakura286/26777ea8204c1819885e093806a4f7ca#file-gst-omx-02-set- dec-out-port-dmabuf-patch)
5.2.4. 4. GStreamer decoding preliminary test#
fakesink
swallows the previous video stream in its entirety and doesn't output the picture (thus not losing performance in the video-sink part), but in combination with fpsdisplaysink you can read the decoding speed. The normal log is as follows:
[TIP]
If there is [omxh264dec-omxh264dec0: Could not initialize supporting library.](https://gist.github.com/Sakura286/ 015fae6792e160268db7ad8a697dd2df), you can install gst-omx
, libomxil-bellagio
and debug-symbol
package related to libc6
, and start the above commands with gdb
for debugging. When debugging, break DWLInit
and then break open
, depending on which part of open
is wrong.
RevyOS adaptation log#
During the adaptation of RevyOS, the following three reasons were found for the failure of initializing dynamic libraries:
- the toolchain used to compile vpu-omxil is not compatible with the current system
- vpu-omxil was not registered with
omxregister-bellagio
. - not adjusting permissions on devices like
hantrodec
,vc8000
,vidmem
in/dev
directory.
5.3. B. Selecting a suitable GStreamer video-sink#
The video-sink
is the last step of the video stream in the whole GStreamer pipeline, and its role is generally to output the video stream to the screen.
The fakesink
in the previous section is just a special video-sink
to test that the decoder is working correctly, optional video-sink -specific-elements.html?gi-language=c) are very numerous, common ones are autovideosink
, ximagesink
, xvimagesink
, fbdevsink
, waylandsink
, glimagesink
, gtkglsink
and so on. gtkglsink`, etc. They are each in different plugin packages and need to be installed as appropriate:
video-sink | Package name | |||
---|---|---|---|---|
waylandsink | gtkglsink | waylandsink | gtkglsink | |
waylandsink | gstreamer1.0-plugins-bad | |||
fbdevsink | gstreamer1.0-plugins-bad | |||
autovideosink | gstreamer1.0-plugins-good | gtkglsink | fbdevsink | gstreamer1.0-plugins-bad |
gtkglsink | gstreamer1.0-plugins-good | gstreamer1.0-plugins-good | gstreamer1.0-plugins-good | |
ximagesink | xvimagesink | gstreamer1.0-plugins-base | |||
glimagesink | gstreamer1.0-plugins-base | gstreamer1.0-gl |
[TIP] Use gst-inspect-1.0 <video-sink-name>
to see what options are available for the corresponding video-sink.
[TIP] Add --gst-debug-level=<lv>
to get more [output logs] (https://gstreamer.freedesktop.org/documentation/tutorials/basic/debugging-tools. html#the-debug-log), where <lv>
stands for 1 to 6, and the verbosity level is from low to high, it is recommended to be at level 4 and below, otherwise the log will be very long.
Please try different video-sinks, and try different plugin parameters, and different environment variables, until you find one that can smoothly harden H264.
5.3.1. RevyOS adaptation log#
**waylandsink**
: Since RevyOS now (20230720) uses the Xfce desktop, it is not possible to support Wayland, sowaylandsink
does not work in principle.**fbdevsink**
and**ximagesink**
: not working**xvimagesink**
: via pipeline graphs and logs, we can determine that playbin or autovideosink will automatically call xvimagesink, and after analyzing with perf, we can find that using xvimagesink will inevitably perform a large number of memcpy operations, which will seriously degrade the decoding performance; this problem still exists after obtaining PTG's dmabuf patch. This problem still exists after obtaining PTG's dmabuf patch, so it is not possible to use**gtkglsink**
: GTK3 does not support EGL on X11, and RevyOS is currently based on x11 and only supports EGL, so it cannot be used.
The only thing left is glimagesink
, according to Running and debugging GStreamer Applications, and looking at other examples that use glimagesink, we can guess that we need to explicitly specify the environment variables GST_GL_API
and GST_GL_PLATFORM
.
Since RevyOS uses a combination of gles2+egl, it was successfully hardened using the following command.
However, GStreamer cannot be passed parameters via environment variables when called by the player, so additional meson compilation parameters should be passed when building gst-plugins-base:
5.4. C. Player support#
After the pipeline of GStreamer is OK, we need to make the player support. Different players use different video-sinks, and also have different levels of dependency on gstreamer.
When adapting a player, the most important thing to do is to either (i) adapt the player to a validated video-sink, or (ii) make the gstreamer pipeline support the video-sink specified by the player, which is what we have done in this RevyOS adaptation.
5.4.1. RevyOS adaptation log#
Simple troubleshooting based on https://gstreamer.freedesktop.org/apps/
Available | Updated | App Name | Remarks |
---|---|---|---|
❌ | Gnash | Flash Player | |
❌ | GEntrans | Debian Uncovered | |
❓ | 20230226 | Kaffeine | ❌ Requires a lot of KDE-related components |
✔️ exists in riscv64 repository | |||
❌ On Debian amd64 Gnome, the playback window is separate from the control window, and VLC is called by default for playback | |||
❌ | Lcdgrilo | Debian not included | |
✔️ | 20230218 | Parole | ✔️ For XFCE |
❓ No Wayland support, only x11 support | |||
✔️ Debian amd64 Gnome Verified by | |||
✔️ exists in riscv64 repository | |||
❌ | Songbird | Debian not included | |
❌ | Snappy | Debian not included | |
❌ | Totem | requires GTK3, which does not support EGL on X11 |
The first player of choice was Totem, but it turns out that Totem can't specify video-sinks other than gtkglsink, and RevyOS doesn't support it.
I then chose Parole, which is written by GObject, and looked for the method parole_gst_constructed when constructing the parole_gst object, and set the video-sink to the glimagesink as verified in the previous section, and the rough adaptation was completed.
5.5. Summary: Adapting to RevyOS#
- Compile the driver module into the kernel, set bootloading, and set device permissions.
- Package the omxil binary library provided by PTG into th1520-vpu.
- modified the dependencies of th1520-vpu to depend on packages like gst-omx, libomxil-bellagio, etc. 2. set some postinstalls.
- set up some postinstall operations, such as registering components with omxregister-bellagio, etc.
- Modify gst-omx
- added support for vpu-omxil component in config
- apply dmabuf patch
- add support for h265 vp9. 4.
- modify gst-base compile-time gl support to gles2+egl. 5. modify parole to support gl.
- modify parole to support glimagesink
5.6. Resources used in this article#
Patch collection:
https://gist.github.com/Sakura286/26777ea8204c1819885e093806a4f7ca
PTG omxil library:
https://drive.google.com/file/d/1pYgCVI7WltfpskltJ-RqzVUCEC21FS56
- Build environment configuration
- Build kernel
- Building uboot
- Build opensbi
- GStreamer Player Adaptation Documentation with PTG omxil Library Support
Gitalking ...