Foreword
Video playback is often encountered in PC development, and there are basically two common solutions.
1. Use the SDK and front-end controls provided by the manufacturer for display. Common Hikvision/Dahua provide relevant SDKs and documents.
2. Turn on the camera onvif protocol and play it through the rtsp video stream. The front end can use the web method or the video control in wpf for display.
The project requirements determined the final method of turning on the camera onvif for power supply and playing it in WPF.
After doing online research for a while, Vlc.DotNet or libvlcsharp.wpf is basically recommended for front-end display.
After referring to a lot of code, whether it is official documents or codes in different blogs, it is difficult to use mvvm to decouple logic.
And Vlc.DotNet is no longer updated.
The design of Libvlcasharp.wpf is somewhat anti-human. You can refer to this article to use LibVLCSharp.WPF to play rtsp in WPF – Naylor – Blog Park (cnblogs.com).
So this part of the logic is very difficult to write, and we need to find other solutions.
I have some free time recently and researched several other open source projects. Everyone has the same idea. The camera opens the onvif protocol to push the rtsp video stream, and the video is transferred locally through ffmpeg, and then pushed to the wpf front-end control.
unosquare/ffmediaelement: FFME: The Advanced WPF MediaElement (based on FFmpeg) (github.com)
SuRGeoNix/Flyleaf: Media Player .NET Library for WinUI 3/ WPF/WinForms (based on FFmpeg/DirectX) (github.com)
There are sample codes for FFME on the Internet. I failed to build it locally. It should be a problem with my ffmpeg compiled version. You can refer to this project.
DG-Wangtao/FFMEVideoPlayer: Uses WPF MideaElement encapsulated by FFmepg to play rtsp video streams. Thanks https://github.com/unosquare/ffmediaelement
Finally, I chose Flyleaf’s solution and simply built a demo for everyone’s reference.
Flyleaf official project address SuRGeoNix/Flyleaf: Media Player .NET Library for WinUI 3/ WPF/WinForms (based on FFmpeg/DirectX) (github.com)
The MVVM framework uses CommunityToolKit.MVVM
Text
The use of Flyleaf is divided into four steps.
1. Configure the dll file address of ffmpeg in App.xaml and App.xaml.cs;
2. Configuration parameters and other information in ViewModel;
3. Configure layout and other information in View;
4. Determine the binding relationship between View and ViewModel in xaml.cs
- Configure the address of ffmpege in App.xaml.cs
1.1 The dll file of ffmpeg, I only used the file in Flyleaf’s official sample, and the version is not the latest.
1.2 Files are placed in the FFmpeg folder in the project
1.3 Build Action is configured as None
1.4 Copy to Output Directory is configured as Copy if newer
1.5 Add startup event in App.xaml
<Application x:Class="FlyleafDemo.App" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:local="clr-namespace:FlyleafDemo" StartupUri="MainWindow.xaml" Startup="Application_Startup"> <Application.Resources> \t </Application.Resources> </Application>
1.6 Configure the ffmpeg dll path in App.xaml.cs. After the project is compiled, the ffmpeg folder and dll will be copied.
using FlyleafLib;
using System;
using System.Collections.Generic;
using System.Configuration;
using System.Data;
using System.Linq;
using System.Threading.Tasks;
using System.Windows;
\t
namespace FlyleafDemo
{
/// <summary>
/// Interaction logic for App.xaml
/// </summary>
public partial class App : Application
{
private void Application_Startup(object sender, StartupEventArgs e)
{
Engine.Start(new EngineConfig()
{
FFmpegPath = System.IO.Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "FFmpeg"),
FFmpegDevices = false, // Prevents loading avdevice/avfilter dll files. Enable it only if you plan to use dshow/gdigrab etc.
\t
#if RELEASE
FFmpegLogLevel = FFmpegLogLevel.Quiet,
LogLevel = LogLevel.Quiet,
\t
#else
FFmpegLogLevel = FFmpegLogLevel.Warning,
LogLevel = LogLevel.Debug,
LogOutput = ":debug",
//LogOutput = ":console",
//LogOutput = @"C:\Flyleaf\Logs\flyleaf.log",
#endif
\t
//PluginsPath = @"C:\Flyleaf\Plugins",
\t
UIRefresh = false, // Required for Activity, BufferedDuration, Stats in combination with Config.Player.Stats = true
UIRefreshInterval = 250, // How often (in ms) to notify the UI
UICurTimePerSecond = true, // Whether to notify UI for CurTime only when it's second changed or by UIRefreshInterval
});
}
}
}
- Configuration parameters in ViewModel, simply put, there is a button, click to switch the URL of rtsp, and the front-end control plays it
using CommunityToolkit.Mvvm.ComponentModel;
using CommunityToolkit.Mvvm.Input;
using FlyleafLib.MediaPlayer;
using FlyleafLib;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Media;
\t
namespace FlyleafDemo
{
public class MainViewModel:ObservableObject
{
private Player player;
\t
public Player Player
{
get => player;
set => SetProperty(ref player, value);
}
\t
private Config config;
\t
public Config Config
{
get => config;
set => SetProperty(ref config, value);
}
\t
private string uriString;
\t
public string UriString
{
get => uriString;
set => SetProperty(ref uriString, value);
}
\t
public IRelayCommand<string> PlayCommand { get; set; }
public MainViewModel()
{
Config = new Config();
Config.Video.BackgroundColor = Colors.Transparent;
//Set the playback delay to 100ms. Maybe I misunderstood it. You can check it in the project issues for details.
// Config.Player.MaxLatency = 100 * 10000;
\t
Player = new Player(Config);
PlayCommand = new RelayCommand<string>(PlayAction);
UriString = uri1;
}
\t
private string currentUri = string.Empty;
private string uri1 = "rtsp://192.168.20.2:554/cam/realmonitor?channel=1 & amp;subtype=0 & amp;unicast=true & amp;proto=Onvif";
private string uri2 = "rtsp://192.168.20.3:554/cam/realmonitor?channel=1 & amp;subtype=0 & amp;unicast=true & amp;proto=Onvif";
private void PlayAction(string uri)
{
if (!string.IsNullOrEmpty(uri))
{
if (currentUri == uri1)
{
//Player.Commands.Stop.Execute(null);
currentUri = uri2;
Player.Commands.Open.Execute(uri2);
}
else
{
//Player.Commands.Stop.Execute(null);
currentUri = uri1;
Player.Commands.Open.Execute(uri1);
}
}
}
}
}
- Configure layout in View
<Window
x:Class="FlyleafDemo.MainWindow"
xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:fl="clr-namespace:FlyleafLib.Controls.WPF;assembly=FlyleafLib"
xmlns:local="clr-namespace:FlyleafDemo"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
Title="MainWindow"
Width="800"
Height="450"
mc:Ignorable="d">
<Grid>
<Grid.RowDefinitions>
<RowDefinition Height="5*" />
<RowDefinition Height="*" />
</Grid.RowDefinitions>
<fl:FlyleafHost
AttachedDragMove="Both"
KeyBindings="Both"
Player="{Binding Player, UpdateSourceTrigger=PropertyChanged, Mode=TwoWay}">
<Viewbox>
<TextBlock Foreground="DarkOrange" Text="Hello Flyleaf Overlay!" />
</Viewbox>
</fl:FlyleafHost>
<Button
Grid.Row="1"
Command="{Binding PlayCommand}"
CommandParameter="{Binding UriString, UpdateSourceTrigger=PropertyChanged, Mode=TwoWay}" />
</Grid>
</Window>
- Determine the binding relationship between View and ViewModel in xaml.cs
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
\t
namespace FlyleafDemo
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
this.DataContext = new MainViewModel();
}
}
}
Summary
Summarize
Front-end control binding is more convenient and reduces coupling logic in xaml.cs
I have tried playing three videos at the same time, and the effect is good, and the system resource consumption is not high.
Many parameters can be configured in Config, and some interaction logic can be executed in Player, which is relatively clear.
However, when switching video streams with a single video control, there will be a certain delay. I have tried using
Player.Commands.Stop.Execute(null);
But the effect is not great.
If you are interested, you can dig into the source code. I am just giving you some ideas here.
Demo code address https://gitee.com/maoleigepu/flyleaf-demo.git, or go to github, maoleigepu/FlyleafDemo (github.com) renderings are as follows
