基于GPU和UNITY的嵌入式图像实时传输方法

智能化无人矿山对作业现场环境的可视化要求较高,现有的可视化方法仍存在诸多问题:数据采集方式单一,存在监控盲区;数据传输线缆布设困难且易被损坏,传输延时较高;表现形式不够全面立体,并且不能用于VR/AR、SLAM、机器人定位避障等应用场景。为了满足智能矿山建设的可视化需求,本文结合当前传感技术、矿用机器人以及5G技术的发展,探讨了从数据采集、服务器部署到接收显示的详细步骤。针对全景及深度影像这类新型三维数据,提出一种基于GPU和UNITY的嵌入式视频实时传输方法,包括实时编码、异步传输、轻量级的嵌入式流媒体系统、利用UNITY实时处理以及元数据的同步传输。借助UNITY平台,将三维可视化任务从CPU转移至GPU,仿真实验表明,最高渲染帧率为60 fps时,GPU占用率在35%以下。最后,以全景和深度传感器为例进行了测试,对数据编码、位移贴图、纹理纠正进行有效性验证,并从延迟、帧率、CPU占用率3个方面评估性能。结果表明,所提关键技术均可有效提高运行效率、减少资源占用,相比FFplay延时更低。全景影像的可视化代替了视角固定的传统监控,深度数据为智能矿山巡检机器人定位及避障提供实时数据源,传输方法整体向下兼容。不仅解决传统方法视角单一、布线困难的问题,而且考虑到了智能矿山建设过程中的新需求。

Embedded Image Real-time Transmission Method Based on GPU and UNITY

Intelligent unmanned mine has higher requirements for the visualization of the on-site environment,the existing visualization methods still have many problems:the data collection method is single,blind area of monitoring,the data transmission cable is difficult to lay and easy to be damaged,high transmission delay,lack of comprehensive three-dimensional representation,and cannot be used in application scenarios such as VR/AR,SLAM,robot positioning and obstacle avoidance.In order to meet the visualization requirements of intelligent mine construction,in combination with the current development of sensing technology,mining robots and 5G technology,the detail of data collection,server deployment,reception and display are discussed.For new 3D data such as panoramic and depth images,a real-time transmission method of embedded video based on GPU and UNITY is proposed,including real-time encoding,asynchronous transmission,lightweight embedded streaming media system,real-time processing using UNITY,and synchronous transmission of metadata.The 3D visualization task is transferred from the CPU to the GPU by the UNITY platform,simulation experiments show that when the highest rendering frame rate is 60 fps,the GPU occupancy rate is below 35%.Finally,it takes the panoramic and depth sensors as examples to test the validity of data encoding,displacement mapping and texture correction,and evaluates the performance from the three aspects of delay,frame rate and CPU utilization.The results show that the proposed key technologies can effectively improve operating efficiency,reduce resource occupation,and have lower latency than FFplay.The visualization of panoramic image replaces the traditional monitoring with fixed perspective,and the depth data provides real-time data source for the positioning and obstacle avoidance of patrol robot,and the transmission method is backward compatible.It not only solves the problem of single perspective of traditional method and difficult wiring,but also takes into account the new requirements in the process of intelligent mine construction.