Chapter 6: Live Streaming

The MSDK V5 is using a cross-platform .so library. Since both the DJI Pilot 2 and MSDK V5 use this library, the performance of live streaming for both apps should be identical.

【The theory of live streaming】:

The theory of live streaming is to collect the YUV data from the SurfaceView, re-encode them into the H.264 format and broadcast them to a targeted URL according to a specific protocol. The URL server will start decoding and rendering on the web after receiving the broadcasted live stream data.

【The difference between MSDK V4 and MSDK V5】:

• Both MSDK V5 and V5 do not support live streaming in the background. The current page must have a rendering surface. You cannot minimize the app or redirect to another page.
• The MSDK V5 supports RTMP(no RTMPS), GB28181, RTSP, and Agora(WebRTC) while the MSDK V4 only supports RTMP.
• The MSDK V5's live streaming does not include audio data, but it will insert some fake audio data because some live stream platform requires both video and audio data. The MSDK V4's live streaming includes audio data.

【Flow diagram】

【The settings of each live streaming protocol】

• RTMP, need to setUrl.
• GB28181, need to setServerIP, setServerPort, setServerID, setAgentID, setPassword, setLocalPort and setChannel.
• RTSP, need to setUserName, setPassword and setPort.
• Agora, need to setChannelId, setUid, setToken and enableSafety.

【Extension】

The SEI section in DJI's H.264 video stream is empty before V3 firmware and we did add some information into the SEI section after V3 firmware. Because of many issues such as maintenance, we do not open the method of SEI decode to the public. We know that some users may want to extract the geological information and aircraft telemetry data from the SEI section. We have provided a feasible solution for you.

• If you want to connect the Mobile SDK with your private cloud server, you may use the software interface from Mobile SDK to get that information and upload it to your server.
• If you want to connect the DJI Pilot 2 with your private cloud server, you can use Cloud API to get that information and upload it to your server.

【Note】

• You must set the isForLiveStream to true in the constructor of VideoDecoder. You can find the details in Chapter 5: An introduction to video stream receive and decode.
• It is suggested to add a listener for LiveStreamStatus. Pay attention to the getPacketLoss, getPacketCacheLen and getRtt to judge the current network status.