Is there any way to directly stream real-time video to remote controller on PSDK?

Comments

12 comments

  • Meg T

    #Additional
    I want to preprocess the video (overlay the graph of the amount of gas), so I don't mind whether it is OSDK or PSDK as long as I can get real-time video stream of 3rd party cam on the ground and also edit the real-time stream.

    0
    Comment actions Permalink
  • DJI Developer Support
    Your requirements seems to send the video stream to RC from third party camera, it is PSDK camera emu function, you shuld connects the third party device to the X-PORT or Skyport and mounted on M300. There is a sample about how to send stream(simulator of h264 file) to RC APP。 https://github.com/dji-sdk/Payload-SDK/tree/master/samples/sample_c/module_sample/camera_emu
    1
    Comment actions Permalink
  • Meg T

    Thank you so much for your advice.

    I have confirmed by debugging that I can activate "DjiTest_CameraEmuBaseStartService()" function in sample_c/platform/linux/manifold2/application/main.c, However Pilot App just display front cam video(window name is FPV) and black screen(window name is PSDK_APPALIAS).

    I cannot see the live streamed video on the Pilot app.(I guess PSDK_0005.h264 should have streamed in the demo app)
    How can I see the emulated camera's stream?

    0
    Comment actions Permalink
  • DJI Developer Support
    For this error, it is recommended to ping the drone IP to see if the IP communication works.
    0
    Comment actions Permalink
  • Meg T

    Thank you.
    I did the ping communication with X-port and ip communication seems to works well.

    And I have found another failure which is similar to this article (https://sdk-forum.dji.net/hc/zh-cn/community/posts/4411736973465), that when I use X-port as a PSDK adapter, PSDK menu on Pilot is not fully displayed. (same as 显示实时数据 error)

    PSDK menu on Pilot is fully displayed when I use OSDK expansion board as a PSDK adapter.

    Then what is the problem?

    0
    Comment actions Permalink
  • DJI Developer Support
    Please show the ping screenshot and "ifconfig"
    0
    Comment actions Permalink
  • DJI Developer Support
    Sorry, because the forum will forward the email system to the DJI sdk team, there may be some missing pictures. The picture you provided shows that the network port used is lo (localhost) You can set it as eth0 in the code, and then connect eth0 port to the RJ45 interface of the adapter board through a network cable ![](https://djisdksupport.zendesk.com/attachments/token/cXXzDGkyqTU3Y6vWSrbIenvXn/?name=image.png)
    1
    Comment actions Permalink
  • Meg T

    Thanks to your advice, I can successfully send video file("media_file/PSDK_0005.h264") to DJI pilot by camera_emu function.

    As a next step, I want to send real-time USB Cam stream(like "/dev/video0") to my DJI Pilot, but I can see only black screen in the PSDK menu in Pilot.

    Is this expected behavior due to the fact that the code only supports video files? Or is this the error due to the camera not meeting the payload criterion?

    #How to reproduce

    1. Change "tempPath" name from "%smedia_file/PSDK_0005.h264" to "%smedia_file/video0" in test_payload_cam_emu_base.c.

    2. Change the pixel format of camera by below code because my web cam is not compatible with h264 streaming.

    3. sudo ./dji_sdk_demo_linux

    ffmpeg     -y -f v4l2 -thread_queue_size 8192 -input_format yuyv422     -video_size 640x480 -framerate 30     -i /dev/video0     -c:v h264_v4l2m2m -b:v 768k     -bufsize 768k -vsync 1 -g 10     -f flv /home/ubuntu/PSDK/samples/sample_c/module_sample/camera_emu/media_file/video0

    0
    Comment actions Permalink
  • Meg T

    ##supplement

    Once I save the video stream of USB cam as a mp4 or h264 file about 30 seconds long , then I can send that file to DJI Pilot properly.

    0
    Comment actions Permalink
  • DJI Developer Support
    It is impossible for the sample to simulate real-time capture of video streams, so H264 file transmission is used. Developers need to combine camera sensors to encode H264 video streams, and then send them through the video transmission API of PSDK.
    1
    Comment actions Permalink
  • nmai

    If you look at the test_payload_cam_emu_media.c file in the PSDK sample, the SDK provides a function that accepts video "frames" and transmits it to the remote control.  If using the ffmpeg library, the corresponding class for these "frames" is AVPacket. 
    In other words, to use the PSDK+ffmpeg you will need to go from:
    Video stream => H264 packets => PSDK DjiPayloadCamera_SendVideoStream(h264 packet data + 6-byte marker) => remote control. 
    We have not figured out how to capture packets from a live video stream yet, but it looks like it can be done with ffmpeg library.  The remaining steps are more or less trivial.  Hope this help and if you figure out the capturing packets part, please give us some pointers.

    1
    Comment actions Permalink
  • nmai

    We ended up taking a different approach, but I believe you can use a parser (AVCodecParserContext) to extract packets from a live stream raw data.  See the details in the file decode_video.c from the ffmpeg examples.  A file is used in the sample, but I think the idea should be the same for raw data stream.  

    1
    Comment actions Permalink

Please sign in to leave a comment.