Wrong coloring when passing YUV Data to jpeg in MSDK-v5.6.0

Hello everyone, I have a frequent problem here related to transforming YUV data received by a YuvDataListener into a JPEG image. In the project I'm working on, every 1 second the drone takes this YUV data and turns it into a JPEG and saves it in the RC Pro Controller's gallery. However, some images when saved have the wrong coloring. I waited 30 seconds (30 photos) and 3 of them came with the wrong color. As you can see in the example below (the red squares are censorship) (No errors are thrown):

A correct quality image:

The code I'm using to save the image and process it is this: (I took it from your v5.6.0 sample and i'm running the code below inside the FPVWidget class)

override fun surfaceCreated(holder: SurfaceHolder) {
    LogUtils.i(logTag, "surfaceCreated", videoChannelType, videoDecoder == null)
    if (videoDecoder == null) {
        videoDecoder = VideoDecoder(
            context,
            videoChannelType,
            DecoderOutputMode.SURFACE_MODE,
            fpvSurfaceView.holder,
            fpvSurfaceView.width,
            fpvSurfaceView.height,
            true
        )
    } else if (videoDecoder?.decoderStatus == DecoderState.PAUSED) {
        videoDecoder?.onResume()
    }
    if (videoDecoderYuv == null) {
        videoDecoderYuv = VideoDecoder(
            context,
            videoChannelType,
            DecoderOutputMode.YUV_MODE,
            fpvSurfaceView.holder,
            fpvSurfaceView.width,
            fpvSurfaceView.height,
            false
        )
        initializeVideoCallback()
    } else if (videoDecoderYuv?.decoderStatus == DecoderState.PAUSED) {
        videoDecoderYuv?.onResume()
    }
}

// Get the YuvData and save then into a variable at the top of the class:
private fun initializeVideoCallback() {
    if(videoDecoderYuv != null) {
        videoDecoderYuv?.addYuvDataListener(object : YuvDataListener {
            override fun onReceive(mediaFormat: MediaFormat?, data: ByteArray?, width: Int, height: Int) {
                yuvByteArray = data
                yuvWidth = width
                yuvHeight = height
                yuvMediaFormat = mediaFormat
            }
        })
    }
}

Take these variables and use them in a method in other class:

private String saveFrame(Context context, String frameName, byte[] yuvByteArray, int yuvWidth, int yuvHeight) {

    try {
        File myDir = new File(context.getFilesDir().getPath() + PATH);
        if (!myDir.exists() || !myDir.isDirectory()) {
            myDir.mkdirs();
        }

        byte[] yuv = prepareYuvDataToJPEG420P(yuvByteArray, yuvWidth, yuvHeight);

        YuvImage yuvImage = new YuvImage(yuv, ImageFormat.NV21, yuvWidth, yuvHeight, null);

        File file = new File(myDir, frameName + ".jpeg");
        FileOutputStream outputFile = new FileOutputStream(file);

        yuvImage.compressToJpeg(
                new Rect(
                        0,
                        0,
                        yuvImage.getWidth(),
                        yuvImage.getHeight()
                ), 100, outputFile
        );

        outputFile.close();

        return file.getPath();
    } catch (Exception e) {
        log.error("Error saving image: ", e);
        throw new RuntimeException(e.getMessage());
    }
}

private byte[] prepareYuvDataToJPEG420P(byte[] yuvByteArray, int yuvWidth, int yuvHeight) {
    if (yuvByteArray == null) {
        return new byte[]{};
    }

    int length = yuvWidth * yuvHeight;
    if (yuvByteArray.length < length) {
        return new byte[]{};
    }

    byte[] u = new byte[yuvWidth * yuvHeight / 4];
    byte[] v = new byte[yuvWidth * yuvHeight / 4];

    for (int i = 0; i < u.length; i++) {
        u[i] = yuvByteArray[length + i];
        v[i] = yuvByteArray[length + u.length + i];
    }

    for (int i = 0; i < u.length; i++) {
        yuvByteArray[length + 2 * i] = v[i];
        yuvByteArray[length + 2 * i + 1] = u[i];
    }

    return yuvByteArray;
}

Xaooo

February 08, 2024 23:09

The drone used was Mavic 3E

DJI Developer Support

February 10, 2024 06:56

Our engineers will be on holiday for the Chinese New Year(2-06~ 2-17), which may cause some delays in responding to your ticket. We appreciate your patience and apologize for the inconvenience this may cause you. You can first search for related articles in our knowledge base https://djisdksupport.zendesk.com/hc/en-us. We will answer your ticket as quickly as possible when the holiday is over.

February 18, 2024 04:22

Could you confirm if the YUV data corresponding to the photos in question also exhibit the same color issues?

February 19, 2024 13:23

I corrected this by starting YuvDataListener only when i want to get the bitmaps and then, removing it. Furthermore, i wait receive 10 frames and then, i return the last one

February 20, 2024 07:53

Understood. However, the method you are using does not directly fix the issue of different colors in the photos. Have you conducted further investigation in this regard? For example, is the abnormality in the YUV data itself causing the incorrect display? Or is there an issue occurring during the conversion to JPEG? I have tested on the sample code and have not replicated a similar issue so far, also waiting for 30 seconds.

February 20, 2024 18:06

I didn't do a deep analysis, but I believe that the YUV data was coming with abnormalities. How do I check if they are abnormal? I still don't understand the YUV data structure very well.

February 21, 2024 07:16

If you have YUV data saved when there are color abnormalities, we can try to analyze it. However, you are using version 5.6, and YUV data has been optimized in version 5.8. If it is causing you significant trouble, you may consider updating to version 5.8. If you would like to conduct further analysis, I can provide you with an upload link for you to send us the YUV data.

Wrong coloring when passing YUV Data to jpeg in MSDK-v5.6.0

Comments

Didn't find what you were looking for?