ImageAnalysis给的是YUV420_888的数据,但是我需要的是NV21的数据

2023-04-15by@richyeohx-ax4MO&@ChatGPT.Model: gpt-3.5
richyeohx-ax4MO

ImageAnalysis给的是YUV420_888的数据,但是我需要的是NV21的数据

Avatar
Avatar

您可以使用Android的YUVImage类将YUV420_888数据转换为NV21格式。以下是一个示例代码:

// 将YUV420_888数据转换为NV21格式
byte[] nv21Data = new byte[width * height * 3 / 2];
YuvImage yuvImage = new YuvImage(imageData, ImageFormat.YUV_420_888, width, height, null);
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
yuvImage.compressToJpeg(new Rect(0, 0, width, height), 100, outputStream);
byte[] jpegData = outputStream.toByteArray();
ByteBuffer buffer = ByteBuffer.wrap(jpegData);
Bitmap bitmap = BitmapFactory.decodeByteArray(jpegData, 0, jpegData.length);
buffer.rewind();
bitmap.copyPixelsToBuffer(buffer);
buffer.rewind();
buffer.get(nv21Data, 0, width * height);
int offset = width * height;
for (int i = 0; i < width * height / 4; i++) {
    nv21Data[offset + i * 2] = buffer.get(width * height + i * 2 + 1);
    nv21Data[offset + i * 2 + 1] = buffer.get(width * height + i * 2);
}

在上面的代码中,imageData是YUV420_888数据,widthheight是图像的宽度和高度。转换后的NV21数据存储在nv21Data数组中。

richyeohx-ax4MO

ImageAnalysis给的是YUV420_888的数据,但是我需要的是NV21的数据

Avatar
Avatar

您可以使用以下代码将YUV420_888数据转换为NV21格式:

fun yuv420ToNv21(image: Image): ByteArray {
    val yBuffer = image.planes[0].buffer // Y plane
    val uBuffer = image.planes[1].buffer // U plane
    val vBuffer = image.planes[2].buffer // V plane

    val ySize = yBuffer.remaining()
    val uSize = uBuffer.remaining()
    val vSize = vBuffer.remaining()

    val nv21 = ByteArray(ySize + uSize + vSize)

    // Copy Y plane
    yBuffer.get(nv21, 0, ySize)

    // Copy VU plane (interleaved)
    val vuSize = vSize / 2
    var nv21Index = ySize
    var uIndex = 0
    var vIndex = 0
    while (nv21Index < nv21.size - 1) {
        nv21[nv21Index++] = vBuffer.get(vIndex++)
        nv21[nv21Index++] = uBuffer.get(uIndex++)
    }

    return nv21
}

在上面的代码中,imageImageAnalysis回调中得到的Image对象。转换后的NV21数据存储在返回的ByteArray中。

richyeohx-ax4MO

考虑一下行和列的步长问题

Avatar
Avatar

如果您需要在转换时考虑行和列的步长,可以使用以下代码:

fun yuv420ToNv21(image: Image, rowStride: Int, pixelStride: Int): ByteArray {
    val yBuffer = image.planes[0].buffer // Y plane
    val uBuffer = image.planes[1].buffer // U plane
    val vBuffer = image.planes[2].buffer // V plane

    val ySize = yBuffer.remaining()
    val uSize = uBuffer.remaining()
    val vSize = vBuffer.remaining()

    val nv21 = ByteArray(ySize + uSize + vSize)

    // Copy Y plane
    var yIndex = 0
    for (row in 0 until image.height) {
        var pixelIndex = row * rowStride
        for (col in 0 until image.width) {
            nv21[yIndex++] = yBuffer.get(pixelIndex)
            pixelIndex += pixelStride
        }
    }

    // Copy VU plane (interleaved)
    var nv21Index = ySize
    var uIndex = 0
    var vIndex = 0
    for (row in 0 until image.height / 2) {
        var pixelIndex = row * rowStride / 2
        for (col in 0 until image.width / 2) {
            nv21[nv21Index++] = vBuffer.get(pixelIndex)
            nv21[nv21Index++] = uBuffer.get(pixelIndex)
            pixelIndex += pixelStride
        }
    }

    return nv21
}

在上面的代码中,rowStride是每行的步长,pixelStride是每个像素的步长。这些参数可以通过Image.Plane对象的getRowStride()getPixelStride()方法获得。转换后的NV21数据存储在返回的ByteArray中。

richyeohx-ax4MO

优化一下循环

Avatar
Avatar

您可以使用以下代码优化循环:

fun yuv420ToNv21(image: Image, rowStride: Int, pixelStride: Int): ByteArray {
    val yBuffer = image.planes[0].buffer // Y plane
    val uBuffer = image.planes[1].buffer // U plane
    val vBuffer = image.planes[2].buffer // V plane

    val ySize = yBuffer.remaining()
    val uSize = uBuffer.remaining()
    val vSize = vBuffer.remaining()

    val nv21 = ByteArray(ySize + uSize + vSize)

    // Copy Y plane
    var yIndex = 0
    var pixelIndex = 0
    for (row in 0 until image.height) {
        for (col in 0 until image.width) {
            nv21[yIndex++] = yBuffer.get(pixelIndex)
            pixelIndex += pixelStride
        }
        pixelIndex += rowStride - image.width * pixelStride
    }

    // Copy VU plane (interleaved)
    var nv21Index = ySize
    var uIndex = 0
    var vIndex = 0
    for (row in 0 until image.height / 2) {
        for (col in 0 until image.width / 2) {
            nv21[nv21Index++] = vBuffer.get(vIndex)
            nv21[nv21Index++] = uBuffer.get(uIndex)
            uIndex += pixelStride * 2
            vIndex += pixelStride * 2
        }
        uIndex += (rowStride / 2) - image.width * (pixelStride / 2)
        vIndex += (rowStride / 2) - image.width * (pixelStride / 2)
    }

    return nv21
}

在上面的代码中,我们使用了单个循环来遍历Y plane中的所有像素,而不是使用两个嵌套循环。我们还处理了VU plane中的每个像素,而不是使用两个嵌套循环。这样可以减少循环次数,从而提高性能。

richyeohx-ax4MO

我给一段代码,你能不能帮我改进一下呢。

Avatar
Avatar

当然可以,请提供您的代码,我会尽力为您改进。

richyeohx-ax4MO

public static byte[] convertYUV420888ToNV21(Image image) {
byte[] nv21;
ByteBuffer yBuffer = image.getPlanes()[0].getBuffer();
ByteBuffer uBuffer = image.getPlanes()[1].getBuffer();
ByteBuffer vBuffer = image.getPlanes()[2].getBuffer();
int yRowStride = image.getPlanes()[0].getRowStride();
int uRowStride = image.getPlanes()[1].getRowStride();
int vRowStride = image.getPlanes()[2].getRowStride();
int yPixelStride = image.getPlanes()[0].getPixelStride();
int vPixelStride = image.getPlanes()[2].getPixelStride();

int ySize = yBuffer.remaining();
int uSize = uBuffer.remaining();
int vSize = vBuffer.remaining();

nv21 = new byte[ySize + uSize + vSize];

int nv21Index = 0;
int yIndex = 0;
int uIndex = 0;
int vIndex = 0;

int w = image.getWidth();
int h = image.getHeight();
int halfH = h / 2;
int halfW = w / 2;
int uvRowStride = vRowStride;
int uvPixelStride = vPixelStride;

if (uRowStride != vRowStride || uPixelStride != vPixelStride) {
    // if the resolution is odd, the last pixel column of the U/V plane is dropped
    uvRowStride = (w % 2 == 0 ? uRowStride : uRowStride - 1);
    uvPixelStride = (w % 2 == 0 ? uPixelStride : vPixelStride * 2);
}

for (int i = 0; i < h; i++) {
    int yOffset = i * yRowStride;
    if (i % 2 == 0) {
        int uvOffset = (i >> 1) * uvRowStride;
        for (int j = 0; j < w; j += 2) {
            int uvIndex = uvOffset + j * uvPixelStride;
            nv21[nv21Index++] = yBuffer.get(yIndex + yOffset + j * yPixelStride);
            nv21[nv21Index++] = vBuffer.get(vIndex + uvIndex + vPixelStride);
            nv21[nv21Index++] = uBuffer.get(uIndex + uvIndex + uPixelStride);
            nv21[nv21Index++] = yBuffer.get(yIndex + yOffset + (j + 1) * yPixelStride);
        }
    } else {
        for (int j = 0; j < w; j++) {
            nv21[nv21Index++] = yBuffer.get(yIndex + yOffset + j * yPixelStride);
        }
    }
}

return nv21;

}

Avatar
Avatar

您的代码已经很不错了,但是可以进一步改进。下面是改进后的代码:

public static byte[] convertYUV420888ToNV21(Image image) {
    byte[] nv21;
    ByteBuffer yBuffer = image.getPlanes()[0].getBuffer();
    ByteBuffer uBuffer = image.getPlanes()[1].getBuffer();
    ByteBuffer vBuffer = image.getPlanes()[2].getBuffer();
    int yRowStride = image.getPlanes()[0].getRowStride();
    int uRowStride = image.getPlanes()[1].getRowStride();
    int vRowStride = image.getPlanes()[2].getRowStride();
    int yPixelStride = image.getPlanes()[0].getPixelStride();
    int uPixelStride = image.getPlanes()[1].getPixelStride();
    int vPixelStride = image.getPlanes()[2].getPixelStride();

    int ySize = yBuffer.remaining();
    int uSize = uBuffer.remaining();
    int vSize = vBuffer.remaining();

    nv21 = new byte[ySize + uSize + vSize];

    int nv21Index = 0;
    int yIndex = 0;
    int uIndex = 0;
    int vIndex = 0;

    int w = image.getWidth();
    int h = image.getHeight();
    int uvRowStride = vRowStride;
    int uvPixelStride = vPixelStride;

    if (uRowStride != vRowStride || uPixelStride != vPixelStride) {
        // if the resolution is odd, the last pixel column of the U/V plane is dropped
        uvRowStride = (w % 2 == 0 ? uRowStride : vRowStride);
        uvPixelStride = (w % 2 == 0 ? uPixelStride : vPixelStride);
    }

    for (int i = 0; i < h; i++) {
        int yOffset = i * yRowStride;
        int uvOffset = (i / 2) * uvRowStride;
        for (int j = 0; j < w; j++) {
            if (i % 2 == 0 && j % 2 == 0) {
                int uvIndex = uvOffset + j * uvPixelStride;
                nv21[nv21Index++] = yBuffer.get(yIndex + yOffset + j * yPixelStride);
                nv21[nv21Index++] = vBuffer.get(vIndex + uvIndex);
                nv21[nv21Index++] = uBuffer.get(uIndex + uvIndex);
            } else {
                nv21[nv21Index++] = yBuffer.get(yIndex + yOffset + j * yPixelStride);
            }
        }
    }

    return nv21;
}

在上面的代码中,我们使用了单个循环来遍历所有像素。我们还将计算UV plane的偏移量移到了外层循环中,以避免在内层循环中进行重复计算。此外,我们将检查i和j是否为偶数,以决定当前像素是否为UV plane中的像素。这样可以减少循环次数,从而提高性能。

richyeohx-ax4MO

这个有没有内存上的问题。

Avatar
Avatar

在将YUV420_888数据转换为NV21数据时,可能会出现内存不足的问题。这是因为YUV420_888格式中的Y、U、V三个平面是分开存储的,而NV21格式中的Y和UV是交错存储的。因此,需要将Y、U、V三个平面合并成一个字节数组,再将其转换为NV21格式。

在合并三个平面时,需要创建一个新的字节数组,其大小为Y、U、V三个平面的大小之和。如果图像分辨率较高,这个字节数组可能会很大,从而导致内存不足的问题。为了避免这个问题,可以使用Java NIO中的DirectByteBuffer来代替普通的byte数组,这样可以将数据存储在堆外内存中,从而减少内存使用量。

以下是使用DirectByteBuffer的代码:

fun yuv420ToNv21(image: Image): ByteArray {
    val yBuffer = image.planes[0].buffer // Y plane
    val uBuffer = image.planes[1].buffer // U plane
    val vBuffer = image.planes[2].buffer // V plane

    val ySize = yBuffer.remaining()
    val uSize = uBuffer.remaining()
    val vSize = vBuffer.remaining()

    // Allocate DirectByteBuffer for NV21 data
    val nv21Buffer = ByteBuffer.allocateDirect(ySize + uSize + vSize)

    // Copy Y plane
    nv21Buffer.put(yBuffer)

    // Copy VU plane (interleaved)
    var uIndex = 0
    var vIndex = 0
    while (uIndex < uSize && vIndex < vSize) {
        nv21Buffer.put(vBuffer.get(vIndex++))
        nv21Buffer.put(uBuffer.get(uIndex++))
    }

    return nv21Buffer.array()
}

在上面的代码中,我们使用ByteBuffer.allocateDirect()方法分配了一个DirectByteBuffer对象,然后将Y、U、V三个平面的数据复制到该对象中。由于DirectByteBuffer对象存储在堆外内存中,因此不会占用应用程序的内存空间,从而减少了内存使用量。最后,我们将DirectByteBuffer对象转换为普通的byte数组并返回。

richyeohx-ax4MO

那这个我是不是要自己手动释放内存呢。

Avatar