android - Converting ImageProxy to Bitmap

Question

So, I wanted to explore new Google's Camera API - CameraX. What I want to do, is take an image from camera feed every second and then pass it into a function that accepts bitmap for machine learning purposes.

I read the documentation on Camera X Image Analyzer:

The image analysis use case provides your app with a CPU-accessible image to perform image processing, computer vision, or machine learning inference on. The application implements an Analyzer method that is run on each frame.

..which basically is what I need. So, I implemented this image analyzer like this:

imageAnalysis.setAnalyzer { image: ImageProxy, _: Int ->
    viewModel.onAnalyzeImage(image)
}

What I get is image: ImageProxy. How can I transfer this ImageProxy to Bitmap?

I tried to solve it like this:

fun decodeBitmap(image: ImageProxy): Bitmap? {
    val buffer = image.planes[0].buffer
    val bytes = ByteArray(buffer.capacity()).also { buffer.get(it) }
    return BitmapFactory.decodeByteArray(bytes, 0, bytes.size)
}

But it returns null - because decodeByteArray does not receive valid (?) bitmap bytes. Any ideas?

Answer 1

You will need to check the image.format to see if it is ImageFormat.YUV_420_888. If so , then you can you use this extension to convert image to bitmap:

fun Image.toBitmap(): Bitmap {
    val yBuffer = planes[0].buffer // Y
    val vuBuffer = planes[2].buffer // VU

    val ySize = yBuffer.remaining()
    val vuSize = vuBuffer.remaining()

    val nv21 = ByteArray(ySize + vuSize)

    yBuffer.get(nv21, 0, ySize)
    vuBuffer.get(nv21, ySize, vuSize)

    val yuvImage = YuvImage(nv21, ImageFormat.NV21, this.width, this.height, null)
    val out = ByteArrayOutputStream()
    yuvImage.compressToJpeg(Rect(0, 0, yuvImage.width, yuvImage.height), 50, out)
    val imageBytes = out.toByteArray()
    return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.size)
}

This works for a number of camera configurations. However, you might need to use a more advanced method that considers pixel strides.