tensorflow lite - Flutter Camera Plugin Transformations

Question

I am trying to use the Camera Plugin for Tensorflow image classification, but I am not understanding how to work with the CameraImage object.

I am able to see it's height, width and 'Planes', which is what eventually is passed into the Tensorflow Lite plugin as a bytelist.

  await Tflite.runModelOnFrame(
      bytesList: image.planes.map((plane) {
        return plane.bytes;
      }).toList(),
      numResults: 5,
  )

However the image is needing all sorts of transformations:

Sure that I need...

1. Crop the top and bottom off to make a square.
2. Scale to 200 by 200 resolution.

Not sure if I need...

3. Remove the alpha component and get the rgb data

4. Quantize the image data, i.e. making the images go from 0-255 to 0-1 in value

I am trying to manipulate the CameraImage but I am not really sure what the planes attribute is. It contains 3 Planes objects, which appear to bytes, which I am guessing vaguely represent pixels on a screen.

The camera readme doesn't appear to document what exactly a plane is: https://pub.dev/packages/camera

All the image manipulation plugins seem to be based around the image_picker plugin and/or Files. I can't find any that work with a CameraImage object and I am unsure of how to work with the Planes directly. I looked at the image_picker plugin but every tensorflow lite tutorial seemed to prefer camera, so I have avoided it for now.

Any advice would be appreciated, thank you.

question from:https://stackoverflow.com/questions/65855920/flutter-camera-plugin-transformations

Answer 1

Waitting for answers