java - MLKit BarCode Sanner Implementation results in internal error when executing ML Kit tasks

Question

Every call to scanner.process(image) is resulting in the onFailure call, with the following error:

Error:com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks

Any further details or ideas as to why the task is failing? Presenting it with a barcode does not seem to change this behavior either.

This error is shown when using a Pixel 3a, but totally crashes a Nexus 5 with a SIGENV error. I've been building my code from snippets while reading through the overview

Entry point activity

package jp.oist.cameraxapp;

import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.appcompat.app.AppCompatActivity;
import androidx.camera.core.Camera;
import androidx.camera.core.CameraSelector;
import androidx.camera.core.ImageAnalysis;
import androidx.camera.core.ImageCapture;
import androidx.camera.core.ImageProxy;
import androidx.camera.core.Preview;
import androidx.camera.lifecycle.ProcessCameraProvider;
import androidx.camera.view.PreviewView;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;
import androidx.lifecycle.LifecycleOwner;

import android.annotation.SuppressLint;
import android.content.Context;
import android.content.pm.PackageInfo;
import android.content.pm.PackageManager;
import android.media.Image;
import android.os.Bundle;
import android.os.Handler;
import android.os.Looper;
import android.util.Log;
import android.widget.Toast;

import com.google.android.gms.tasks.OnFailureListener;
import com.google.android.gms.tasks.OnSuccessListener;
import com.google.android.gms.tasks.Task;
import com.google.common.util.concurrent.ListenableFuture;
import com.google.mlkit.vision.barcode.Barcode;
import com.google.mlkit.vision.barcode.BarcodeScanner;
import com.google.mlkit.vision.barcode.BarcodeScannerOptions;
import com.google.mlkit.vision.barcode.BarcodeScanning;
import com.google.mlkit.vision.common.InputImage;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;

public class MainActivity extends AppCompatActivity {

    private static final int PERMISSION_REQUESTS = 1;
    private ListenableFuture<ProcessCameraProvider> cameraProviderFuture;
    private ExecutorService executor;
    private static String TAG = "abcvlibCameraX";

    @Override
    public void onCreate(@Nullable Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        if (!allPermissionsGranted()) {
            getRuntimePermissions();
        }

        setContentView(R.layout.activity_main);

        executor = Executors.newSingleThreadExecutor();

        PreviewView previewView = findViewById(R.id.previewView);

        cameraProviderFuture = ProcessCameraProvider.getInstance(this);

        cameraProviderFuture.addListener(() -> {
            try {
                // Camera provider is now guaranteed to be available
                ProcessCameraProvider cameraProvider = cameraProviderFuture.get();

                // Set up the view finder use case to display camera preview
                Preview preview = new Preview.Builder().build();

                // Choose the camera by requiring a lens facing
                CameraSelector cameraSelector = new CameraSelector.Builder()
                        .requireLensFacing(CameraSelector.LENS_FACING_FRONT)
                        .build();

                // Connect the preview use case to the previewView
                preview.setSurfaceProvider(
                        previewView.createSurfaceProvider());

                // Set up the capture use case to allow users to take photos
                ImageCapture imageCapture = new ImageCapture.Builder()
                        .setCaptureMode(ImageCapture.CAPTURE_MODE_MINIMIZE_LATENCY)
                        .build();

                ImageAnalysis imageAnalysis =
                        new ImageAnalysis.Builder()
                                .build();

                imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this), new ImageAnalysis.Analyzer() {

                    private BarcodeScanner scanner = buildBarCodeScanner();
                    @Override
                    public void analyze(ImageProxy imageProxy) {
                        @SuppressLint("UnsafeExperimentalUsageError") Image mediaImage = imageProxy.getImage();
                        if (mediaImage != null) {
                            InputImage image =
                                    InputImage.fromMediaImage(mediaImage, imageProxy.getImageInfo().getRotationDegrees());
                            // Pass image to an ML Kit Vision API
                            Task<List<Barcode>> result = scanner.process(image);

                            result.addOnSuccessListener(new OnSuccessListener<List<Barcode>>() {
                                @Override
                                public void onSuccess(List<Barcode> barcodes) {
                                    // Task completed successfully
                                    Log.i("CameraXApp3", "scanner task successful");
                                    processBarCodes(barcodes);
                                }
                            }).addOnFailureListener(new OnFailureListener() {
                                @Override
                                public void onFailure(@NonNull Exception e) {
                                    // Task failed with an exception
                                    Log.i("CameraXApp3", "scanner task failed. Error:" + e);

                                }
                            });
                            mediaImage.close();
                            imageProxy.close();
                        }
                    }
                    private BarcodeScanner buildBarCodeScanner() {
                        BarcodeScannerOptions options =
                                new BarcodeScannerOptions.Builder()
                                        .setBarcodeFormats(
                                                Barcode.FORMAT_QR_CODE)
                                        .build();
                        return BarcodeScanning.getClient(options);
                    }
                    //
                    private void processBarCodes(List<Barcode> barcodes) {
                        for (Barcode barcode : barcodes) {
                            String rawValue = barcode.getRawValue();
                            int valueType = barcode.getValueType();
                            // See API reference for complete list of supported types
                            if (valueType == Barcode.TYPE_TEXT) {
                                toast(MainActivity.this, "Received Message:" + rawValue);
                            }
                        }
                    }

                    public void toast(final Context context, final String text) {
                        Handler handler = new Handler(Looper.getMainLooper());
                        handler.post(() -> Toast.makeText(context, text, Toast.LENGTH_LONG).show());
                    }
                });

                // Attach use cases to the camera with the same lifecycle owner
                Camera camera = cameraProvider.bindToLifecycle(
                        ((LifecycleOwner) this),
                        cameraSelector,
                        preview,
                        imageCapture,
                        imageAnalysis);

            } catch (InterruptedException | ExecutionException e) {
                // Currently no exceptions thrown. cameraProviderFuture.get() should
                // not block since the listener is being called, so no need to
                // handle InterruptedException.
            }
        }, ContextCompat.getMainExecutor(this));
    }

    private void getRuntimePermissions() {
        List<String> allNeededPermissions = new ArrayList<>();
        for (String permission : getRequiredPermissions()) {
            if (!isPermissionGranted(this, permission)) {
                allNeededPermissions.add(permission);
            }
        }

        if (!allNeededPermissions.isEmpty()) {
            ActivityCompat.requestPermissions(
                    this, allNeededPermissions.toArray(new String[0]), PERMISSION_REQUESTS);
        }
    }

    private static boolean isPermissionGranted(Context context, String permission) {
        if (ContextCompat.checkSelfPermission(context, permission)
                == PackageManager.PERMISSION_GRANTED) {
            Log.i(TAG, "Permission granted: " + permission);
            return true;
        }
        Log.i(TAG, "Permission NOT granted: " + permission);
        return false;
    }

    private boolean allPermissionsGranted() {
        for (String permission : getRequiredPermissions()) {
            if (!isPermissionGranted(this, permission)) {
                return false;
            }
        }
        return true;
    }

    private String[] getRequiredPermissions() {
        try {
            PackageInfo info =
                    this.getPackageManager()
                            .getPackageInfo(this.getPackageName(), PackageManager.GET_PERMISSIONS);
            String[] ps = info.requestedPermissions;
            if (ps != null && ps.length > 0) {
                return ps;
            } else {
                return new String[0];
            }
        } catch (Exception e) {
            return new String[0];
        }
    }

}

App-level Gradle File

apply plugin: 'com.android.application'

android {
    compileSdkVersion 29
    buildToolsVersion "30.0.0"

    defaultConfig {
        applicationId "jp.oist.cameraxapp"
        minSdkVersion 21
        targetSdkVersion 29
        versionCode 1
        versionName "1.0"

        testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
}

dependencies {
    implementation fileTree(dir: "libs", include: ["*.jar"])
    implementation 'androidx.appcompat:appcompat:1.1.0'
    implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
    implementation 'androidx.camera:camera-camera2:1.0.0-beta07'
    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'androidx.test.ext:junit:1.1.1'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
    // CameraX core library using the camera2 implementation
    def camerax_version = "1.0.0-beta07"
    implementation "androidx.camera:camera-core:${camerax_version}"
    implementation "androidx.camera:camera-camera2:${camerax_version}"
    implementation "androidx.camera:camera-lifecycle:${camerax_version}"
    implementation "androidx.camera:camera-view:1.0.0-alpha14"
    implementation 'com.google.mlkit:barcode-scanning:16.0.

Answer 1

Inadvertently found the cause of the error by reading this. Adding an OnCompleteListener to the result Task and placing the imageProxy.close() inside, resolved this. i.e.:

Errors

Task<List<Barcode>> result = scanner.process(image);

result.addOnSuccessListener(new OnSuccessListener<List<Barcode>>() {
    @Override
    public void onSuccess(List<Barcode> barcodes) {
        // Task completed successfully
        Log.i("CameraXApp3", "scanner task successful");
        processBarCodes(barcodes);
    }
}).addOnFailureListener(new OnFailureListener() {
    @Override
    public void onFailure(@NonNull Exception e) {
        // Task failed with an exception
        Log.i("CameraXApp3", "scanner task failed. Error:" + e);

    }
});
mediaImage.close();
imageProxy.close();

No Errors

Task<List<Barcode>> result = scanner.process(image);

result.addOnSuccessListener(new OnSuccessListener<List<Barcode>>() {
    @Override
    public void onSuccess(List<Barcode> barcodes) {
        // Task completed successfully
        Log.i("CameraXApp3", "scanner task successful");
        processBarCodes(barcodes);
    }
}).addOnFailureListener(new OnFailureListener() {
    @Override
    public void onFailure(@NonNull Exception e) {
        // Task failed with an exception
        Log.i("CameraXApp3", "scanner task failed. Error:" + e);

    }
}).addOnCompleteListener(new OnCompleteListener<List<Barcode>>() {
    @Override
    public void onComplete(@NonNull Task<List<Barcode>> task) {
        mediaImage.close();
        imageProxy.close();
    }
});

Will wait to mark as answer in case @Hoi wants to grab the points, which he deserves. :)