speech recognition - example of AlwaysOnHotwordDetector in Android

Question

Can someone provide an example of how to use the new AlwaysOnHotwordDetector class in Android?

I'd like to build an app, that when the app is running in the background, can detect a hotword like "next", or "back", or "pause".

Answer 1

Unless I have a huge blind spot, I don't think third-party applications can make use of this API. Its strange that AlwaysOnHotwordDetector (and related classes VoiceInteractionService etc.) have been granted public access.

---

If you are building a privileged app, look through these test projects from AOSP:

• Voice Interaction - Basic AlwaysOnHotwordDetector usage/implementation http://androidxref.com/5.0.0_r2/xref/frameworks/base/tests/VoiceInteraction/
• Voice Enrollment - http://androidxref.com/5.0.0_r2/xref/frameworks/base/tests/VoiceEnrollment/

---

While trying to make this work, I came upon this:

AlwaysOnHotwordDetector's constructor:

/**
 * @param text The keyphrase text to get the detector for.
 * @param locale The java locale for the detector.
 * @param callback A non-null Callback for receiving the recognition events.
 * @param voiceInteractionService The current voice interaction service.
 * @param modelManagementService A service that allows management of sound models.
 *
 * @hide
 */
public AlwaysOnHotwordDetector(String text, Locale locale, Callback callback,
        KeyphraseEnrollmentInfo keyphraseEnrollmentInfo,
        IVoiceInteractionService voiceInteractionService,
        IVoiceInteractionManagerService modelManagementService) {
    mText = text;
    mLocale = locale;
    mKeyphraseEnrollmentInfo = keyphraseEnrollmentInfo;
    mKeyphraseMetadata = mKeyphraseEnrollmentInfo.getKeyphraseMetadata(text, locale);
    mExternalCallback = callback;
    mHandler = new MyHandler();
    mInternalCallback = new SoundTriggerListener(mHandler);
    mVoiceInteractionService = voiceInteractionService;
    mModelManagementService = modelManagementService;
    new RefreshAvailabiltyTask().execute();
}

The statement of interest here is:

mKeyphraseMetadata = mKeyphraseEnrollmentInfo.getKeyphraseMetadata(text, locale);

What does KeyphraseEnrollmentInfo#getKeyphraseMetadata(String, Locale) do?

/**
 * Gets the {@link KeyphraseMetadata} for the given keyphrase and locale, null if any metadata
 * isn't available for the given combination.
 *
 * @param keyphrase The keyphrase that the user needs to be enrolled to.
 * @param locale The locale for which the enrollment needs to be performed.
 *        This is a Java locale, for example "en_US".
 * @return The metadata, if the enrollment client supports the given keyphrase
 *         and locale, null otherwise.
 */
public KeyphraseMetadata getKeyphraseMetadata(String keyphrase, Locale locale) {
    if (mKeyphrases == null || mKeyphrases.length == 0) {
        Slog.w(TAG, "Enrollment application doesn't support keyphrases");
        return null;
    }
    for (KeyphraseMetadata keyphraseMetadata : mKeyphrases) {
        // Check if the given keyphrase is supported in the locale provided by
        // the enrollment application.
        if (keyphraseMetadata.supportsPhrase(keyphrase)
                && keyphraseMetadata.supportsLocale(locale)) {
            return keyphraseMetadata;
        }
    }
    Slog.w(TAG, "Enrollment application doesn't support the given keyphrase/locale");
    return null;
}

At this point, my example project kept telling me: Enrollment application doesn't support keyphrases. So, digging a bit further - to support keyphrases, we need to provide additional meta-data:

// Taken from class KeyphraseEnrollmentInfo
/**
 * Name under which a Hotword enrollment component publishes information about itself.
 * This meta-data should reference an XML resource containing a
 * <code>&lt;{@link
 * android.R.styleable#VoiceEnrollmentApplication
 * voice-enrollment-application}&gt;</code> tag.
 */
private static final String VOICE_KEYPHRASE_META_DATA = "android.voice_enrollment";

Additionally, we will need the android.permission.MANAGE_VOICE_KEYPHRASES permission. This is where the example project gets stuck:

<!-- Must be required by hotword enrollment application,
     to ensure that only the system can interact with it.
     @hide <p>Not for use by third-party applications.</p> -->
<permission android:name="android.permission.MANAGE_VOICE_KEYPHRASES"
    android:label="@string/permlab_manageVoiceKeyphrases"
    android:description="@string/permdesc_manageVoiceKeyphrases"
    android:protectionLevel="signature|system" />

The permission required to support hotword detection is not available to third-party applications. I still can't figure out why package android.service.voice package has public access. Perhaps I am missing something here.