gcloud - What does google cloud ml-engine do when a Json request contains "_bytes" or "b64"?

Question

The google cloud documentation (see Binary data in prediction input) states:

Your encoded string must be formatted as a JSON object with a single key named b64. The following Python example encodes a buffer of raw JPEG data using the base64 library to make an instance:

{"image_bytes":{"b64": base64.b64encode(jpeg_data)}}

In your TensorFlow model code, you must name the aliases for your input and output tensors so that they end with '_bytes'.

I would like to understand more about how this process works on the google cloud side.

• Is the ml-engine automatically decoding any content after the "b64" string to byte data?

• When the request has this nested structure, does it only pass in the "b64" section to the serving input function and remove the "image_bytes" key?

• Is each request passed individually to the serving input function or are they batched?

• Do we define the input output aliases in the ServingInputReceiver returned by the serving input function?

I have found no way to create a serving input function which uses this nested structure to define the feature placeholders. I only use "b64" in mine and I am not sure what the gcloud ml-engine does on receiving the requests.

Additionally when predicting locally using gcloud ml-engine local predict, sending the request with the nested structure fails, (unexpected key image_bytes as it is not defined in the serving input function). But when predicting using gcloud ml-engine predict, sending requests with the nested structure works even when the serving input function contains no reference to "image_bytes". The gcloud predict also works when leaving out "image_bytes" and passing in just "b64".

An example serving input function

def serving_input_fn():
    feature_placeholders = {'b64': tf.placeholder(dtype=tf.string,
                                                  shape=[None],
                                                  name='source')}
    single_image = tf.decode_raw(feature_placeholders['b64'], tf.float32)
    inputs = {'image': single_image}
    return tf.estimator.export.ServingInputReceiver(inputs, feature_placeholders)

I gave the example using images but I assume the same should apply to all types of data sent as bytes and base64 encoded.

There are a lot of stackoverflow questions which contain references to the need to include "_bytes" with snippets of information, but I would find it useful if someone could explain a bit more in detail whats going on as then I wouldn't be so hit and miss when formatting requests.

Stackoverflow questions on this topic

• how make correct predictions of jpeg image in cloud-ml

• How convert a jpeg image into json file in Google machine learning

• how make correct predictions of jpeg image in cloud-ml

• Base64 images with Keras and Google Cloud ML

• How to read a utf-8 encoded binary string in tensorflow?

Answer 1

Waitting for answers