Question:
I attempting to deploy the universal-sentence-encoder model to a aws Sagemaker endpoint and am getting the error raise ValueError('no SavedModel bundles found!')
I have shown my code below, I have a feeling that one of my paths is incorrect
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
import tensorflow as tf import tensorflow_hub as hub import numpy as np from sagemaker import get_execution_role from sagemaker.tensorflow.serving import Model def tfhub_to_savedmodel(model_name,uri): tfhub_uri = uri model_path = 'encoder_model/' + model_name with tf.Session(graph=tf.Graph()) as sess: module = hub.Module(tfhub_uri) input_params = module.get_input_info_dict() dtype = input_params['text'].dtype shape = input_params['text'].get_shape() # define the model inputs inputs = {'text': tf.placeholder(dtype, shape, 'text')} # define the model outputs # we want the class ids and probabilities for the top 3 classes logits = module(inputs['text']) outputs = { 'vector': logits, } # export the model sess.run([tf.global_variables_initializer(), tf.tables_initializer()]) tf.saved_model.simple_save( sess, model_path, inputs=inputs, outputs=outputs) return model_path sagemaker_role = get_execution_role() !tar -C "$PWD" -czf encoder.tar.gz encoder_model/ model_data = Session().upload_data(path='encoder.tar.gz',key_prefix='model') env = {'SAGEMAKER_TFS_DEFAULT_MODEL_NAME': 'universal-sentence-encoder-large'} model = Model(model_data=model_data, role=sagemaker_role, framework_version=1.12, env=env) predictor = model.deploy(initial_instance_count=1, instance_type='ml.t2.medium') |
Answer:
I suppose you started from this example? https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-python-sdk/tensorflow_serving_container
It looks like you’re not saving the TF Serving bundle properly: the model version number is missing, because of this line:
|
1 2 |
model_path = 'encoder_model/' + model_name |
Replacing it with this should fix your problem:
|
1 2 |
model_path = '{}/{}/00000001'.format('encoder_model/', model_name) |
Your model artefact should look like this (I used the model in the notebook above):
|
1 2 3 4 5 6 7 8 |
mobilenet/ mobilenet/mobilenet_v2_140_224/ mobilenet/mobilenet_v2_140_224/00000001/ mobilenet/mobilenet_v2_140_224/00000001/saved_model.pb mobilenet/mobilenet_v2_140_224/00000001/variables/ mobilenet/mobilenet_v2_140_224/00000001/variables/variables.data-00000-of-00001 mobilenet/mobilenet_v2_140_224/00000001/variables/variables.index |
Then, upload to S3 and deploy.