Use a TensorFlow model exported from Azure Custom Vision

Azure Custom Vision is an Azure Cognitive Services service that lets you build and deploy your own image classification and object detection models. You can build computer vision models using either the Custom Vision web portal or the Custom Vision SDK and your preferred programming language. In the previous posts, I have shown you how to create image classification and object detection models through the Custom Vision web portal, test and publish your models and use the prediction API in Python apps.

To find out more about previous posts, check out the links below:

In this article, we will export our Custom Vision model in TensorFlow format for use with Python apps. You will learn how to:

  • Export a Custom Vision model through the web portal.
  • Run offline a TensorFlow model in a Python app for object detection.

To complete the exercise, you will need:

In this article, I will use a Custom Vision model for Vegetables detection. If you are interested in learning more about this project, you can read my article “Object detection model for grocery checkout with Azure Custom Vision”.

Export a Custom Vision model

Azure Custom Vision Service lets you export your image classification and object detection models to run locally on a device. For example, you can utilize the exported models in mobile applications or run a model in a microcontroller to deploy a Computer Vision application at the edge.

You can export a Custom Vision model in the following formats:

  • TensorFlow, TensorFlow Lite, TensorFlow.js,
  • CoreML,
  • ONNX,
  • Vision AI Developer Kit, and
  • Docker containers.
Export options in Azure Custom Vision

To export a Custom Vision model, you should use a Compact domain. Compact domains are optimized for real-time classification and object detection on edge devices.

Select a compact domain

In this section, you will convert the domain of your existing Custom Vision model to a compact one.

If you have already trained a Custom Vision model using a compact domain, you can skip the following steps.

  1. Sign in to the Custom Vision web portal and select your project.

  2. Then, click on the settings icon (⚙) at the top toolbar to view the project settings.

  3. Under Domains, select one of the Compact domains and click Save Changes.

    Select a compact domain

  4. Select the Train button to train the model using the new domain.

    Note that a slight variation in model’s performance metrics may be observed. Based on Microsoft Docs, classifiers built with a compact domain may be slightly less accurate than a standard domain with the same amount of training data.

Export the model

  1. Navigate to the Performance tab, select the latest Iteration, and then click the Export button.

    Export a model

  2. In the Choose your platform window, select TensorFlow – Android.

    Choose an export format

  3. Then, select TensorFlow in the dropdown list and click Export.

    Export a TensorFlow model

  4. The downloaded .zip file contains the following files:

    • a model.pb file which represents the trained model,
    • a labels.txt file which contains the classification labels, and
    • a sample Python application.

Run the sample app

To complete this exercise, you will need to install:

  • Python 3, and
  • Visual Studio Code.

Next, you will need to install the following packages:

1
2
3
4

pip install tensorflow
pip install pillow
pip install numpy
pip install opencv-python
In the following examples, I use a custom dataset with vegetable images to test my model. You can download these images from my GitHub repository.

  1. Unzip the downloaded folder and open the predict.py file.

  2. Open the cmd and run the Python script passing as argument the filename of an image.

  3. The prediction results will be displayed in the terminal.

    Prediction results

In the following sections, we will improve the way that the prediction results are displayed and try to build a “real-time” object detection app using OpenCV and our web-camera!

Want to view the whole Python script at once? You can find it on GitHub.

Modify the sample Python app

In the predict.py file, replace the main function with the function detect_image(), which opens a local image and returns the prediction results.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

def detect_image(image_filename):
    # Load a TensorFlow model
    graph_def = tf.compat.v1.GraphDef()
    with tf.io.gfile.GFile(MODEL_FILENAME, 'rb') as f:
        graph_def.ParseFromString(f.read())
    # Load labels
    with open(LABELS_FILENAME, 'r') as f:
        labels = [l.strip() for l in f.readlines()]
    od_model = TFObjectDetection(graph_def, labels)
    image = Image.open(image_filename)
    return od_model.predict_image(image)

Detect objects in images from camera with OpenCV

  1. Create a new Python file (camera.py) and import the following packages.

    1
2

    import cv2
from predict import detect_image

  2. Use the following code to take an image from your camera and save it in a file.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10

    # Set the size of the image (in pixels)
img_width = 640
img_height = 480
print("\n", "*"*8, "Starting camera!", "*"*8, "\n")
# Take an image from the camera and save it
camera = cv2.VideoCapture(0, cv2.CAP_DSHOW)
camera.set(cv2.CAP_PROP_FRAME_WIDTH, img_width)
camera.set(cv2.CAP_PROP_FRAME_HEIGHT, img_height)
ret, image = camera.read()
cv2.imwrite('capture.png', image)

  3. Then, use the function detect_image() to get the prediction results.

    1

    results = detect_image('capture.png')

  4. Now, you can display the predicted probabilities and a bounding box around every detected object.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

    # Select color for the bounding box (BGR)
colors = {
    "tomato": (0,215,255),
    "cucumber": (255,215,0),
    "pepper": (66,174,255)
}
# Display the results
for prediction in results:
    if prediction['probability'] > 0.3:
        print(f"{prediction['tagName']}: {prediction['probability'] * 100 :.2f}%")
        color = colors[prediction['tagName']]
        left = prediction['boundingBox']['left'] * img_width
        top = prediction['boundingBox']['top'] * img_height
        height = prediction['boundingBox']['height'] * img_height
        width =  prediction['boundingBox']['width'] * img_width
        result_image = cv2.rectangle(image, (int(left), int(top)), (int(left + width), int(top + height)), color, 3)
        cv2.putText(result_image, f"{prediction['tagName']}: {prediction['probability'] * 100 :.2f}%", (int(left), int(top)-10), fontFace = cv2.FONT_HERSHEY_SIMPLEX, fontScale = 0.5, color = color, thickness = 2)
        cv2.imwrite('result.png', result_image)
    OpenCV uses the BGR (Blue, Green, Red) color channel (instead of the RGB color space).

  5. Finally, release the camera.

    1

    camera.release()

Summary and next steps

In this article, you learned how to export an Azure Custom Vision model to run offline and use a TensorFlow model in a Python app. You can learn more about the available export options of Azure Custom Vision service on Microsoft Docs.

Here are some additional resources:

Foteini Savvidou

Written By

Foteini Savvidou

I am an Electrical and Computer Engineering student at Aristotle University of Thessaloniki, Greece, interested in AI, IoT, Wireless Communications, and Cloud Technologies, as well as in applications of technology in healthcare and education. I am also Microsoft AI MVP and Gold Microsoft Learn Student Ambassador.

You May Also Like