Build a flower classification model with Azure Custom Vision

Azure Custom Vision is an Azure Cognitive Services service that lets you build and deploy your own image classification and object detection models. Image classification models apply labels to an image, while object detection models return the bounding box coordinates in the image where the applied labels can be found.

In this article, we will explore the pre-trained models of Azure Custom Vision service for image classification. We will build and deploy a custom computer vision model for flower classification. You will learn how to:

• Provision a Custom Vision resource.
• Build and train a custom image classification model in Azure Custom Vision.
• Deploy and consume the model.

To complete the exercise, you will need an Azure subscription. If you don’t have one, you can sign up for an Azure free account.

What is Custom Vision?

Study the following sketch note to learn how Azure Custom Vision works.

You can find more information and how-to-guides about Custom Vision on Microsoft Learn and Microsoft Docs.

Understand the problem

Every machine learning project starts with a question. Our question is, can we identify a flower’s category from an image of a flower, to help document the different types of flowers in our city?

Collect the data

Now that we know what to ask the model, we want to find data that would help us answer the question that we’re interested in. To build and train our machine learning model, we will use the 17 Category Flower Dataset from the Visual Geometry Group (University of Oxford). We will use 3 out of 17 flower categories: Iris, Tigerlily and Tulip. You can download the 3 Category Flower Dataset from my GitHub repository. You can download the full dataset from the Visual Geometry Group’s website.

Create a Custom Vision Resource

To use the Custom Vision service, you can either create a Custom Vision resource or a Cognitive Services resource. If you plan to use Custom Vision along with other cognitive services, you can create a Cognitive Services resource.

In this exercise, you will create a Custom Vision resource.

1. Sign in to the Azure Portal and select Create a resource.

2. Search for Custom Vision and in the Custom Vision card click Create.

3. Create a Custom Vision resource with the following settings:

   • Create options: Select Both.
   • Subscription: Your Azure subscription.
   • Resource group: Select an existing resource group or create a new one.
   • Name: This would be your custom domain name in your endpoint. Enter a unique name.
   • Training Resource:
     • Location: Choose any available region, for example East US.
     • Pricing tier: You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier.
   • Prediction Resource:
     • Location: Choose any available region, for example East US.
     • Pricing tier: You can use the free pricing tier (F0) to try the service, and upgrade later to a paid tier.
   

4. Select Review + Create and wait for deployment to complete.

5. Once the deployment is complete, select Go to resource. Two Custom Vision resources are provisioned, one for training and one for prediction.

Create a new Custom Vision project

You can build and train your model by using the web portal or the Custom Vision SDKs and your preferred programming language. In this article, I will show you how to build a computer vision model using the Custom Vision web portal.

1. Navigate to the Custom Vision portal and sign in.

2. Create a new project with the following settings:

   • Name: ClassifyFlowers
   • Description: Image classification for flowers.
   • Resource: The Custom Vision resource you created in the previous step.
   • Project Types: Classification
   • Classification Types: Multiclass (single tag per image)
   • Domains: General. Learn more about Custom Vision project domains at Microsoft Docs.
   

3. Select Create project.

Upload and tag images

1. In your Custom Vision project, select Add images.

2. Select all the images in the Daisy folder you extracted previously. Then upload the image files and specify the tag daisy.

   

3. Repeat the previous step to upload the images in the Iris and Tigerlily folders with the tags iris and tigerlily, respectively.

   

4. Explore the images that you have uploaded. There should be 76 images of each flower category.

Train the model

1. In the top menu bar, click the Train button to train the model using the tagged images.

2. Then, in the Choose Training Type window, select Quick Training and wait for the training iteration to complete.

   

Evaluate the classification model

1. When the training finishes, information about the model’s performance is estimated and displayed.

   

2. The Custom Vision service calculates three metrics:

   • Precision indicates the percentage of the class predictions that were correct.
   • Recall indicates the percentage of class predictions that were correctly identified.
   • Average precision (AP) measures model performance by computing the precision and recall at different thresholds.

Test the model

Before publishing our model, let’s test it and see how it performs on new data. We will use the flower images in the Test folder you extracted previously.

1. In the top menu bar, select Quick Test.

2. In the Quick Test window, click the Browse local files button and select a local image. The prediction is shown in the window.

   

3. The images that you uploaded appear in the Predictions tab. You can add these images to your model and then retrain your model.

   

Deploy the model

Once your model is performing at a satisfactory level, you can deploy it.

Publish the model

1. In the Performance tab, select Publish.

   

2. In the Publish Model window, under Prediction resource, select the name of your Custom Vision prediction resource and then click Publish.

   

3. Once your model has been successfully published, you’ll see a Published label appear next to your iteration name in the left sidebar.

Get the ID of your project

In the Custom Vision portal, click the settings icon (⚙) at the top toolbar to view the project settings. Then, under General, copy the Project ID.

Get the key and endpoint of the prediction resource

Navigate to the Custom Vision portal homepage and select the settings icon (⚙) at the top right. Expand your prediction resource and save the Key and the Endpoint, because you will need these values to build the Python app.

Test the prediction endpoint in a Python app

To create an image classification app with Custom Vision for Python, you’ll need to install the Custom Vision client library. Install the Azure Cognitive Services Custom Vision SDK for Python package with pip:

1

pip install azure-cognitiveservices-vision-customvision

Then, use the following code to call the prediction API in Python (code source: Microsoft Docs – Python Quickstart).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
from msrest.authentication import ApiKeyCredentials
import os
# Get path to images folder
dirname = os.path.dirname(__file__)
images_folder = os.path.join(dirname, 'images/Test')
# Create variables for your project
publish_iteration_name = "Iteration1"
project_id = "<YOUR_PROJECT_ID>"
# Create variables for your prediction resource
prediction_key = "<YOUR_KEY>"
endpoint = "<YOUR_ENDPOINT>"
prediction_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
predictor = CustomVisionPredictionClient(endpoint, prediction_credentials)
# Open an image and make a prediction
with open(os.path.join(images_folder, "tigerlily4.jpg"), "rb") as image_contents:
    results = predictor.classify_image(project_id, publish_iteration_name, image_contents.read())
# Display the results
for prediction in results.predictions:
    print(f"{prediction.tag_name}: {prediction.probability * 100 :.2f}%")

Replace <YOUR_PROJECT_ID>, <YOUR_KEY> and <YOUR_ENDPOINT> with the ID of your project, the Key and the Endpoint of your prediction resource, respectively.

Use the following code to display the predicted class of all the images in the Test folder:

1
2
3
4
5
6
7
8

images = os.listdir(images_folder)
for i in range(len(images)):
    # Open the image, and use the custom vision model to classify it
    image_contents = open(os.path.join(images_folder, images[i]), "rb")
    results = predictor.classify_image(project_id, publish_iteration_name, image_contents.read())

    # Print the predicted class
    print(f"Image {images[i]}: {results.predictions[0].tag_name} {results.predictions[0].probability * 100 :.2f}%")

Summary and next steps

In this article, you learned how to use Azure Custom Vision service to create an image classification model. In the next article, you will build an object detection model!

Clean-up

If you have finished learning, you can delete the resource group from your Azure subscription:

1. In the Azure Portal, select Resource groups on the right menu and then select the resource group that you have created.

2. Click Delete resource group.