Identify plant leaf diseases with the Custom Vision SDK for Python

One important research area in precision agriculture is the detection of diseases from images of plant leaves. How about using the Azure Custom Vision service to help farmers identify plant diseases from images?

In this article, you will build and deploy a custom vision model for identifying tomato leaf diseases. You will learn how to:

• Build and train a custom image classification model in Custom Vision SDK for Python.
• Deploy and consume the model using Python.

To complete the exercise, you will need:

• An Azure subscription. If you don’t have one, you can sign up for an Azure free account.
• A Custom Vision resource.

You will also need to install Python 3 and Visual Studio Code or another code editor.

What is Azure Custom Vision?

Azure Custom Vision is an Azure Cognitive Services service that lets you build and deploy your own image classification and object detection models. You can train your models using either the Custom Vision web portal or the Custom Vision SDKs.

Do you want to learn more about Azure Custom Vision? You can read my previous articles about creating a Custom Vision model for flower classification and an object detection model for grocery checkout.

Collect the data

Images of tomato leaf diseases were taken from the Plant Leaf Diseases database by Arun Pandian et al. This dataset is composed of 61,486 images, which are divided into 39 classes of plant leaf diseases. To build and train our Custom Vision model, we will only consider the following classes:

• Tomato_bacterial_spot
• Tomato_early_blight
• Tomato_healthy
• Tomato_late_blight
• Tomato_leaf_mold
• Tomato_septoria_leaf_spot
• Tomato_spider_mites_two-spotted_spider_mite
• Tomato_target_spot
• Tomato_mosaic_virus
• Tomato_yellow_leaf_curl_virus

We will use 60 images per class for the first training iteration. You can download the dataset employed in this project from the Mendeley Data website.

Get the key and endpoint of the Custom Vision resource

Once you have created a Custom Vision resource in the Azure portal, you will need the key and endpoint of your training and prediction resources to connect your Python application to Custom Vision. You can find these values both in the Azure portal and the Custom Vision web portal.

1. Navigate to customvision.ai and sign in.

2. Click the settings icon (⚙) at the top toolbar.

3. Expand your prediction resource and save the Key, the Endpoint, and the Prediction resource ID.

4. Expand the training resource and save the Key and the Endpoint.

   

Set up your application

Install the client library

To create an image classification project with Custom Vision for Python, you’ll need to install the Custom Vision client library. Install the Azure Cognitive Services Custom Vision SDK for Python package with pip:

1

pip install azure-cognitiveservices-vision-customvision

Create a configuration file

Create a configuration file (.env) and save the Azure’s keys and endpoints you copied in the previous step.

Create a new Python application

1. Create a new Python file (app.py) and import the following libraries:

   1 2 3 4 5 6

   from azure.cognitiveservices.vision.customvision.training import CustomVisionTrainingClient from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient from azure.cognitiveservices.vision.customvision.training.models import ImageFileCreateBatch, ImageFileCreateEntry from msrest.authentication import ApiKeyCredentials from dotenv import load_dotenv import os, time

2. Add the following code to load the keys and endpoints from the configuration file.

   1 2 3 4 5 6

   load_dotenv() training_endpoint = os.getenv('TRAINING_ENDPOINT') training_key = os.getenv('TRAINING_KEY') prediction_endpoint = os.getenv('PREDICTION_ENDPOINT') prediction_key = os.getenv('PREDICTION_KEY') prediction_resource_id = os.getenv('PREDICTION_RESOURCE_ID')

Authenticate the client

Use the following code to create a CustomVisionTrainingClient and CustomVisionPredictionClient object. You will use the trainer object to create a new Custom Vision project and train an image classification model and the predictor object to make a prediction using the published endpoint.

1
2
3
4

credentials = ApiKeyCredentials(in_headers={"Training-key": training_key})
trainer = CustomVisionTrainingClient(training_endpoint, credentials)
prediction_credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
predictor = CustomVisionPredictionClient(prediction_endpoint, prediction_credentials)

Create a new project

To create a new Custom Vision project, you use the create_project method of the trainer object. By default, the domain of the new project is set to General. If you want to export your model to run locally on a device, you should select one of the compact domains.

Add the following code to create a new “Multiclass” classification project in the General (compact) domain.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

# Find the domain id
classification_domain = next(domain for domain in trainer.get_domains() if domain.type == "Classification" and domain.name == "General (compact)")

# Create a new project
publish_iteration_name = "Iteration1"
project_name = "Tomato leaf diseases"
project_description = "A Custom Vision project to classify tomato leaf diseases"
domain_id = classification_domain.id
classification_type = "Multiclass"
print ("Creating project...")
project = trainer.create_project(project_name, project_description, domain_id, classification_type)

Create the tags

The images that we will use to build our model are grouped into 10 classes (tags). Insert the following code after the project creation. This code specifies the description of the tags and the names of the folders with the training images of each class and adds classification tags to the project using the create_tag method.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

# Add helper variables
tags_folder_names = [ "Bacterial_spot", "Early_blight", "Healthy", "Late_blight", "Leaf_Mold",
                      "Septoria_leaf_spot", "Spider_mites", "Target_Spot", "Tomato_mosaic_virus",
                      "Tomato_Yellow_Leaf_Curl_Virus" ]
tags_description = [ "Bacterial spot", "Early blight", "Healthy", "Late blight", "Leaf Mold",
                      "Septoria leaf spot", "Spider mites", "Target Spot", "Tomato mosaic virus",
                      "Tomato Yellow Leaf Curl Virus" ]

# Add tags
tags = [trainer.create_tag(project.id, tag_description) for tag_description in tags_description]

Upload and tag images

The following code uploads the training images and their corresponding tags. In each batch, it uploads 60 images of the same tag.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

# Upload and tag images
images_folder = os.path.join(os.path.dirname(__file__), "images", "Train")

print("Adding images...")

for i in range(0, 10):
    image_list = []
    for image_num in range(1, 61):
        file_name = f"{tags_folder_names[i]} ({image_num}).JPG"
        with open(os.path.join(images_folder, tags_folder_names[i], file_name), "rb") as image_contents:
            image_list.append(ImageFileCreateEntry(name=file_name, contents=image_contents.read(), tag_ids=[tags[i].id]))

    upload_result = trainer.create_images_from_files(project.id, ImageFileCreateBatch(images=image_list))
    if not upload_result.is_batch_successful:
        print("Image batch upload failed.")
        for image in upload_result.images:
            print("Image status: ", image.status)
        exit(-1)
    print(f"{tags_folder_names[i]} Uploaded")

Train and evaluate the model

Use the train_project method of the trainer object to train your model using all the images in the training set.

1
2
3
4
5
6
7
8

# Training
print ("Training...")
iteration = trainer.train_project(project.id)
while (iteration.status != "Completed"):
    iteration = trainer.get_iteration(project.id, iteration.id)
    print ("Training status: " + iteration.status)
    print ("Waiting 10 seconds...")
    time.sleep(10)

The Custom Vision service calculates three performance metrics:

• precision,
• recall, and
• average precision.

The following code displays performance information for the latest training iteration using a threshold value equal to 50%. You can also use the get_iteration_performance method to get the standard deviation of the above-mentioned performance values.

1
2
3
4
5
6
7

# Get iteration performance information
threshold = 0.5
iter_performance_info = trainer.get_iteration_performance(project.id, iteration.id, threshold)
print("Iteration Performance: ")
print(f"\tPrecision: {iter_performance_info.precision*100 :.2f}%\n"
      f"\tRecall: {iter_performance_info.recall*100 :.2f}%\n"
      f"\tRecall: {iter_performance_info.average_precision*100 :.2f}%")

Publish the iteration

Add the following code, which publishes the current iteration. Once the iteration is published, you can use the prediction endpoint to make predictions.

1
2
3

# Publish the current iteration
trainer.publish_iteration(project.id, iteration.id, publish_iteration_name, prediction_resource_id)
print ("Iteration published!")

Test the prediction endpoint

Use the classify_image method of the predictor object to send a new image for analysis and retrieve the prediction result. Add the following code to the end of the program and run the application.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

# Test - Make a prediction
test_images_folder_path = os.path.join(os.path.dirname(__file__), "images", "Test")

print("Testing the prediction endpoint...")
test_image_filename = "Bacterial_spot (1).JPG"
with open(os.path.join(test_images_folder_path, test_image_filename), "rb") as image_contents:
    results = predictor.classify_image(project.id, publish_iteration_name, image_contents.read())

    # Display the results
    print(f"Testing image {test_image_filename}...")
    for prediction in results.predictions:
        print(f"\t{prediction.tag_name}: {prediction.probability*100 :.2f}%")

If the application ran successfully, navigate to the Custom Vision website to see the newly created project.

Summary and next steps

In this article, you learned how to use the Azure Custom Vision SDK for Python to build and train an image classification model. You may check out the following article on Microsoft Docs about how to retrain the model in order to make it more accurate: How to improve your Custom Vision model – Microsoft Docs.

Clean-up

If you want to delete this project, navigate to the Custom Vision project gallery page and select the trash icon under the project.

If you have finished learning, you can delete the resource group from your Azure subscription:

1. In the Azure portal, select Resource groups on the right menu and then select the resource group that you have created.
2. Click Delete resource group.

References:

[1] J, ARUN PANDIAN; GOPAL, GEETHARAMANI (2019), “Data for: Identification of Plant Leaf Diseases Using a 9-layer Deep Convolutional Neural Network”, Mendeley Data, V1, doi: 10.17632/tywbtsjrjv.1

[2] Geetharamani G., Pandian A., Identification of plant leaf diseases using a nine-layer deep convolutional neural network, Computers and Electrical Engineering, 2019, doi: 10.1016/j.compeleceng.2019.04.011