Analyze receipts with Form Recognizer

Azure Form Recognizer is an Azure Applied AI service that lets you easily analyze documents and extract text fields, data and tables and identify key-value pairs. In this article, you will learn how to extract common fields from sales receipts using the Azure Form Recognizer client library for Python.

You will learn how to:

• Provision a Form Recognizer resource.
• Use the Form Recognizer resource and the Python SDK to extract information from receipts.

To complete the exercise, you will need to install:

• Python 3,
• Visual Studio code or another code editor.

You will also need an Azure subscription. If you don’t have one, you can sign up for an Azure free account.

What is the Form Recognizer?

Form Recognizer is one of Azure’s cloud-based artificial intelligence services that can be used to build applications for automatic document analysis and information extraction. Form Recognizer uses deep learning models to analyze document data, extract text and values, and map relationships between data fields. Using the Form Recognizer API, you can automate the process of analyzing documents and extracting data at scale with high accuracy.

Azure Form Recognizer provides numerous pre-built models for analyzing common documents, such as invoices, receipts, and business cards, while you can also build custom models to analyze documents specific to your business.

In this article, we will be focusing on analyzing common sales receipts using a pre-trained receipt model.

Create a Form Recognizer Resource

To use the Azure Form Recognizer service, you can either create a Form Recognizer resource or a Cognitive Services resource.

In this exercise, you will create a single Form Recognizer resource. If you plan to use Form Recognizer along with other cognitive services, such as Computer Vision, you can create a Cognitive Services resource, or else you can create a Computer Vision resource.

1. Sign in to Azure Portal and select Create a resource.

2. Search for Form Recognizer and then click Create.

3. Create a Form Recognizer resource with the following settings:

   • Subscription: Your Azure subscription.
   • Resource group: Select an existing resource group or create a new one.
   • Region: Choose any available region, for example, North Europe.
   • Name: This would be your custom domain name in your endpoint. Enter a unique name.
   • Pricing tier: Free F0.
   

4. Select Review + Create and then select the Create button and wait for the deployment to complete.

5. Once the resource has been deployed, select Go to resource.

Get Endpoint and Keys

Navigate to the Keys and Endpoint page. Save the Key 1 and the Endpoint. You’ll need these values to connect your Python application to the Form Recognizer API.

Set up your application

Install the client library

To analyze documents with Form Recognizer, you’ll need to install the Form Recognizer client library for Python. You can install the latest version of the client library using pip:

1

pip install azure-ai-formrecognizer

Create a configuration file

Create a configuration file (.env) and save the Azure’s key and endpoint you copied in the previous step.

Create a new Python application

1. Create a new Python file (form-recognizer.py) and import the following libraries:

   1 2 3 4

   from azure.ai.formrecognizer import FormRecognizerClient from azure.core.credentials import AzureKeyCredential from dotenv import load_dotenv import os

2. Add the following code to load the key and endpoint from the configuration file.

   1 2 3

   load_dotenv() key = os.getenv('KEY') endpoint = os.getenv('ENDPOINT')

Authenticate the client

To interact with the Form Recognizer resource, create a FormRecognizerClient. Using the methods of the FormRecognizerClient, you will be able to recognize common fields from receipts.

The following code creates a FormRecognizerClient using the key and endpoint variables you defined above.

1

form_recognizer_client = FormRecognizerClient(endpoint, AzureKeyCredential(key))

Analyze receipts

In this article, we will be focusing on analyzing and extracting common fields and semantic values from sales receipts.

Analyze the given receipt

Insert the following code after the form recognizer client creation to analyze a receipt from a URL. This code calls the begin_recognize_receipts_from_url method to extract values from the receipt and then retrieves the result. The input parameter of the above-mentioned method must be the URL of the receipt to be analyzed.

1
2
3
4
5
6
7
8
9

# Specify the URL of the receipt you will be recognizing
receiptUrl = "<ADD_YOUR_URL>"

# Start the recognition
poller = form_recognizer_client.begin_recognize_receipts_from_url(receiptUrl)
results = poller.result()

# Get the first receipt in the results (we have only one!)
receipt = results[0].fields

Display general information

Add the following code to print general information about the given receipt, such as the receipt type and the transaction date.

1
2
3
4
5

# General information
receipt_type = receipt.get("ReceiptType")
print(f"Receipt Type: {receipt_type.value} (confidence {receipt_type.confidence*100 :.2f}%)")
receipt_date = receipt.get("TransactionDate")
print(f"Transaction Date: {receipt_date.value} (confidence {receipt_date.confidence*100 :.2f}%)")

Display receipt items

The following code displays information about the items (such as item name, price, and quantity along with their confidence score) recognized by the pre-trained receipt model.

1
2
3
4
5
6

# Receipt items
print("Receipt Items:")
for idx, items in enumerate(receipt.get("Items").value):
    print("...Item #{}".format(idx + 1))
    for item_name, item in items.value.items():
        print(f"......{item_name}: {item.value} (confidence {item.confidence*100 :.2f}%)")

Display the total price

Use the following code to print the total tax and the transaction total and subtotal along with the confidence score.

1
2
3
4
5
6
7

# Total price
subtotal = receipt.get("Subtotal")
print(f"Subtotal: {subtotal.value} (confidence {subtotal.confidence*100 :.2f}%)")
tax = receipt.get("Tax")
print(f"Tax: {tax.value} (confidence {tax.confidence*100 :.2f}%)")
total = receipt.get("Total")
print(f"Total: {total.value} (confidence {total.confidence*100 :.2f}%)")

Summary and next steps

In this article, you learned how to analyze sales receipts using the Form Recognizer service and the client library for Python. Here are some additional resources from Microsoft Learn:

• Analyze receipts with the Form Recognizer service
• Extract data from forms with Form Recognizer

Clean-up

If you have finished learning, you can delete the resource group from your Azure subscription:

1. In the Azure portal, select Resource groups on the right menu and then select the resource group that you have created.
2. Click Delete resource group.