Build an AI web app with Azure Cognitive Services and Flask

Welcome to part 3 of the “Digitize and translate your notes with Azure Cognitive Services and Python” series. In the previous posts, you explored the READ API and the Translator service and created a Python app that extracts and translates handwritten notes.

To find out more about previous posts, check out the links below:

In this blog post, you will build a website using Flask and Azure Cognitive Services to extract and translate text from notes. You will learn how to:

  • Create a simple website using Flask.
  • Use Flask to build a form.
  • Use Azure Computer Vision and Translator to extract and translate text.

To complete the exercise, you will need to install:

  • Python 3, Flask, and
  • Visual Studio Code.

Create a Flask app

To start, we will create the landing page for our website, which will display a form to the user.

Create an app

Create a new Python file named app.py inside your app’s folder.

Add the following code to create a plain Flask app.

1
2

from flask import Flask, request, render_template
app = Flask(__name__)

Add the route

Our application will use only the default route (/). Add the following code at the end of the app.py script to define the default route:

1
2
3

@app.route('/', methods=['GET'])
def index():
    return render_template('index.html')

Create the landing page

Create a new folder named templates and a new HTML file for the landing page of our website (index.html). Add the following HTML, which creates a simple form.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename = 'style.css') }}">
    <meta name="description" content="Digitize and Translate your notes with Azure Cognitive Services and Python.">
    <title>Digitize and Translate your notes</title>
</head>
<body>
    <div class="container">
        <div class="header">
            <h1 id="title">Digitize and Translate your notes with Azure Cognitive Services</h1>
            <p id="description">Enter the URL of your image (handwritten notes), choose a language, and click Translate!</p>
        </div>

        <div>
            <form method="POST" id="form">
                <div class="form-group">
                    <label for="image">Handwritten notes:</label>
                    <input type="url" id="image" name="image" class="form-control" placeholder="Enter the URL of your image" required>
                </div>
                <div class="form-group">
                    <label for="language">Language:</label>
                    <select id="nagage" name="language" class="form-control">
                        <option value="en">English</option>
                        <option value="el">Greek</option>
                        <option value="fr">French</option>
                        <option value="it">Italian</option>
                        <option value="de">German</option>
                        <option value="ja">Japanese</option>
                    </select>
                </div>
                <div class="form-group">
                    <button type="submit" class="submit-button">Translate!</button>
                </div>
            </form>
        </div>
    </div>
    <div class="footer">
        <p>Created by Foteini Savvidou</p>
    </div>
</body>
</html>

The users perform the following tasks:

  1. Enter the URL of image that they wish to translate in the input field.
  2. Select the target language from the dropdown list (select).

If you want to make your app prettier, you can download the style.css file from my GitHub repository.

Test your app

Run the application using flask run and you should see the form displayed.

Call the READ API and the Translator service

In this section, you will use the functions that you have created in the previous articles to call the READ API and the Translator service and create a template to display the results.

Define functions for text extraction and translation

  1. Define the get_text() function which extracts text from an online image. The URL of the image is given as parameter. (If you don’t understand this code, you can read the first post of this series.)

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

    def get_text(image_url, computervision_client):
    # Open local image file
    read_response = computervision_client.read(image_url, raw=True)
    # Get the operation location (URL with an ID at the end)
    read_operation_location = read_response.headers["Operation-Location"]
    # Grab the ID from the URL
    operation_id = read_operation_location.split("/")[-1]
    # Retrieve the results
    while True:
        read_result = computervision_client.get_read_result(operation_id)
        if read_result.status.lower() not in ["notstarted", "running"]:
            break
        time.sleep(1)
    # Get the detected text
    text = ""
    if read_result.status == OperationStatusCodes.succeeded:
        for page in read_result.analyze_result.read_results:
            for line in page.lines:
                # Get text in each detected line and do some fixes to the structure
                if (not text) or text[-1].strip() == "." or text[-1].strip() == ":":
                    text = text + "\n" + line.text
                else:
                    text = text + " " + line.text
    text = text.replace(" .", ".").replace(" ,", ",").replace(" :", ":")
    return text

  2. Define the detect_language() function which uses the Translator service to automatically detect the language of the given text.

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

    def detect_language(text, key, region, endpoint):
    # Use the Translator detect function
    path = "/detect"
    url = endpoint + path
    # Build the request
    params = {
        "api-version": "3.0"
    }
    headers = {
    "Ocp-Apim-Subscription-Key": key,
    "Ocp-Apim-Subscription-Region": region,
    "Content-type": "application/json"
    }
    body = [{
        "text": text
    }]
    # Send the request and get response
    request = requests.post(url, params=params, headers=headers, json=body)
    response = request.json()
    # Get language
    language = response[0]["language"]
    # Return the language
    return language

  3. Define the translate() function which translates the given text using the Translator service. (If you don’t understand this code, you can read the second post of this series.)

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

    def translate(text, source_language, target_language, key, region, endpoint):
    # Use the Translator translate function
    url = endpoint + "/translate"
    # Build the request
    params = {
        "api-version": "3.0",
        "from": source_language,
        "to": target_language
    }
    headers = {
        "Ocp-Apim-Subscription-Key": key,
        "Ocp-Apim-Subscription-Region": region,
        "Content-type": "application/json"
    }
    body = [{
        "text": text
    }]
    # Send the request and get response
    request = requests.post(url, params=params, headers=headers, json=body)
    response = request.json()
    # Get translation
    translation = response[0]["translations"][0]["text"]
    # Return the translation
    return translation

Add code to call the services

  1. At the top of app.py add the following code to import the necessary libraries and load the values from .env.

    1
2
3
4
5
6
7

    from flask import Flask, request, render_template
import requests, os, time
from azure.cognitiveservices.vision.computervision import ComputerVisionClient
from msrest.authentication import CognitiveServicesCredentials
from azure.cognitiveservices.vision.computervision.models import OperationStatusCodes
from dotenv import load_dotenv
load_dotenv()

  2. At the bottom of the app.py, add the following code which

    • reads the URL of image the user submitted on the form and the target language they selected,
    • loads the values from .env and creates a ComputerVisionClient object,
    • calls the functions that you defined in the previous section to extract text from the submitted image and translate the extracted text into the selected language,
    • calls the render_template to display the response page.
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

    @app.route('/', methods=['POST'])
def index_post():
    # Read the values from the form
    image_url = request.form['image']
    target_language = request.form['language']

    # Load the values from .env
    key = os.getenv("COG_SERVICE_KEY")
    region = os.getenv("COG_SERVICE_REGION")
    endpoint = os.getenv("ENDPOINT")
    COG_endpoint = os.getenv("COG_SERVICE_ENDPOINT")

    # Authenticate Computer Vision client
    computervision_client = ComputerVisionClient(COG_endpoint, CognitiveServicesCredentials(key))
    # Extract text
    text = get_text(image_url, computervision_client)

    # Detect language
    language = detect_language(text, key, region, endpoint)
    # Translate text
    translated_text = translate(text, language, target_language, key, region, endpoint)
    # Call render template
    return render_template(
        'results.html',
        translated_text=translated_text,
        original_text=text,
        target_language=target_language
    )

Create a template to display the results

Create a new file named results.html in the templates folder and add the following HTML:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename = 'style.css') }}">
    <meta name="description" content="Digitize and Translate your notes with Azure Cognitive Services and Python.">
    <title>Digitize and Translate your notes</title>
</head>
<body>
    <div class="container">
        <div class="header">
            <h1 id="title">Digitize and Translate your notes with Azure Cognitive Services</h1>
            <h2 id="description">Results</h2>
        </div>
        <div class="results">
            <div>
                <p><strong>Original text:</strong> </p>
                <p>{{ original_text }}</p>
            </div>
            <div>
                <p><strong>Translated text:</strong></p>
                <p>{{ translated_text }}</p>
            </div>
            <div>
                <p><strong>Target language code:</strong> {{ target_language }}</p>
            </div>
            <div>
                <a href="{{ url_for('index') }}">Try another one!</a>
            </div>
        </div>
    </div>
    <div class="footer">
        <p>Created by Foteini Savvidou</p>
    </div>
</body>
</html>

In the example above, we used the {{ }} to print content passed as parameter in the render_template.

Test your app

Run the application using flask run and submit an image for analysis. Select Translate and you should see the results.

Summary and next steps

In this article, you built an AI web app using Flask and Azure Cognitive Services. In the last article of this series, you will learn how to deploy your app to Azure and protect your Azure Cognitive Services keys.

Check out the other parts of the “Digitize and translate your notes with Azure Cognitive Services and Python” series:

Clean-up

If you have finished learning, you can delete the resource group from your Azure subscription:

  1. In the Azure Portal, select Resource groups on the right menu and then select the resource group that you have created.

  2. Click Delete resource group.

Foteini Savvidou

Written By

Foteini Savvidou

I am an Electrical and Computer Engineering student at Aristotle University of Thessaloniki, Greece, interested in AI, IoT, Wireless Communications, and Cloud Technologies, as well as in applications of technology in healthcare and education. I am also Microsoft AI MVP and Gold Microsoft Learn Student Ambassador.

You May Also Like