Deploying TensorFlow ML model using Flask for Beginners
This tutorial explains how to deploy a machine learning model using Flask web framework.In this tutorial we will cover following topics to deploy a sentence similarity model using Flask.
- Downloading and setting up TensorFlow pre-trained model from TensorFlow Hub
- Generating embeddings with pre-trained TensorFlow model and calculate cosine similarity
- Setting up Flask Application
- Integrating TensorFlow model with Flask Application
Before diving deep into coding and set up lets have a look to some the terms used above.
What is TensorFlow Hub
TensorFlow Hub is a repository of trained machine learning models ready for fine-tuning and deployable anywhere. TensorFlow Hub provides many pre-trained model, in this tutorial we will use a pre-trained model to generate sentence embeddings. Checkout the models provided by TensorFlow hub here.
What is cosine similarity
Cosine similarity measures the similarity between vectors by calculating the cosine angle between the two vectors. TensorFlow provides tf.keras.losses.cosine_similarity function to compute cosine similarity between labels and predictions.Cosine similarity is a number number between -1 and 1.Cosine values closer to -1 indicate greater similarity and values closer to 1 indicate greater dissimilarity.
What is Flask Framework
Flask is a micro web framework written in Python. It is classified as a microframework because it does not require particular tools or libraries.Flask supports extensions that can add application features as if they were implemented in Flask itself.
Now we know about the tools and framework used in this tutorial, lets gets started with the coding part.
Deploying ML model with Flask
Install required libraries
pip install tensorflow
pip install tensorflow-hub
pip install flask
Using pre-trained model from TensorFlow Hub
1. Download,set up and test pre-trained model from TensorFlow Hub.
- Download nnlm-en-dim50 model for English language from here
- Create a new directory flask-ml and open it with Visual Studio Code.
- Copy the downloaded model inside flask-ml directory and extract the zip file.
- Create a new directory pre_trained_model inside flask-ml and move "assets, variables, saved_model.pb" to pre_trained_model.
After this you should be having below directory structure.
2. Generate embeddings with downloaded model and calculate cosine similarity
Create "test.py" inside flask-ml and write below code.
import tensorflow as tf
import tensorflow_hub as hub
embed = hub.load("./pre_trained_model")
embeddings = embed(["cat is on the floor", "dog is on the floor"])
print(embeddings)
sim = tf.keras.losses.cosine_similarity(embeddings[0], embeddings[1], axis=0)
print(sim)
print(sim.numpy())
- First two lines are importing required libraries.
- Next we are loading the downloaded model with
hub.load - After this we are creating embedding by passing sentences to the
embed - And finally we are calculating cosine similarity, for more information on cosine similarity refer
Kerasdocumentation. For this article it is suffice to know that cosine value near to "-1" indicates greater similarity while value near to "1" indicates higher dissimilarity
Execute above code snippet in test.py, if you are getting output similar to as shown below than the set up is working fine. Now as our model set up is working fine we can delete the test.py file.
tf.Tensor(
[[ 0.16589954 0.0254965 0.1574857 0.17688066 0.02911299 -0.03092718
0.19445257 -0.05709129 -0.08631689 -0.04391516 0.13032274 0.10905275
-0.08515751 0.01056632 -0.17220995 -0.17925954 0.19556305 0.0802278
-0.03247919 -0.49176937 -0.07767699 -0.03160921 -0.13952136 0.05959712
0.06858718 0.22386682 -0.16653948 0.19412343 -0.05491862 0.10997339
-0.15811177 -0.02576607 -0.07910853 -0.258499 -0.04206644 -0.20052543
0.1705603 -0.15314153 0.0039225 -0.28694248 0.02468278 0.11069503
0.03733957 0.01433943 -0.11048374 0.11931834 -0.11552787 -0.11110869
0.02384969 -0.07074881]
[ 0.22075905 0.05648435 0.12771143 0.11615398 0.06471531 0.02444898
0.13712886 -0.06288064 -0.08560256 -0.07659081 0.06722884 0.08384311
-0.06657998 -0.03186746 -0.12207588 -0.19775932 0.13974944 0.03533671
-0.07951755 -0.45845005 -0.0685077 0.08533238 -0.12752111 0.11610293
0.12298352 0.16091679 -0.16929056 0.15839423 -0.07143378 0.09375338
-0.17743316 -0.0500968 -0.06825224 -0.21238105 -0.07613859 -0.19069687
0.1318697 -0.07961286 0.01077813 -0.3535859 0.00888554 0.13555478
0.04420736 0.08488994 -0.01886176 0.17098431 -0.11895745 -0.10908322
0.01940353 -0.00427027]], shape=(2, 50), dtype=float32)
tf.Tensor(-0.95126975, shape=(), dtype=float32)
-0.95126975
Setting up Flask application
1. Create app.py file
Create a new file app.py inside root directory
flask-ml and write below code in app.py.
from flask import Flask, render_template, request, render_template_string
app = Flask(__name__)
@app.route("/")
def index():
return render_template_string("Hello from flask")
if __name__ == "__main__":
app.run(debug=True)
You should see "Hello from flask" in browser on navigating to http://127.0.0.1:5000/
2. Develop front-end using Bootstrap, HTML and CSS
- Create two new directories templates and static inside flask-ml.
- In templates directory create a file index.html
- In static directory create css directory and in css create a file main.css
2. Verify directory structure
After these steps you should have directory structure as show below
3. Prepare index.html
Write below code inside index.html
<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
<!-- <link href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.0/css/bootstrap.min.css" rel="stylesheet" id="bootstrap-css"> -->
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css"
integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
<link href="{{ url_for('static', filename='css/main.css') }}" rel="stylesheet">
</head>
<body>
<div class="container">
<div class="row">
<div class="col-lg-6">
<div class="form_main">
<h4 class="heading"><strong>Sentence </strong> Similarity <span></span></h4>
<div class="form">
<form action="{{ url_for('index', ) }}" method="post" id="contactFrm" name="contactFrm">
<input type="text" required="" placeholder="First Sentence" value="" name="sentence-1"
class="txt">
<input type="text" required="" placeholder="Second Sentence" value="" name="sentence-2"
class="txt">
<input type="submit" value="Check Similarity" name="Sumbit" class="txt2">
</form>
</div>
</div>
</div>
</div>
</div>
<script src="https://code.jquery.com/jquery-3.4.1.slim.min.js"
integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous">
</script>
<script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js"
integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous">
</script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js"
integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous">
</script>
</body>
</html>
4. Prepare main.css
Write below code inside main.css
form_main {
width: 100%;
}
.form_main h4 {
font-family: roboto;
font-size: 20px;
font-weight: 300;
margin-bottom: 15px;
margin-top: 20px;
text-transform: uppercase;
}
.heading {
border-bottom: 1px solid #fcab0e;
padding-bottom: 9px;
position: relative;
}
.heading span {
background: #9e6600 none repeat scroll 0 0;
bottom: -2px;
height: 3px;
left: 0;
position: absolute;
width: 75px;
}
.form {
border-radius: 7px;
padding: 6px;
}
.txt[type="text"] {
border: 1px solid #ccc;
margin: 10px 0;
padding: 10px 0 10px 5px;
width: 100%;
}
.txt_3[type="text"] {
margin: 10px 0 0;
padding: 10px 0 10px 5px;
width: 100%;
}
.txt2[type="submit"] {
background: #242424 none repeat scroll 0 0;
border: 1px solid #4f5c04;
border-radius: 25px;
color: #fff;
font-size: 16px;
font-style: normal;
line-height: 35px;
margin: 10px 0;
padding: 0;
text-transform: uppercase;
width: 30%;
}
.txt2:hover {
background: rgba(0, 0, 0, 0) none repeat scroll 0 0;
color: #5793ef;
transition: all 0.5s ease 0s;
}
.notice {
padding: 15px;
background-color: #fafafa;
border-left: 6px solid #7f7f84;
margin-bottom: 10px;
-webkit-box-shadow: 0 5px 8px -6px rgba(0,0,0,.2);
-moz-box-shadow: 0 5px 8px -6px rgba(0,0,0,.2);
box-shadow: 0 5px 8px -6px rgba(0,0,0,.2);
}
.notice-sm {
padding: 10px;
font-size: 80%;
}
.notice-lg {
padding: 35px;
font-size: large;
}
.notice-success {
border-color: #80D651;
}
.notice-success>strong {
color: #80D651;
}
.notice-info {
border-color: #45ABCD;
}
.notice-info>strong {
color: #45ABCD;
}
.notice-warning {
border-color: #FEAF20;
}
.notice-warning>strong {
color: #FEAF20;
}
.notice-danger {
border-color: #d73814;
}
.notice-danger>strong {
color: #d73814;
}
5. Update app.py
Change the code in app.py as shown below
from flask import Flask, render_template, request, render_template_string
app = Flask(__name__)
@app.route("/")
def index():
return render_template("index.html")
if __name__ == "__main__":
app.run(debug=True)
6. Test the flask application set up.
Run the app.py and you should see below output in browser.
Integrating TensorFlow model with Flask app
1. Create utils.py file
Create a file utils.py inside flask-ml and write below code in utils.py
import tensorflow as tf
import tensorflow_hub as hub
def similarity_value(s1, s2):
embed = hub.load("./pre_trained_model")
embeddings = embed([s1, s2])
sim = tf.keras.losses.cosine_similarity(
embeddings[0], embeddings[1], axis=0)
print(sim.numpy())
if sim.numpy() <= -0.6:
msg = f"Cosine Similarity is {sim.numpy()}, this indicates high similarity"
elif sim.numpy() >= 0.6:
msg = f"Cosine Similarity is {sim.numpy()}, this indicates high dissimilarity"
else:
msg = f"Cosine Similarity is {sim.numpy()}, can't decide much with this value"
return msg
2. Verify directory structure
After this step you should have directory structure as shown below
3. Update app.py
Replace code in app.py with the below code
from flask import Flask, render_template, request, redirect
import utils
app = Flask(__name__)
@app.route("/", methods=['GET', 'POST'])
def index():
if request.method == 'GET':
return render_template("index.html")
if request.method == 'POST':
result = request.form.to_dict(flat=True)
sentence_1 = result.get("sentence-1")
sentence_2 = result.get("sentence-2")
sim_msg = utils.similarity_value(sentence_1, sentence_2)
result["sim_msg"] = sim_msg
return render_template("index.html", result=result)
if __name__ == "__main__":
app.run(debug=True)
4. Update index.html
Replace code in index.html with the below code.
<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
<!-- <link href="//maxcdn.bootstrapcdn.com/bootstrap/3.3.0/css/bootstrap.min.css" rel="stylesheet" id="bootstrap-css"> -->
<link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css"
integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
<link href="{{ url_for('static', filename='css/main.css') }}" rel="stylesheet">
</head>
<body>
<div class="container">
<div class="row">
<div class="col-lg-6">
<div class="form_main">
<h4 class="heading"><strong>Sentence </strong> Similarity <span></span></h4>
<div class="form">
<form action="{{ url_for('index', ) }}" method="post" id="contactFrm" name="contactFrm">
<input type="text" required="" placeholder="First Sentence" value="" name="sentence-1"
class="txt">
<input type="text" required="" placeholder="Second Sentence" value="" name="sentence-2"
class="txt">
<input type="submit" value="Check Similarity" name="Sumbit" class="txt2">
</form>
</div>
</div>
</div>
</div>
{% if result %}
<div class="row">
<div class="col-lg-6">
<div class="form_main">
<h4 class="heading"><strong>Cosine </strong> Similarity Value<span></span></h4>
<div class="notice notice-info">
<strong>Sentence 1</strong> : {{result["sentence-1"]}}
</div>
<div class="notice notice-info">
<strong>Sentence 2</strong> : {{result["sentence-2"]}}
</div>
<div class="notice notice-success">
<strong>Similarity</strong> {{result["sim_msg"]}}
</div>
</div>
</div>
</div> {% endif %}
</div>
<script src="https://code.jquery.com/jquery-3.4.1.slim.min.js"
integrity="sha384-J6qa4849blE2+poT4WnyKhv5vZF5SrPo0iEjwBvKU7imGFAV0wwj1yYfoRSJoZ+n" crossorigin="anonymous">
</script>
<script src="https://cdn.jsdelivr.net/npm/popper.js@1.16.0/dist/umd/popper.min.js"
integrity="sha384-Q6E9RHvbIyZFJoft+2mJbHaEWldlvI9IOYy5n3zV9zzTtmI3UksdQRVvoxMfooAo" crossorigin="anonymous">
</script>
<script src="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/js/bootstrap.min.js"
integrity="sha384-wfSDF2E50Y2D1uUdj0O3uMBJnjuUD4Ih7YwaYd1iqfktj0Uod8GCExl3Og8ifwB6" crossorigin="anonymous">
</script>
</body>
</html>
3. Test the Flask application with TensorFlow Model
Run the app.py, provide input sentences and click on CHECK SIMILARITY, you should see output similar to as
shown below
With this we successfully deployed sentence similarity machine learning model and served using Flask application that is using Bootstrap and Jinja 2 for the front-end. Let us know your feedback in the comment section or any issues you faced while following steps mentioned in this article.
Category: TensorFlow
