Deploying TensorFlow Models on Flask Part 1 - Set up trained model from TensorFlow Hub

Note: This tutorial has been updated follow the updated version of this tutorial at Deploying TensorFlow ML model using Flask for Beginners

In this tutorial series we will learn how to use trained and ready to deploy machine learning models available in TensorFlow Hub. We will than create a Flask web application for deploying the model. We will utilize trained embedding model from TensorFlow Hub and build a application for sentence similarity.

This is a multi part tutorials series, we will cover end-to-end process in below parts

• Part 1: Download,set up and test trained model from TensorFlow Hub
• Part 2: Set up Flask application
• Part 3: Integrating TensorFlow ML model with Flask application

Part 1: Download,set up and test trained model from TensorFlow Hub

After this you should be having below directory structure.

  
  └───flask-ml
      │   tf2-preview_nnlm-en-dim50_1.tar
      │   tf2-preview_nnlm-en-dim50_1.tar.gz
      │
      └───pre_trained_model
          │   saved_model.pb
          │
          ├───assets
          │       tokens.txt
          │
          └───variables
                  variables.data-00000-of-00001
                  variables.index
  
  

Generate embeddings with model and calculate cosine similiarity

For executing below code TensorFlow2.0 or later version is required, if you don't have tensorflow installed, run pip install tensorflow. Create "test.py" inside flask-ml and write below code. Lets understand code snippet.

• First two lines are importing required libraries.
• Next we are loading the downloaded model with hub.load
• After this we are creating embedding by passing sentences to the embed
• And finally we are calculating cosine similarity, for more information on cosine similarity refer Keras documentation. For this article it is suffice to know that cosine value near to "-1" indicates greater similarity while value near to "1" indicates higher dissimilarity

  import tensorflow as tf
  import tensorflow_hub as hub
  
  embed = hub.load("./pre_trained_model")
  embeddings = embed(["cat is on the floor", "dog is on the floor"])
  print(embeddings)
  
  sim = tf.keras.losses.cosine_similarity(embeddings[0], embeddings[1], axis=0)
  print(sim)
  print(sim.numpy())
  

If you are getting output similar to as shown below than the set up is working fine

  
    (tf2) C:\Users\vikas\Desktop\react_tuts\New folder\flask-ml>C:/ProgramData/Anaconda3/envs/tf2/python.exe "c:/Users/vikas/Desktop/react_tuts/New folder/flask-ml/test.py"
    2020-10-26 13:36:20.132956: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
    tf.Tensor(
    [[ 0.16589954  0.0254965   0.1574857   0.17688066  0.02911299 -0.03092718
       0.19445257 -0.05709129 -0.08631689 -0.04391516  0.13032274  0.10905275
      -0.08515751  0.01056632 -0.17220995 -0.17925954  0.19556305  0.0802278
      -0.03247919 -0.49176937 -0.07767699 -0.03160921 -0.13952136  0.05959712
       0.06858718  0.22386682 -0.16653948  0.19412343 -0.05491862  0.10997339
      -0.15811177 -0.02576607 -0.07910853 -0.258499   -0.04206644 -0.20052543
       0.1705603  -0.15314153  0.0039225  -0.28694248  0.02468278  0.11069503
       0.03733957  0.01433943 -0.11048374  0.11931834 -0.11552787 -0.11110869
       0.02384969 -0.07074881]
     [ 0.22075905  0.05648435  0.12771143  0.11615398  0.06471531  0.02444898
       0.13712886 -0.06288064 -0.08560256 -0.07659081  0.06722884  0.08384311
      -0.06657998 -0.03186746 -0.12207588 -0.19775932  0.13974944  0.03533671
      -0.07951755 -0.45845005 -0.0685077   0.08533238 -0.12752111  0.11610293
       0.12298352  0.16091679 -0.16929056  0.15839423 -0.07143378  0.09375338
      -0.17743316 -0.0500968  -0.06825224 -0.21238105 -0.07613859 -0.19069687
       0.1318697  -0.07961286  0.01077813 -0.3535859   0.00888554  0.13555478
       0.04420736  0.08488994 -0.01886176  0.17098431 -0.11895745 -0.10908322
       0.01940353 -0.00427027]], shape=(2, 50), dtype=float32)
    tf.Tensor(-0.95126975, shape=(), dtype=float32)
    -0.95126975
  

In the next part 2 we will set up Flask application for serving ML model