Hosting and Deploying The Tensorflow Model

Upload Your Tensorflow Model

In this lesson, you'll learn how to write a simple algorithm using a pre-trained Tensorflow model we've provided. Before you begin, you may wish to review Creating an Algorithm and Using Hosted Data in Algorithms.

This algorithm shows how to deploy a Tensorflow model for the classification of handwritten digits from image data. Yes, the classic MNIST example that we are all familiar with. If not, check out the example on the Tensorflow docs which this model code is based on.

Note that for any model that you are deploying on your Algorithmia instance, you'll need to train and save the serialized model. For Tensorflow models you can use the tf.saved_model.Builder for the serialization and the tf.saved_model.loader for the deserialization of your model for most Tensorflow versions. For more information and details about saving your model, you can check out our Tensorflow Reference Guides.

Before you get started, you'll need the files located in the GitHub repository associated with this lesson, or you can download the files directly from this lesson page under "Summary" in the left hand column (note it won't show if you're viewing this page at full screen mode). Once you've downloaded the files, go ahead and click the full screen mode icon at the bottom of this page to the right of the "Next" button to better view this lesson.

Now that you have the lesson files downloaded onto your computer, you'll need to upload the zipped model file, and the image files to your data collection. 

Remember how you learned to create data collections in the Algorithmia Hosted Data course? We'll be calling our data collection "tensorflow_mnist_data", but you can name yours as you like.

Once you've created a data collection, you can upload the files: t10k-labels-idx1-ubyte.gzt10k-images-idx3-ubyte.gz from where you stored the files on your computer:

 

Now go ahead and make another data collection called "tensorflow_mnist_model" and upload the model.zip file:

Notice that there are two model files shown, although you'll only have one for the lesson. The "model_1.9.zip" file is an updated model file. When you want to update your model, all you need to do add your new model file to your data collection or other data source such as Amazon S3, and append a version number to it, then publish a new algorithm that is pointing to that new model path. Then if you want to compare your models, you can call each different version number and use our Evaluations Tool

Take note of the path created that starts with "data://" and shows your username and the data collection name along with your file name:

"data://demo/tensorflow_mnist_model/model.zip"

You'll want to use this path in your algorithm in order to point to your own data and model path, so we recommend keeping this data collection page open, and open a new tab where you can create your algorithm. This way you can easily copy and paste the paths from your data collections when you're ready to add them to the demo code sample.

Now go ahead and click the "Plus" icon in the navigation, and create your Tensorflow algorithm, naming it as you like. You've already learned how to create an algorithm in a previous course so we won't go through the steps here, but note that you'll want to choose: "Python3.x Beta" and "Python 3.6 + Tensorflow GPU 1.12" as shown here:

Remember from the Editing Your Algorithm lesson, once you create your algorithm you'll be able to edit it either through the CLI tools or the Web IDE. It's your choice how you want to interact with your algorithm, but this lesson will show working in the Web IDE:

Because we used a Tensorflow package set, and our mnist algorithm doesn't require any other dependencies downloaded from PyPi. So for this algorithm, we don't need to add anything here since Tensorflow GPU 1.12, is already added to our dependency file.

Now you'll want to remove the boilerplate code that exists in your newly created algorithm and copy and paste the code in "demo.py" that is found in either this lesson under the "Summary" section or the GitHub Repository. If you need to exit full screen mode to view the file, click on the icon to the right of the "Next" button on the bottom right of your screen.

Now go ahead and click "Save" in the upper right of the IDE.

Here in our main algorithm source code called "TensorflowDemoGPU" (you might have named it differently when you created your algorithm), notice in the first few lines of our script, we are importing the Python packages required by our algorithm. Then on line 14 we are creating the variable "client" in global scope to use throughout our algorithm. This will enable us to access our data in data collections via the Data API.

Before we go through the rest of our code in our main module, notice that there is a "loadmnistdata.py" file as well under the "Summary" section in this lesson. Again, you can also access that code in the GitHub Repository as well.

For the code in the "loadmnistdata.py" file, you'll need to create a new file under your algorithm directory in the grey section to the left of the code portion of the IDE. See the "+ NEW FILE"? Go ahead and click that and we'll call our new Python module: "loadmnistdata.py".

Now copy and paste the code from the "loadmnistdata.py" file that has been provided, and click the blue "Save" button at the top right of the IDE. We won't go through the code in this file, just note that it handles the pre-processing of the MNIST data stored in our data collections.

Going back to our main module "TensorflowGPUDemo", you'll see on line 16 there is a "load_data()" function, where we'll be using the data api to load our data from the data collection where we stored the images earlier:

Then if you go to line 44 there is the "extract_model()" function, where we unpack our zipped model file from our data collections. Notice that we are making unpacking our files into the "/tmp/" directory.:

Here on line 63, we create a function called "generate_gpu_config()":

This is because Tensorflow allocates all available GPU memory to your algorithm, so you'll want to control how much you allow your model to use.

For more information check out our Tensorflow Reference Guides or the Tensorflow Docs.

Next, we'll load our model on line 88 and create our Tensorflow session:

This is important to note because you'll always want to load the model outside of the "apply()" function.

This is so the model file only gets loaded into memory during the initial call within that session, so while the first call to your algorithm might take a bit of time depending on the size of your model, subsequent calls will be much faster. If you were to load your model inside the "apply()" function, then the model would get loaded with each call of your algorithm.

Also, if you are tempted to add your model file as a Python module, and then import that module into your algorithm file, this will result in a loss of performance and we don't recommend it.

In the function "create_session()" notice that we are pointing to our graph which is in the "/tmp/" folder where we unpacked it earlier. We pass in the "generate_gpu_config()" function with the fraction set to "0.6" to the "tf.Session()" function, and then then load the model using the "tf.saved_model.loader()" function within the session. 

Next, we come to the "predict()" function starting on line 90, where we use our model to predict classes on our test images. We return the predicted value and the accuracy of the model:

Now we come to the all important "apply()" function, the last function in our code. This function is required for every algorithm and allows you to chain different algorithms together using the output of one, as the input to another, all as JSON formatted data structures.

The "input" argument is the data or any other input from the user that gets passed into your algorithm. It's important to support inputs from multiple data sources with exception handling, going beyond data collections like shown. You or other users might also for data files hosted in S3, Azure Blobs, or other data sources that we have data connectors for. For a great example of handling multiple types of files, or if you want to see a PyTorch algorithm in action, check out the Open Anomaly Detection algorithm on the Algorithmia Marketplace. You don't need an account to view the algorithm or the docs! 

Now we will want to click the "Build" button on the top right of the web IDE. This will commit our code to a git repository. Every algorithm is backed by a git repository and as you are developing your algorithm, whenever you hit "Build", that will commit your code and return a hash version of your algorithm which you'll see in the Algorithmia console:

You can use that hash version to call your algorithm locally using one of the language clients for testing purposes while you work on perfecting your algorithm.

Note that you'll get a semantic version number once you publish your algorithm.

Now we are ready to test our algorithm. Go back to that data collections that we got our model path from, and copy/paste the path for our csv file into the Algorithmia console (wrapping it in quotes so it's a proper JSON formatted string). The format is on line 18 of the code within the comments and should look like this, replacing your username:

{"mnist_images": "data://YOUR_USERNAME/tensorflow_mnist_data/t10k-images-idx3-ubyte.gz", "mnist_labels": "data://YOUR_USERNAME/tensorflow_mnist_data/t10k-labels-idx1-ubyte.gz" }

Hit return/enter on your keyboard:

 

If you're happy with the results, you can now publish your model, using the input you just tested as your your sample input. Great work!