For full instructions on how to deploy to Akash, please refer to my previous Guide to Deploying Applications to Akash DeCloud. Basically, you just need to create the following SDL file (name it deploy.yml
), replaceUSERNAME
with your Docker Hub username, and execute a couple of commands as outlined in the deployment guide:
---
version: "2.0"services:
web:
image: USERNAME/mnist-tf-serve:1.0
expose:
- port: 8501
as: 80
to:
- global: true
profiles:
compute:
web:
resources:
cpu:
units: 0.2
memory:
size: 512Mi
storage:
size: 512Mi
placement:
westcoast:
attributes:
organization: ovrclk.com
signedBy:
anyOf:
- "akash1vz375dkt0c60annyp6mkzeejfq0qpyevhseu05"
pricing:
web:
denom: uakt
amount: 1000
deployment:
web:
westcoast:
profile: web
count: 1
Note that we’re exposing the container’s port 8501 (TensorFlow Serving’s REST API port according to their documentation) as port 80.
After deploying, for me, the lease access details are:
{
"services": {
"web": {
"name": "web",
"available": 1,
"total": 1,
"uris": [
"qu10pc89gta87est6uc34g18sc.provider4.akashdev.net"
],
"observed-generation": 0,
"replicas": 0,
"updated-replicas": 0,
"ready-replicas": 0,
"available-replicas": 0
}
},
"forwarded-ports": {}
}
which means my model serving endpoint is at http://qu10pc89gta87est6uc34g18sc.provider4.akashdev.net/v1/models/mnist. If I navigate to that link in my browser, I get the following response:
{
"model_version_status": [
{
"version": "1",
"state": "AVAILABLE",
"status": {
"error_code": "OK",
"error_message": ""
}
}
]
}
You should follow along with whatever link is under the uris
of your lease details.
Let’s query the model using the predict API to verify that it works.
Our CNN model takes as input a grayscale 28 pixels by 28 pixels image represented as a 28 x 28 x 1 dimensional matrix, where each value in the matrix is a grayscale value between 0 and 1. Consider the following (magnified) 28 by 28 pixels grayscale image of the digit 7:
The 28 x 28 x 1 array representation of this image is available in jupyter/test_input_7.txt
of the GitHub repo. We can query the model with this input using the predict API by sending a POST request to our model server:
curl -d '{ "instances": ['"$(cat test_input_7.txt)"'] }' -X POST http://qu10pc89gta87est6uc34g18sc.provider4.akashdev.net/v1/models/mnist:predict
The response contains predictions as an array of probabilities, indexed by the digit classes 0 to 9, e.g.:
{
"predictions": [
[
3.40557971e-10,
9.78066308e-11,
4.42975676e-07,
6.55190945e-07,
2.91424957e-12,
5.24897e-11,
1.64641738e-16,
0.999998331,
1.02023101e-08,
5.57226485e-07
]
]
}
According to the above example response, the model correctly predicts that the image contains a “7” with high confidence (probability 0.999998331 for class “7”, and ~0 for all the other possible digits).