Updating ML models on an edge node
Assuming that your EI apps have been containerized and assuming that you are using an edge application orchestrator like IBM’s Edge Application Manager to deploy and to maintain those apps, how about keeping your ML models on your edge nodes up to date?
The worst-case scenario would be if your ML models are tightly coupled with your containerized app in a way that each modification of your ML model would require a rebuild of your app container. Imagine that if your app code is considerably larger than your ML model that you need to refresh unnecessarily your complete app/ML model container on all edge nodes affected.
A slightly better approach could be to separate your ML models from your app(s) and encapsulate them into their own containers. That in return would mean that you need to rebuild your ML container(s) each time you’ll have an updated ML model and you then need to redeploy those updated containers like your app container via your edge application manager. You are still facing a certain overhead since you need to build a container around your ML model.
What if you could treat you ML models as files (or a set of files) which you could simply deploy to your edge nodes w/o any additional container overhead? That approach has been implemented by Linux Foundation Edge’s Open Horizon edge manager and its commercial sibling, the IBM Edge Application Manager (IEAM).
In the following section I am using the reference to the Open Horizon components as a synonym to the equivalent IEAM components.
So how does that work?
The Open Horizon edge manager and the related Open Horizon edge agent have implemented a so-called model management system (MMS). Three key components make up that MMS:
- The Cloud Sync Service (CSS) resides on the Open Horizon edge manager hub and uses a mongoDB database to store objects (e.g., ML models). It also maintains the status of all edge nodes.
- The Edge Sync Service (ESS) is part of the Open Horizon edge agent. It provides two major functions: to poll the CSS for any updates on any objects which are relevant for the edge node on which the agent runs on and to provide a local REST API so that any edge application on that node can easily interact with the ESS.
- Objects which are described by their metadata and their content are then used to exchange the actual data/files between the management hub and the edge nodes.
Since those objects can be anything not just ML models in Open Horizon maybe the MMS should be sometimes renamed into Object Management System (OMS)? 😉
Anyway, let’s have a brief look on how to create, publish and consume an object by utilizing Open Horizon’s and IEAM’s MMS…
Step 1: Create a metadata template JSON document for your object by executing the Open Horizon’s agent CLI command
hzn mms object new > myobject_meta.json
Here is how such a metadata JSON template looks like:
{"objectID": "", /* Required: A unique identifier of the object. */"objectType": "", /* Required: The type of the object. */"destinationOrgID": "$HZN_ORG_ID", /* Required: The organization ID
of the object (an object belongs
to exactly one organization). */"destinationID": "", /* The node id (without org prefix) where the
object should be placed. */
/* If omitted the object is sent to all nodes
with the same destinationType. */
/* Delete this field when you are using
destinationPolicy. */"destinationType": "", /* The pattern in use by nodes that should
receive this object. */
/* If omitted (and if destinationsList is
omitted too) the object is broadcast to all
known nodes. */
/* Delete this field when you are using
policy. */"destinationsList": null, /* The list of destinations as an array
of pattern:nodeId pairs that should
receive this object. */
/* If provided, destinationType and
destinationID must be omitted. */
/* Delete this field when you are using
policy. */"destinationPolicy": { /* The policy specification that should be
used to distribute this object. */
/* Delete these fields if the target node is
using a pattern. */"properties": [ /* A list of policy properties that
describe the object. */
{
"name": "",
"value": null,
"type": "" /* Valid types are string, bool, int, float,
list of string (comma separated), version. */
/* Type can be omitted if the type is
discernable from the value, e.g. unquoted
true is boolean. */
}
],"constraints": [ /* A list of constraint expressions of the
form <property name> <operator>
<property value>, separated by boolean
operators AND (&&) or OR (||). */
""
],"services": [ /* The service(s) that will use this object. */
{
"orgID": "", /* The org of the service. */
"serviceName": "", /* The name of the service. */
"arch": "", /* Set to '*' to indcate services of any
hardware architecture. */
"version": "" /* A version range. */
}
]
},"expiration": "", /* A timestamp/date indicating when the
object expires (it is automatically deleted).
The timestamp should be provided in RFC3339
format. */"version": "", /* Arbitrary string value. The value is not
semantically interpreted. The Model Management
System does not keep multiple version of an
object. */"description": "", /* An arbitrary description. */"activationTime": "" /* A timestamp/date as to when this object
should automatically be activated. The
timestamp should be provided in RFC3339
format. */
}
Let’s have a look at a simplified example to make it easier for the reader to follow.
I am borrowing my example from the Open Horizon demo ‘helloMMS’ edge service.
In that demo the metadata JSON file looks like:
{
"objectID": "config.json",
"objectType": "$HZN_DEVICE_ID.hello-mms",
"destinationOrgID": "$HZN_ORG_ID",
"destinationID": "$HZN_DEVICE_ID",
"destinationType": "pattern-${SERVICE_NAME}-$ARCH"
}
Step 2: Here is the content of the actual object (file) config.json which should be eventually synchronized with the edge node:
{
"HW_WHO": "IBM Cloud Engagement Hub"
}
Step 3: Before we send our config.json file to the edge node, let’s make sure that the ibm.hello-mms service runs on that node by executing the following commands on your edge node:
hzn register -p IBM/pattern ibm.hello-mms-$(hzn architecture) -s ibm.hello-mms — serviceorg IBM
Step 4: Now let’s publish the config.json file to the MMS so that the ibm.hello-mms service can eventually access it through the Open Horizon agent’s Edge Sync Service (make sure that the environment variables $HZN_DEVICE_ID and $HZN_ORG_ID have been correctly set):
hzn mms object publish -m object.json -f config.json
Eventually your helloMMS service will pick up the new config.json file through the ESS and will write a different message into the service’s message log.
In my YouTube video below I am using a slightly modified version of the helloMMS demo to demonstrate how easy it is to send updated objects to a running edge application through the IEAM’s MMS.