云鉴权>> charm-tf-serving>> 返回
项目作者： canonical

项目描述 ：
Charmed TF Serving
高级语言： Python
项目地址: git://github.com/canonical/charm-tf-serving.git
创建时间： 2020-06-19T15:20:10Z
项目社区：https://github.com/canonical/charm-tf-serving
开源协议：
下载


Charmed TensorFlow Serving

Deploying

To deploy TensorFlow Serving with a model, start by deploying the charm. For a single model, it
will look something like this:

  1. juju deploy cs:~kubeflow-charmers/tf-serving \
  2.     --storage models=$NAME_OF_STORAGE_CLASS,, \
  3.     --config model-name=/path/to/base/dir/model-name
  4.     --config model-base-path=relative/path/to/model

For multiple models, you can specify a configuration file like this:

  1. juju deploy cs:~kubeflow-charmers/tf-serving \
  2.     --storage models=$NAME_OF_STORAGE_CLASS,,  \
  3.     --config model-conf=/path/to/model.conf

For both of these, change $NAME_OF_STORAGE_CLASS to the name of a storage class available on
the Kubernetes cluster that you’re deploying this charm to. To list available storage classes,
you can run juju list-storage-pools. For example, you would replace $NAME_OF_STORAGE_CLASS
with kubernetes with the below listing:

  1. $ juju list-storage-pools
  2. Name        Provider    Attrs
  3. kubernetes  kubernetes

Next, you will need to copy the files onto the storage device backing the workload pod. How you
do this will vary by type of storage class used. For a simple example, see the full MicroK8s
example below.

Logging

You can control the verbosity of the TF Serving logs by setting the tf-logging-level config option
to a value between 0 and 3, inclusive. For example, you could set the logging level as:

  1. juju config tf-serving tf-logging-level=3

MicroK8s example

To start, clone this git repository locally https://github.com/tensorflow/serving:

  1. git clone https://github.com/tensorflow/serving.git

It has example models that we’ll be deploying down below.

Then, ensure that you’ve enabled storage in MicroK8s:

  1. microk8s.enable storage

Next, deploy the charm:

  1. juju deploy cs:~kubeflow-charmers/tf-serving \
  2.     --storage models=kubernetes,, \
  3.     --config model-name=saved_model_half_plus_two_cpu \
  4.     --config model-base-path=/models/testdata/saved_model_half_plus_two_cpu

Note: In further steps, you will copy the model from your local into the tf-serving pod. Keep in mind that the storage
will be mounted in /models, thus your model-base-path should be appended with /models, as seen previously.

You can use any of the models stored in the repository under
tensorflow_serving/servables/tensorflow/testdata/, instead of saved_model_half_plus_two_cpu
if you’d like, just deploy the charm with model-name and model-base-path configured
appropriately.

Next you’ll need to load the models into the pod. In MicroK8s, you can do this easily by
copying files to the pod’s volume located under the default storage location of
/var/snap/microk8s/common/default-storage/. The volume will have a name in the form of
$NAMESPACE-$PVC-pvc-*. You should substitute in the name of the model that you
deployed the charm to for $NAMESPACE, and $PVC can be found with this command:

  1. microk8s.kubectl get pods -n $NAMESPACE tf-serving-0 -o=jsonpath="{.spec.volumes[0].persistentVolumeClaim.claimName}"

So, to copy over the files that you cloned earlier, you can run a command that looks like this:

  1. cp -r serving/tensorflow_serving/servables/tensorflow/testdata /var/snap/microk8s/common/default-storage/kubeflow-models-12345678-tf-serving-0-pvc-1234-5678/

TensorFlow Serving should then see the files and start serving them. You can contact TensorFlow
Serving by getting the IP address of the associated Service:

  1. $ microk8s.kubectl get -n kubeflow service/tf-serving -o=jsonpath='{.spec.clusterIP}'
  2. 10.152.183.131

And then contacting it via that address:

  1. # Check on the status of the model
  2. $ curl http://10.152.183.131:9001/v1/models/saved_model_half_plus_two_cpu
  3. {
  4.  "model_version_status": [
  5.   {
  6.    "version": "123",
  7.    "state": "AVAILABLE",
  8.    "status": {
  9.     "error_code": "OK",
  10.     "error_message": ""
  11.    }
  12.   }
  13.  ]
  14. }
  16. # Use the model to predict some data points
  17. $ curl http://10.152.183.131:9001/v1/models/saved_model_half_plus_two_cpu:predict -d '{"instances": [1, 2, 3]}'
  18. {
  19.     "predictions": [2.5, 3.0, 3.5]
  20. }

General Kubernetes example

If you’re using a Kubernetes cluster that doesn’t support simply copying files over, you can
deploy this charm as above, and then copy the files manually with kubectl cp:

  1. # Start by cloning the example serving artifacts
  2. git clone https://github.com/tensorflow/serving.git
  4. # Ensure that you have your `kubeconfig` set up properly. For example, with Charmed Kubernetes:
  5. juju scp -m default kubernetes-master/0:~/config ~/.kube/config
  7. # Then copy the files to the tf-serving pod
  8. kubectl cp -n kubeflow serving/tensorflow_serving/servables/tensorflow/testdata tf-serving-0:/models/

After you’ve copied the files over, you can interact with tf-serving by port forwarding:

  1. kubectl port-forward -n kubeflow service/tf-serving 9001:9001

TensorFlow Serving will then be available at localhost:

  1. $ curl http://localhost:9001/v1/models/saved_model_half_plus_two_cpu:predict -d '{"instances": [1, 2, 3]}'
  2. {
  3.     "predictions": [2.5, 3.0, 3.5]
  4. }

S3 Example

You can also deploy TF Serving with a model that’s hosted on S3 instead of in the container itself.
For example, if you have a model at s3://YOURBUCKET/testdata/saved_model_half_plus_two_cpu, you
would deploy this charm like this:

  1. juju deploy cs:~kubeflow-charmers/tf-serving \
  2.     --config model-name=saved_model_half_plus_two_cpu \
  3.     --config model-base-path=s3://YOURBUCKET/testdata \
  4.     --config aws-access-key-id=... \
  5.     --config aws-region=us-east-1 \
  6.     --config aws-secret-access-key=... \
  7.     --config s3-endpoint=x.x.x.x