项目作者: aerospike

项目描述 :
Aerospike database operator for Kubernetes
高级语言: Go
项目地址: git://github.com/aerospike/aerospike-kubernetes-operator.git
创建时间: 2020-05-08T19:46:24Z
项目社区:https://github.com/aerospike/aerospike-kubernetes-operator

开源协议:Apache License 2.0

下载


Aerospike Kubernetes Operator

Overview

The Aerospike Kubernetes Operator automates the deployment and management of Aerospike enterprise clusters on
Kubernetes. The Operator provides a controller that manages
a Custom Resource Definition (CRD)
to extend the Kubernetes API for Aerospike Enterprise clusters. Aerospike cluster deployment and life cycle management
are performed by updating an Aerospike cluster Custom Resource (CR).

Documentation

For full documentation please visit the official documentation.

What features does it provide?

The goal of the Operator is to give you the ability to deploy multi-node Aerospike clusters, recover automatically from
node failures, scale up or down automatically as load changes, ensure nodes are evenly split across racks or zones,
automatically update to new versions of Aerospike and manage configuration changes in your clusters.

The Operator supports the following capabilities:

  • Deploy Aerospike clusters
  • Scale up and down existing Aerospike clusters
  • Version upgrade and downgrade
  • Configure persistent storage and resource allocation
  • Standardize and validate configurations
  • Cluster security management
  • Aerospike cluster monitoring
  • Backup and restore Aerospike clusters

Building and quick start

Generate CRD manifests

  1. make generate
  2. make manifests

Build and push operator image

Run the following command with the appropriate name and version for the operator’s image.

  1. IMAGE_TAG_BASE=aerospike/aerospike-kubernetes-operator-nightly
  2. VERSION=4.0.0
  3. make docker-buildx IMG=${IMAGE_TAG_BASE}:${VERSION} PLATFORMS=linux/amd64

Note: Change PLATFORMS var as per host machine or remove it to build multi-arch image

Developer testing

You can use the following for quickly trying out the operator without
using OLM.

Deploy

Make sure cert-manager is deployed on your Kubernetes cluster using
instructions here.

To deploy the operator build in the previous step run

  1. make deploy IMG=${IMAGE_TAG_BASE}:${VERSION}

Undeploy

To undeploy run

  1. make undeploy IMG=${IMAGE_TAG_BASE}:${VERSION}

Note: This will also delete the deployed Aerospike clusters because the CRD definitions and all operator related
objects are deleted.

Operator Lifecycle Manager (OLM) integration bundle

Operator Lifecycle Manager (OLM) is a tool to help manage the Operators running on your cluster. This is the preferred
way to manage Kubernetes operators in production. This section describes how to generate the OLM bundle and run the
operator using OLM.

Install operator-sdk

Install operator-sdk version 1.39.1 using the
installation guide

Build the bundle

Make sure the operator’s image has also been pushed.

Set up the environment with image names.

  1. export ACCOUNT=aerospike
  2. export IMAGE_TAG_BASE=${ACCOUNT}/aerospike-kubernetes-operator
  3. export VERSION=4.0.0
  4. export IMG=docker.io/${IMAGE_TAG_BASE}-nightly:${VERSION}
  5. export BUNDLE_IMG=docker.io/${IMAGE_TAG_BASE}-bundle-nightly:${VERSION}
  6. export CATALOG_IMG=docker.io/${IMAGE_TAG_BASE}-catalog-nightly:${VERSION}

Create the bundle

  1. make bundle

Build bundle image and publish

  1. make bundle-build bundle-push

Build catalog image and publish

  1. make docker-buildx-catalog

Deploy operator with OLM

Install OLM if not already done

  1. operator-sdk olm install

Create aerospike namespace if it does not exist

  1. kubectl create namespace aerospike

Deploy the operator using custom CatalogSource:

Deploy custom CatalogSource:

  1. kubectl apply -f - <<EOF
  2. apiVersion: operators.coreos.com/v1alpha1
  3. kind: CatalogSource
  4. metadata:
  5. name: aerospike
  6. namespace: aerospike
  7. spec:
  8. displayName: Aerospike operator
  9. publisher: Aerospike operator
  10. sourceType: grpc
  11. image: "${CATALOG_IMG}"
  12. updateStrategy:
  13. registryPoll:
  14. interval: 10m
  15. EOF

Create Operator Group for the targeted namespaces:

  • Targeting single namespace

    1. kubectl apply -f - <<EOF
    2. apiVersion: operators.coreos.com/v1
    3. kind: OperatorGroup
    4. metadata:
    5. name: test-operator-group
    6. namespace: test
    7. spec:
    8. targetNamespaces:
    9. - aerospike
    10. upgradeStrategy: Default
    11. EOF
  • Targeting multiple namespaces
    Assuming you want the operator to target two other namespaces ns1 and ns2, create operator group with MultiNamespace install mode.

    1. ```shell
    2. kubectl apply -f - <<EOF
    3. apiVersion: operators.coreos.com/v1
    4. kind: OperatorGroup
    5. metadata:
    6. name: test-operator-group
    7. namespace: test
    8. spec:
    9. targetNamespaces:
    10. - ns1
    11. - ns2
    12. upgradeStrategy: Default
    13. EOF

Create Subscription to deploy operator:

  1. kubectl apply -f - <<EOF
  2. apiVersion: operators.coreos.com/v1alpha1
  3. kind: Subscription
  4. metadata:
  5. name: aerospike-kubernetes-operator
  6. namespace: test
  7. spec:
  8. channel: stable
  9. installPlanApproval: Automatic
  10. name: aerospike-kubernetes-operator
  11. source: aerospike
  12. sourceNamespace: aerospike
  13. EOF

Deploy your Aerospike clusters

Deploy Aerospike clusters using the Operator
documentation here.

Undeploy operator with OLM

  1. operator-sdk cleanup aerospike-kubernetes-operator --namespace=aerospike

Running tests

The operator tests require the following prerequisites

  • A running Kubernetes cluster with at least 3 nodes with at least 12 CPUs
  • A storage class named “ssd” that allows provisioning of filesystem and block devices
  • The kubectl command configured to use the above cluster
  • OLM bundle image created with as described here
  • No production services should be running on this cluster - including the operator or production Aerospike clusters

The operator tests create and use 4 namespaces

  • test
  • test1
  • test2
  • aerospike

Run the entire test suite

  1. ./test/test.sh -b $BUNDLE_IMG -c $CATALOG_IMG

Run tests matching a regex

  1. ./test/test.sh -b $BUNDLE_IMG -c $CATALOG_IMG '-ginkgo.focus=".*MultiCluster.*"'

Architecture

The Aerospike Kubernetes Operator has a custom controller (written in go) that allows us to embed specific lifecycle
management logic to effectively manage the state of an Aerospike cluster. It does so by managing a Custom Resource
Definition (CRD) to extend the Kubernetes API for Aerospike clusters. Regular maintenance to the Aerospike cluster
deployment and management can be performed by updating an Aerospike cluster Custom Resource (CR).

The Operator is deployed with StatefulSet and operates as a headless service to handle the DNS resolution of pods in the
deployment. Kubernetes StatefulSets is the workload API object that is used to manage stateful applications. It is
important because it manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and
uniqueness of these Pods (e.g. as unique addressable identities).

A layered approach is taken to orchestration which allows the Operator to manage Aerospike Cluster tasks outside the
Aerospike deployment.

See also