第6章编写Operator

到目前为止，在第一章“控制器和operator”我们从概念上了解了自定义控制器和operator，在第5章中，我们讨论了如何使用Kubernetes代码生成器 - 一种在低层次处理这个问题的方法。在本章中，我们将介绍三个开发自定义控制器和操作员的方案，并介绍更多的可选方案。

使用本章中讨论的3个解决方案中任何一个都会帮助您避免编写大量重复代码，并使您能够专注于业务逻辑，而不是框架代码。它应该会让您更快地上手并提高您的工作效率。

注意

目前（2019年中期），我们在本章中讨论到的operator开发工具仍在迅速发展。您在此处看到的某些命令或输出可能与您在使用时看到的不完全相同。基于这点考虑，我们建议您始终使用相应工具的最新版本，密切关注相应的问题跟踪，邮件列表和Slack通道，寻找和解决您遇到的问题。

对于本文讨论的解决方案，网上已经有一些比较。这里我们不会向您推荐具体的解决方案，但是我们鼓励您自己评估和比较它们，并选择最适合您的那个。

准备

我们将使用cnat（云原生的at示例，详见“示例” ）作为本章中不同解决方案的示例。如果你想使用这个列子，请检查您的环境是否具备：

安装了Go1.12或更高版本并正确设置。
能够正常访问Kubernetes1.12或以上版本的集群并且正确配置了kubectl，能够访问（可以通过kind或k3d本机搭建集群，也可以通过远程访问云服务提供商的集群）
git clone我们提供的GitHub存储库。这个代码是完整、可用的，可以结合下文介绍的命令来操作。请注意，我们将会从头开始演示。如果您想查看结果，可以直接克隆存储库并执行命令来安装CRD，安装CR并启动自定义控制器。

做完准备工作，让我们开始编写operator：我们将介绍三种方式sample-controller，Kubebuilder和Operator SDK。

准备好了吗？让我们开始吧！

sample-controller方式

让我们基于k8s.io/sample-controller来实现cnat，它直接使用了client-go库。sample-controller使用k8s.io/code-generator生成一个类型化的客户端，informers，listers，以及深拷贝方法。每当自定义控制器中的API类型发生变化时（例如，在自定义资源中添加新字段）您必须使用update-codegen.sh脚本来重新生成源代码文件（请参阅GitHub中的源代码）。

警告

您可能已经注意到k8s.io被用作整本书中的基本URL。我们在第3章介绍了它的用法; 需要注意，它实际上是kubernetes.io的别名，在Go语言的包管理中，它被解析为github.com/kubernetes。而k8s.io不会自动转换。所以，对于k8s.io / sample-control包路径对应的源代码路径应该查看github.com/kubernetes/sample-controller。

好的，让我们按照sample-controller的方式，使用client-go实现我们的cnatoperator。（请参阅我们仓库中的相应目录）

引导

开始执行go get k8s.io/sample-controller 命令，将下载源代码和依赖项到我们本机，它将被下载到$ GOPATH / src / k8s.io / sample-controller目录中。

将sample-controller目录的内容复制到您选择的目录中（例如，我们在repo中使用cnat-client-go），您可以顺序运行以下命令来构建和运行基本的控制器（包含默认实现，还没有加入cnat业务逻辑）：

# build custom controller binary:
$ go build -o cnat-controller .

# launch custom controller locally:
$ ./cnat-controller -kubeconfig=$HOME/.kube/config

此命令将启动自定义控制器，等待您注册CRD并创建自定义资源。在第二个终端，输入：

$ kubectl apply -f artifacts/examples/crd.yaml

想确认CRD已正确注册，可以执行命令：

$ kubectl get crds
NAME                            CREATED AT
foos.samplecontroller.k8s.io    2019-05-29T12:16:57Z

您应该在返回信息中看到很多CRD，实际的返回的CRD信息列表与您的环境相关，但是其中应该包含foos.samplecontroller.k8s.io。

接下来，我们创建自定义资源实例foo.samplecontroller.k8s.io/example-foo，并检查控制器是否正常执行了它的业务逻辑：

$ kubectl apply -f artifacts/examples/example-foo.yaml
foo.samplecontroller.k8s.io/example-foo created

$ kubectl get po,rs,deploy,foo
NAME                                           READY   STATUS    RESTARTS   AGE
pod/example-foo-5b8c9679d8-xjhdf               1/1     Running   0          67s

NAME                                           DESIRED   CURRENT   READY AGE
replicaset.extensions/example-foo-5b8c9679d8   1         1         1     67s

NAME                                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.extensions/example-foo              1/1     1            1           67s

NAME                                           AGE
foo.samplecontroller.k8s.io/example-foo        67s

可以看到，它正常执行了！我们现在可以继续实现cnat业务逻辑。

业务逻辑

下一步实现业务逻辑，我们首先将现有目录pkg / apis / samplecontroller重命名为pkg / apis / cnat，然后创建我们自己的CRD和自定义资源，如下所示：

$ cat artifacts/examples/cnat-crd.yaml
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: ats.cnat.programming-kubernetes.info
spec:
  group: cnat.programming-kubernetes.info
  version: v1alpha1
  names:
    kind: At
    plural: ats
  scope: Namespaced

$ cat artifacts/examples/cnat-example.yaml
apiVersion: cnat.programming-kubernetes.info/v1alpha1
kind: At
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: example-at
spec:
  schedule: "2019-04-12T10:12:00Z"
  command: "echo YAY"

请注意，每当API类型发生更改时（例如，当您向At CRD类型添加新字段时），您必须执行update-codegen.sh脚本，如下所示：

$ ./hack/update-codegen.sh

这将自动生成以下内容：

pkg/apis /cnat/v1alpha1/zz_generated.deepcopy.go
pkg/generated/*

在业务逻辑方面，我们在operator中实现了两个部分：

在types.go中我们修改AtSpec结构添加相应的字段，例如schedule和command。请注意，每当您在此处更改某些内容时都必须运行update-codegen.sh，以便重新生成相关文件。
在controller.go中，我们更改NewController()和syncHandler()函数以及添加帮助函数，包括创建pods和检查调度时间。

在types.go中，请注意代表At资源三个阶段的三个常量：在预定的时间之前处于PENDING，然后开始执行时RUNNING状态，最后完成是DONE状态：

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

const (
    PhasePending = "PENDING"
    PhaseRunning = "RUNNING"
    PhaseDone    = "DONE"
)

// AtSpec defines the desired state of At
type AtSpec struct {
    // Schedule is the desired time the command is supposed to be executed.
    // Note: the format used here is UTC time https://www.utctime.net
    Schedule string `json:"schedule,omitempty"`
    // Command is the desired command (executed in a Bash shell) to be
    // executed.
    Command string `json:"command,omitempty"`
}

// AtStatus defines the observed state of At
type AtStatus struct {
    // Phase represents the state of the schedule: until the command is
    // executed it is PENDING, afterwards it is DONE.
    Phase string `json:"phase,omitempty"`
}

请注意显式定义的标记用法+k8s:deepcopy-gen:interfaces（请参阅第5章），用于自动生成相应的源代码。

我们现在可以实现自定义控制器的业务逻辑。也就是说，我们在 controller.go中实现三个状态PhasePending、PhaseRunning和PhaseDone的状态转换。

在“工作队列”中，我们介绍了client-go提供的工作队列。现在，我们来实践一下：在文件controller.go中processNextWorkItem()方法（准确地说，在176行至186行）有以下（生成）的代码：

if when, err := c.syncHandler(key); err != nil {
    c.workqueue.AddRateLimited(key)
    return fmt.Errorf("error syncing '%s': %s, requeuing", key, err.Error())
} else if when != time.Duration(0) {
    c.workqueue.AddAfter(key, when)
} else {
    // Finally, if no error occurs we Forget this item so it does not
    // get queued again until another change happens.
    c.workqueue.Forget(obj)
}

这个代码片段展示了syncHandler()是如何调用我们的自定义函数（尚未编写的，稍后解释）它处理以下三种逻辑：

第一个if分支当错误发生时，通过AddRateLimited()方法调用重新入队操作。
第二个分支，else if通过AddAfter()方法调用重新入队操作以避免hot-looping。
最后一种情况else是，该事件对象已成功处理，使用Forget()方法丢弃。

现在我们已经对通用处理有了充分的了解，让我们继续讨论特定于业务逻辑的功能。关键是上述syncHandler()功能，我们正在实现自定义控制器的业务逻辑。它有以下签名：

// syncHandler compares the actual state with the desired state and attempts
// to converge the two. It then updates the Status block of the At resource
// with the current status of the resource. It returns how long to wait
// until the schedule is due.
func (c *Controller) syncHandler(key string) (time.Duration, error) {
    ...
}

此syncHandler()函数实现以下状态转换：1

...
// If no phase set, default to pending (the initial phase):
if instance.Status.Phase == "" {
    instance.Status.Phase = cnatv1alpha1.PhasePending
}

// Now let's make the main case distinction: implementing
// the state diagram PENDING -> RUNNING -> DONE
switch instance.Status.Phase {
case cnatv1alpha1.PhasePending:
    klog.Infof("instance %s: phase=PENDING", key)
    // As long as we haven't executed the command yet, we need
    // to check if it's time already to act:
    klog.Infof("instance %s: checking schedule %q", key, instance.Spec.Schedule)
    // Check if it's already time to execute the command with a
    // tolerance of 2 seconds:
    d, err := timeUntilSchedule(instance.Spec.Schedule)
    if err != nil {
        utilruntime.HandleError(fmt.Errorf("schedule parsing failed: %v", err))
        // Error reading the schedule - requeue the request:
        return time.Duration(0), err
    }
    klog.Infof("instance %s: schedule parsing done: diff=%v", key, d)
    if d > 0 {
        // Not yet time to execute the command, wait until the
        // scheduled time
        return d, nil
    }

    klog.Infof(
       "instance %s: it's time! Ready to execute: %s", key,
       instance.Spec.Command,
    )
    instance.Status.Phase = cnatv1alpha1.PhaseRunning
case cnatv1alpha1.PhaseRunning:
    klog.Infof("instance %s: Phase: RUNNING", key)

    pod := newPodForCR(instance)

    // Set At instance as the owner and controller
    owner := metav1.NewControllerRef(
        instance, cnatv1alpha1.SchemeGroupVersion.
        WithKind("At"),
    )
    pod.ObjectMeta.OwnerReferences = append(pod.ObjectMeta.OwnerReferences, *owner)

    // Try to see if the pod already exists and if not
    // (which we expect) then create a one-shot pod as per spec:
    found, err := c.kubeClientset.CoreV1().Pods(pod.Namespace).
        Get(pod.Name, metav1.GetOptions{})
    if err != nil && errors.IsNotFound(err) {
        found, err = c.kubeClientset.CoreV1().Pods(pod.Namespace).Create(pod)
        if err != nil {
            return time.Duration(0), err
        }
        klog.Infof("instance %s: pod launched: name=%s", key, pod.Name)
    } else if err != nil {
        // requeue with error
        return time.Duration(0), err
    } else if found.Status.Phase == corev1.PodFailed ||
        found.Status.Phase == corev1.PodSucceeded {
        klog.Infof(
            "instance %s: container terminated: reason=%q message=%q",
            key, found.Status.Reason, found.Status.Message,
        )
        instance.Status.Phase = cnatv1alpha1.PhaseDone
    } else {
        // Don't requeue because it will happen automatically
        // when the pod status changes.
        return time.Duration(0), nil
    }
case cnatv1alpha1.PhaseDone:
    klog.Infof("instance %s: phase: DONE", key)
    return time.Duration(0), nil
default:
    klog.Infof("instance %s: NOP")
    return time.Duration(0), nil
}

// Update the At instance, setting the status to the respective phase:
_, err = c.cnatClientset.CnatV1alpha1().Ats(instance.Namespace).
    UpdateStatus(instance)
if err != nil {
    return time.Duration(0), err
}

// Don't requeue. We should be reconcile because either the pod or
// the CR changes.
return time.Duration(0), nil

此外，为了设置informer和控制器，我们实现NewController()方法：

// NewController returns a new cnat controller
func NewController(
    kubeClientset kubernetes.Interface,
    cnatClientset clientset.Interface,
    atInformer informers.AtInformer,
    podInformer corev1informer.PodInformer) *Controller {

    // Create event broadcaster
    // Add cnat-controller types to the default Kubernetes Scheme so Events
    // can be logged for cnat-controller types.
    utilruntime.Must(cnatscheme.AddToScheme(scheme.Scheme))
    klog.V(4).Info("Creating event broadcaster")
    eventBroadcaster := record.NewBroadcaster()
    eventBroadcaster.StartLogging(klog.Infof)
    eventBroadcaster.StartRecordingToSink(&typedcorev1.EventSinkImpl{
        Interface: kubeClientset.CoreV1().Events(""),
    })
    source := corev1.EventSource{Component: controllerAgentName}
    recorder := eventBroadcaster.NewRecorder(scheme.Scheme, source)

    rateLimiter := workqueue.DefaultControllerRateLimiter()
    controller := &Controller{
        kubeClientset: kubeClientset,
        cnatClientset: cnatClientset,
        atLister:      atInformer.Lister(),
        atsSynced:     atInformer.Informer().HasSynced,
        podLister:     podInformer.Lister(),
        podsSynced:    podInformer.Informer().HasSynced,
        workqueue:     workqueue.NewNamedRateLimitingQueue(rateLimiter, "Ats"),
        recorder:      recorder,
    }

    klog.Info("Setting up event handlers")
    // Set up an event handler for when At resources change
    atInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
        AddFunc: controller.enqueueAt,
        UpdateFunc: func(old, new interface{}) {
            controller.enqueueAt(new)
        },
    })
    // Set up an event handler for when Pod resources change
    podInformer.Informer().AddEventHandler(cache.ResourceEventHandlerFuncs{
        AddFunc: controller.enqueuePod,
        UpdateFunc: func(old, new interface{}) {
            controller.enqueuePod(new)
        },
    })
    return controller
}

为了使它工作，我们还需要两个帮助方法：一个用来计算距离调度执行的时间间隔，如下所示：

func timeUntilSchedule(schedule string) (time.Duration, error) {
    now := time.Now().UTC()
    layout := "2006-01-02T15:04:05Z"
    s, err := time.Parse(layout, schedule)
    if err != nil {
        return time.Duration(0), err
    }
    return s.Sub(now), nil
}

另一个使用busybox容器镜像创建一个带有要执行命令的pod ：

func newPodForCR(cr *cnatv1alpha1.At) *corev1.Pod {
    labels := map[string]string{
        "app": cr.Name,
    }
    return &corev1.Pod{
        ObjectMeta: metav1.ObjectMeta{
            Name:      cr.Name + "-pod",
            Namespace: cr.Namespace,
            Labels:    labels,
        },
        Spec: corev1.PodSpec{
            Containers: []corev1.Container{
                {
                    Name:    "busybox",
                    Image:   "busybox",
                    Command: strings.Split(cr.Spec.Command, " "),
                },
            },
            RestartPolicy: corev1.RestartPolicyOnFailure,
        },
    }
}

这里您可以稍作熟悉，在本章稍后我们会在syncHandler()中再次使用到这两个帮助方法和业务逻辑的基本流程。

请注意，从At资源的角度来看，pod是辅助资源，控制器必须确保清理这些pod，否则它们将会孤立的存在。

现在，通过sample-controller您领略了如何开发operator，但通常您希望专注于创建业务逻辑而不是处理框架代码。因此，您可以选择另外两个项目：Kubebuilder和Operator SDK，来简化这一过程。让我们看看如何使用它们实现cnat。

Kubebuilder

Kubebuilder，由Kubernetes特殊兴趣小组（SIG）API Machinery创建并维护，是一个工具和一套库，使您能够以简单有效的方式构建operator。深度了解Kubebuilder的最佳资源是在线Kubebuilder文档，它将向您介绍内部组件和用法。不过，我们将重点关注使用Kubebuilder 实现我们的operatorcnat（请参阅我们的Git存储库中的相应目录）。

首先，确保已安装所有依赖项 - 即dep，kustomize（参见“Kustomize”）和Kubebuilder本身：

$ dep version
dep:
 version     : v0.5.1
 build date  : 2019-03-11
 git hash    : faa6189
 go version  : go1.12
 go compiler : gc
 platform    : darwin/amd64
 features    : ImportDuringSolve=false

$ kustomize version
Version: {KustomizeVersion:v2.0.3 GitCommit:a6f65144121d1955266b0cd836ce954c04122dc8
          BuildDate:2019-03-18T22:15:21+00:00 GoOs:darwin GoArch:amd64}

$ kubebuilder version
Version: version.Version{
  KubeBuilderVersion:"1.0.8",
  KubernetesVendor:"1.13.1",
  GitCommit:"1adf50ed107f5042d7472ba5ab50d5e1d357169d",
  BuildDate:"2019-01-25T23:14:29Z", GoOs:"unknown", GoArch:"unknown"
}

我们将从头开始编写cnat operator。首先，创建一个目录（我们在我们的仓库中使用cnat-kubebuilder），在此目录下执行所有其他命令。

警告

在撰写本文时，Kubebuilder正在发布新版本（v2）。由于它还不稳定，我们会使用版本v1的命令和设置。

引导

创建cnatoperator的第一步，我们首先执行init命令（请注意，这可能需要几分钟，具体取决于您的环境）：

$ kubebuilder init \
              --domain programming-kubernetes.info \
              --license apache2 \
              --owner "Programming Kubernetes authors"
Run `dep ensure` to fetch dependencies (Recommended) [y/n]?
y
dep ensure
Running make...
make
go generate ./pkg/... ./cmd/...
go fmt ./pkg/... ./cmd/...
go vet ./pkg/... ./cmd/...
go run vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go all
CRD manifests generated under 'config/crds'
RBAC manifests generated under 'config/rbac'
go test ./pkg/... ./cmd/... -coverprofile cover.out
?       github.com/mhausenblas/cnat-kubebuilder/pkg/apis        [no test files]
?       github.com/mhausenblas/cnat-kubebuilder/pkg/controller  [no test files]
?       github.com/mhausenblas/cnat-kubebuilder/pkg/webhook     [no test files]
?       github.com/mhausenblas/cnat-kubebuilder/cmd/manager     [no test files]
go build -o bin/manager github.com/mhausenblas/cnat-kubebuilder/cmd/manager

完成此命令后，Kubebuilder已经为operator搭建了框架，生成了一堆文件，从自定义控制器到CRD样例。您的目录结构应该类似于以下内容（为清晰起见，排除了vendor目录）：

$ tree -I vendor
.
├── Dockerfile
├── Gopkg.lock
├── Gopkg.toml
├── Makefile
├── PROJECT
├── bin
│   └── manager
├── cmd
│   └── manager
│       └── main.go
├── config
│   ├── crds
│   ├── default
│   │   ├── kustomization.yaml
│   │   ├── manager_auth_proxy_patch.yaml
│   │   ├── manager_image_patch.yaml
│   │   └── manager_prometheus_metrics_patch.yaml
│   ├── manager
│   │   └── manager.yaml
│   └── rbac
│       ├── auth_proxy_role.yaml
│       ├── auth_proxy_role_binding.yaml
│       ├── auth_proxy_service.yaml
│       ├── rbac_role.yaml
│       └── rbac_role_binding.yaml
├── cover.out
├── hack
│   └── boilerplate.go.txt
└── pkg
    ├── apis
    │   └── apis.go
    ├── controller
    │   └── controller.go
    └── webhook
        └── webhook.go

13 directories, 22 files

接下来，我们创建一个API资源，使用create api命令创建一个自定义控制器（这应该比上一个命令快，但仍然需要一点时间）：

$ kubebuilder create api \
              --group cnat \
              --version v1alpha1 \
              --kind At
Create Resource under pkg/apis [y/n]?
y
Create Controller under pkg/controller [y/n]?
y
Writing scaffold for you to edit...
pkg/apis/cnat/v1alpha1/at_types.go
pkg/apis/cnat/v1alpha1/at_types_test.go
pkg/controller/at/at_controller.go
pkg/controller/at/at_controller_test.go
Running make...
go generate ./pkg/... ./cmd/...
go fmt ./pkg/... ./cmd/...
go vet ./pkg/... ./cmd/...
go run vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go all
CRD manifests generated under 'config/crds'
RBAC manifests generated under 'config/rbac'
go test ./pkg/... ./cmd/... -coverprofile cover.out
?       github.com/mhausenblas/cnat-kubebuilder/pkg/apis        [no test files]
?       github.com/mhausenblas/cnat-kubebuilder/pkg/apis/cnat   [no test files]
ok      github.com/mhausenblas/cnat-kubebuilder/pkg/apis/cnat/v1alpha1  9.011s
?       github.com/mhausenblas/cnat-kubebuilder/pkg/controller  [no test files]
ok      github.com/mhausenblas/cnat-kubebuilder/pkg/controller/at       8.740s
?       github.com/mhausenblas/cnat-kubebuilder/pkg/webhook     [no test files]
?       github.com/mhausenblas/cnat-kubebuilder/cmd/manager     [no test files]
go build -o bin/manager github.com/mhausenblas/cnat-kubebuilder/cmd/manager

让我们看看发生了什么变化，重点关注以下两个目录：

$ tree config/ pkg/
config/
├── crds
│   └── cnat_v1alpha1_at.yaml
├── default
│   ├── kustomization.yaml
│   ├── manager_auth_proxy_patch.yaml
│   ├── manager_image_patch.yaml
│   └── manager_prometheus_metrics_patch.yaml
├── manager
│   └── manager.yaml
├── rbac
│   ├── auth_proxy_role.yaml
│   ├── auth_proxy_role_binding.yaml
│   ├── auth_proxy_service.yaml
│   ├── rbac_role.yaml
│   └── rbac_role_binding.yaml
└── samples
    └── cnat_v1alpha1_at.yaml
pkg/
├── apis
│   ├── addtoscheme_cnat_v1alpha1.go
│   ├── apis.go
│   └── cnat
│       ├── group.go
│       └── v1alpha1
│           ├── at_types.go
│           ├── at_types_test.go
│           ├── doc.go
│           ├── register.go
│           ├── v1alpha1_suite_test.go
│           └── zz_generated.deepcopy.go
├── controller
│   ├── add_at.go
│   ├── at
│   │   ├── at_controller.go
│   │   ├── at_controller_suite_test.go
│   │   └── at_controller_test.go
│   └── controller.go
└── webhook
    └── webhook.go

11 directories, 27 files

请注意在config/crds/路径中生成了cnat_v1alpha1_at.yaml文件，这是CRD的定义文件。在config / samples /路径中生成了cnat_v1alpha1_at.yaml文件，这是一个CR样例（与CRD文件是同名的）。此外，在pkg /中我们看到生成了许多新文件，最重要的是apis / cnat / v1alpha1 / at_types.go和controller / at / at_controller.go，我们将在下面修改这两个文件。

接下来，我们在Kubernetes中创建一个专用的命名空间cnat并将其设置为默认命名空间，（推荐总是使用专用的命名空间，而不是default）：

$ kubectl create ns cnat && \
  kubectl config set-context $(kubectl config current-context) --namespace=cnat

安装CRD：

$ make install
go run vendor/sigs.k8s.io/controller-tools/cmd/controller-gen/main.go all
CRD manifests generated under 'config/crds'
RBAC manifests generated under 'config/rbac'
kubectl apply -f config/crds
customresourcedefinition.apiextensions.k8s.io/ats.cnat.programming-kubernetes.info created

在本机启动operator：

$ make run
go generate ./pkg/... ./cmd/...
go fmt ./pkg/... ./cmd/...
go vet ./pkg/... ./cmd/...
go run ./cmd/manager/main.go
{"level":"info","ts":1559152740.0550249,"logger":"entrypoint",
  "msg":"setting up client for manager"}
{"level":"info","ts":1559152740.057556,"logger":"entrypoint",
  "msg":"setting up manager"}
{"level":"info","ts":1559152740.1396701,"logger":"entrypoint",
  "msg":"Registering Components."}
{"level":"info","ts":1559152740.1397,"logger":"entrypoint",
  "msg":"setting up scheme"}
{"level":"info","ts":1559152740.139773,"logger":"entrypoint",
  "msg":"Setting up controller"}
{"level":"info","ts":1559152740.139831,"logger":"kubebuilder.controller",
  "msg":"Starting EventSource","controller":"at-controller",
  "source":"kind source: /, Kind="}
{"level":"info","ts":1559152740.139929,"logger":"kubebuilder.controller",
  "msg":"Starting EventSource","controller":"at-controller",
  "source":"kind source: /, Kind="}
{"level":"info","ts":1559152740.139971,"logger":"entrypoint",
  "msg":"setting up webhooks"}
{"level":"info","ts":1559152740.13998,"logger":"entrypoint",
  "msg":"Starting the Cmd."}
{"level":"info","ts":1559152740.244628,"logger":"kubebuilder.controller",
  "msg":"Starting Controller","controller":"at-controller"}
{"level":"info","ts":1559152740.344791,"logger":"kubebuilder.controller",
  "msg":"Starting workers","controller":"at-controller","worker count":1}

保留正在运行的终端会话，打开一个新的会话，再执行命令安装CRD，并创建示例自定义资源，如下所示：

$ kubectl apply -f config/crds/cnat_v1alpha1_at.yaml
customresourcedefinition.apiextensions.k8s.io/ats.cnat.programming-kubernetes.info
configured

$ kubectl get crds
NAME                                   CREATED AT
ats.cnat.programming-kubernetes.info   2019-05-29T17:54:51Z

$ kubectl apply -f config/samples/cnat_v1alpha1_at.yaml
at.cnat.programming-kubernetes.info/at-sample created

现在查看以下刚才执行make run的会话，会看到以下输出：

...
{"level":"info","ts":1559153311.659829,"logger":"controller",
  "msg":"Creating Deployment","namespace":"cnat","name":"at-sample-deployment"}
{"level":"info","ts":1559153311.678407,"logger":"controller",
  "msg":"Updating Deployment","namespace":"cnat","name":"at-sample-deployment"}
{"level":"info","ts":1559153311.6839428,"logger":"controller",
  "msg":"Updating Deployment","namespace":"cnat","name":"at-sample-deployment"}
{"level":"info","ts":1559153311.693443,"logger":"controller",
  "msg":"Updating Deployment","namespace":"cnat","name":"at-sample-deployment"}
{"level":"info","ts":1559153311.7023401,"logger":"controller",
  "msg":"Updating Deployment","namespace":"cnat","name":"at-sample-deployment"}
{"level":"info","ts":1559153332.986961,"logger":"controller",#
  "msg":"Updating Deployment","namespace":"cnat","name":"at-sample-deployment"}

这说明我们设置成功！现在我们已经完成了基础框架的搭建并成功启动了cnat operator，我们可以继续：使用Kubebuilder 实现业务逻辑cnat。

业务逻辑

首先，我们将config / crds / cnat_v1alpha1_at.yaml和config / samples / cnat_v1alpha1_at.yaml更改为我们自己的cnat对应的CRD定义和CR样例，使用与“sample-controller示例”相同的结构。

在业务逻辑方面，我们在operator中实现了两个部分：

在pkg / apis / cnat / v1alpha1 / at_types.go文件中，我们修改AtSpec结构增加相应的字段，例如schedule和command。请注意，每当您在此处更改某些内容时都必须运行make，以便重新生成相关文件。Kubebuilder使用Kubernetes生成器（在第5章中描述）并发布一组它自己的生成器（例如，生成CRD 定义文件）。
在pkg / controller / at / at_controller.go中，我们修改了Reconcile(request reconcile.Request)方法，按照Spec.Schedule中在定义的时间点去创建pod。

at_types.go文件：

const (
    PhasePending = "PENDING"
    PhaseRunning = "RUNNING"
    PhaseDone    = "DONE"
)

// AtSpec defines the desired state of At
type AtSpec struct {
    // Schedule is the desired time the command is supposed to be executed.
    // Note: the format used here is UTC time https://www.utctime.net
    Schedule string `json:"schedule,omitempty"`
    // Command is the desired command (executed in a Bash shell) to be executed.
    Command string `json:"command,omitempty"`
}

// AtStatus defines the observed state of At
type AtStatus struct {
    // Phase represents the state of the schedule: until the command is executed
    // it is PENDING, afterwards it is DONE.
    Phase string `json:"phase,omitempty"`
}

在at_controller.go文件中，我们实现三个阶段之间的状态转变，PENDING到RUNNING到DONE：

func (r *ReconcileAt) Reconcile(req reconcile.Request) (reconcile.Result, error) {
    reqLogger := log.WithValues("namespace", req.Namespace, "at", req.Name)
    reqLogger.Info("=== Reconciling At")
    // Fetch the At instance
    instance := &cnatv1alpha1.At{}
    err := r.Get(context.TODO(), req.NamespacedName, instance)
    if err != nil {
        if errors.IsNotFound(err) {
            // Request object not found, could have been deleted after
            // reconcile request—return and don't requeue:
            return reconcile.Result{}, nil
        }
            // Error reading the object—requeue the request:
        return reconcile.Result{}, err
    }

    // If no phase set, default to pending (the initial phase):
    if instance.Status.Phase == "" {
        instance.Status.Phase = cnatv1alpha1.PhasePending
    }

    // Now let's make the main case distinction: implementing
    // the state diagram PENDING -> RUNNING -> DONE
    switch instance.Status.Phase {
    case cnatv1alpha1.PhasePending:
        reqLogger.Info("Phase: PENDING")
        // As long as we haven't executed the command yet, we need to check if
        // it's already time to act:
        reqLogger.Info("Checking schedule", "Target", instance.Spec.Schedule)
        // Check if it's already time to execute the command with a tolerance
        // of 2 seconds:
        d, err := timeUntilSchedule(instance.Spec.Schedule)
        if err != nil {
            reqLogger.Error(err, "Schedule parsing failure")
            // Error reading the schedule. Wait until it is fixed.
            return reconcile.Result{}, err
        }
        reqLogger.Info("Schedule parsing done", "Result", "diff",
            fmt.Sprintf("%v", d))
        if d > 0 {
            // Not yet time to execute the command, wait until the scheduled time
            return reconcile.Result{RequeueAfter: d}, nil
        }
        reqLogger.Info("It's time!", "Ready to execute", instance.Spec.Command)
        instance.Status.Phase = cnatv1alpha1.PhaseRunning
    case cnatv1alpha1.PhaseRunning:
        reqLogger.Info("Phase: RUNNING")
        pod := newPodForCR(instance)
        // Set At instance as the owner and controller
        err := controllerutil.SetControllerReference(instance, pod, r.scheme)
        if err != nil {
            // requeue with error
            return reconcile.Result{}, err
        }
        found := &corev1.Pod{}
        nsName := types.NamespacedName{Name: pod.Name, Namespace: pod.Namespace}
        err = r.Get(context.TODO(), nsName, found)
        // Try to see if the pod already exists and if not
        // (which we expect) then create a one-shot pod as per spec:
        if err != nil && errors.IsNotFound(err) {
            err = r.Create(context.TODO(), pod)
            if err != nil {
            // requeue with error
                return reconcile.Result{}, err
            }
            reqLogger.Info("Pod launched", "name", pod.Name)
        } else if err != nil {
            // requeue with error
            return reconcile.Result{}, err
        } else if found.Status.Phase == corev1.PodFailed ||
                  found.Status.Phase == corev1.PodSucceeded {
            reqLogger.Info("Container terminated", "reason",
                found.Status.Reason, "message", found.Status.Message)
            instance.Status.Phase = cnatv1alpha1.PhaseDone
        } else {
            // Don't requeue because it will happen automatically when the
            // pod status changes.
            return reconcile.Result{}, nil
        }
    case cnatv1alpha1.PhaseDone:
        reqLogger.Info("Phase: DONE")
        return reconcile.Result{}, nil
    default:
        reqLogger.Info("NOP")
        return reconcile.Result{}, nil
    }

    // Update the At instance, setting the status to the respective phase:
    err = r.Status().Update(context.TODO(), instance)
    if err != nil {
        return reconcile.Result{}, err
    }

    // Don't requeue. We should be reconcile because either the pod
    // or the CR changes.
    return reconcile.Result{}, nil
}

请注意，最后的Update调用操作在/ status子资源上（参见“Status subresource”）而不是整个CR。因此，在这里我们遵循spec-status分离的最佳实践。

当我们创建了CR实例 example-at，我们就会在本地执行的operator的会话中看到以下输出：

$ make run
...
{"level":"info","ts":1555063897.488535,"logger":"controller",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063897.488621,"logger":"controller",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063897.4886441,"logger":"controller",
  "msg":"Checking schedule","namespace":"cnat","at":"example-at",
  "Target":"2019-04-12T10:12:00Z"}
{"level":"info","ts":1555063897.488703,"logger":"controller",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 10:12:00 +0000 UTC with a diff of 22.511336s"}
{"level":"info","ts":1555063907.489264,"logger":"controller",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063907.489402,"logger":"controller",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063907.489428,"logger":"controller",
  "msg":"Checking schedule","namespace":"cnat","at":"example-at",
  "Target":"2019-04-12T10:12:00Z"}
{"level":"info","ts":1555063907.489486,"logger":"controller",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 10:12:00 +0000 UTC with a diff of 12.510551s"}
{"level":"info","ts":1555063917.490178,"logger":"controller",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063917.4902349,"logger":"controller",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063917.490247,"logger":"controller",
  "msg":"Checking schedule","namespace":"cnat","at":"example-at",
  "Target":"2019-04-12T10:12:00Z"}
{"level":"info","ts":1555063917.490278,"logger":"controller",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 10:12:00 +0000 UTC with a diff of 2.509743s"}
{"level":"info","ts":1555063927.492718,"logger":"controller",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063927.49283,"logger":"controller",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063927.492857,"logger":"controller",
  "msg":"Checking schedule","namespace":"cnat","at":"example-at",
  "Target":"2019-04-12T10:12:00Z"}
{"level":"info","ts":1555063927.492915,"logger":"controller",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 10:12:00 +0000 UTC with a diff of -7.492877s"}
{"level":"info","ts":1555063927.4929411,"logger":"controller",
  "msg":"It's time!","namespace":"cnat","at":
  "example-at","Ready to execute":"echo YAY"}
{"level":"info","ts":1555063927.626236,"logger":"controller",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063927.626303,"logger":"controller",
  "msg":"Phase: RUNNING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063928.07445,"logger":"controller",
  "msg":"Pod launched","namespace":"cnat","at":"example-at",
  "name":"example-at-pod"}
{"level":"info","ts":1555063928.199562,"logger":"controller",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063928.199645,"logger":"controller",
  "msg":"Phase: DONE","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063937.631733,"logger":"controller",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555063937.631783,"logger":"controller",
  "msg":"Phase: DONE","namespace":"cnat","at":"example-at"}
...

想验证我们的自定义控制器是否已完成其工作，可以执行以下命令：

$ kubectl get at,pods
NAME                                                  AGE
at.cnat.programming-kubernetes.info/example-at        11m

NAME                 READY   STATUS        RESTARTS   AGE
pod/example-at-pod   0/1     Completed     0          38s

太棒了！example-at-pod 已被创建，现在我们来看看命令执行的结果：

$ kubectl logs example-at-pod
YAY

以上我们完成自定义控制器的开发后，并在本地执行。您可能希望将operator构建成镜像，在Kubernetes部署。可以使用以下命令生成容器镜像，并将其推送到镜像仓库，例如 quay.io/pk/cnat：

$ export IMG=quay.io/pk/cnat:v1

$ make docker-build

$ make docker-push

在了解如何使用Kubebuilder之后，让我们来看看如何使用operator SDK做同样的事情。

Operator SDK

为了更容易构建Kubernetes应用程序，CoreOS / Red Hat将operator和代码框架整合在一起。Operator SDK就是其中的一部分，它使开发人员无需深入了解Kubernetes API即可开发出operators。

Operator SDK提供了构建，测试和打包operator的工具。SDK中包含了更多其他可用的功能，特别是在测试时，不过在这里我们专注于使用SDK 实现我们的运算符cnat（请参阅我们的Git存储库中的相应目录）。

首先要做的事情是：确保安装Operator SDK并检查所有依赖项是否可用：

$ dep version
dep:
 version     : v0.5.1
 build date  : 2019-03-11
 git hash    : faa6189
 go version  : go1.12
 go compiler : gc
 platform    : darwin/amd64
 features    : ImportDuringSolve=false

 $ operator-sdk --version
operator-sdk version v0.6.0

引导

现在开始按如下方式创建cnat operator：

$ operator-sdk new cnat-operator && cd cnat-operator

接下来，和Kubebuilder非常相似，我们添加一个API - 或简单地说：初始化自定义控制器，如下所示：

$ operator-sdk add api \
               --api-version =cnat.programming-kubernetes.info/v1alpha1 \
               --kind =At

$ operator-sdk add controller \
               --api-version =cnat.programming-kubernetes.info/v1alpha1 \
               --kind =At

这些命令将产生所需的框架代码以及一些辅助功能，如深拷贝方法DeepCopy()，DeepCopyInto()和DeepCopyObject()。

现在我们可以在Kubernetes集群中创建自动生成的CRD：

$ kubectl apply -f deploy/crds/cnat_v1alpha1_at_crd.yaml

$ kubectl get crds
NAME                                             CREATED AT
ats.cnat.programming-kubernetes.info             2019-04-01T14:03:33Z

我们在本地启动自定义控制器cnat。启动后，它就可以开始处理请求：

$ OPERATOR_NAME=cnatop operator-sdk up local --namespace "cnat"
INFO[0000] Running the operator locally.
INFO[0000] Using namespace cnat.
{"level":"info","ts":1555041531.871706,"logger":"cmd",
  "msg":"Go Version: go1.12.1"}
{"level":"info","ts":1555041531.871785,"logger":"cmd",
  "msg":"Go OS/Arch: darwin/amd64"}
{"level":"info","ts":1555041531.8718028,"logger":"cmd",
  "msg":"Version of operator-sdk: v0.6.0"}
{"level":"info","ts":1555041531.8739321,"logger":"leader",
  "msg":"Trying to become the leader."}
{"level":"info","ts":1555041531.8743382,"logger":"leader",
  "msg":"Skipping leader election; not running in a cluster."}
{"level":"info","ts":1555041536.1611362,"logger":"cmd",
  "msg":"Registering Components."}
{"level":"info","ts":1555041536.1622112,"logger":"kubebuilder.controller",
  "msg":"Starting EventSource","controller":"at-controller",
  "source":"kind source: /, Kind="}
{"level":"info","ts":1555041536.162519,"logger":"kubebuilder.controller",
  "msg":"Starting EventSource","controller":"at-controller",
  "source":"kind source: /, Kind="}
{"level":"info","ts":1555041539.978822,"logger":"metrics",
  "msg":"Skipping metrics Service creation; not running in a cluster."}
{"level":"info","ts":1555041539.978875,"logger":"cmd",
  "msg":"Starting the Cmd."}
{"level":"info","ts":1555041540.179469,"logger":"kubebuilder.controller",
  "msg":"Starting Controller","controller":"at-controller"}
{"level":"info","ts":1555041540.280784,"logger":"kubebuilder.controller",
  "msg":"Starting workers","controller":"at-controller","worker count":1}

在我们创建CR ats.cnat.programming-kubernetes.info 之前自定义控制器日志信息不会再发生滚动。接下来我们创建CR实例：

$ cat deploy/crds/cnat_v1alpha1_at_cr.yaml
apiVersion: cnat.programming-kubernetes.info/v1alpha1
kind: At
metadata:
  name: example-at
spec:
  schedule: "2019-04-11T14:56:30Z"
  command: "echo YAY"

$ kubectl apply -f deploy/crds/cnat_v1alpha1_at_cr.yaml

$ kubectl get at
NAME                                             AGE
at.cnat.programming-kubernetes.info/example-at   54s

业务逻辑

在业务逻辑方面，我们在operator中实现了两个部分：

在pkg / apis / cnat / v1alpha1 / at_types.go中，我们修改AtSpecstruct添加相应的字段，例如schedule和command，并使用operator-sdk generate k8s重新生成代码，同时使用operator-sdk generate openapi 命令生成OpenAPI。
在pkg / controller / at / at_controller.go中，我们修改了Reconcile(request reconcile.Request)方法，按照Spec.Schedule中在定义的时间点去创建pod。

详细地修改如下，文件at_types.go：

// AtSpec defines the desired state of At
// +k8s:openapi-gen=true
type AtSpec struct {
    // Schedule is the desired time the command is supposed to be executed.
    // Note: the format used here is UTC time https://www.utctime.net
    Schedule string `json:"schedule,omitempty"`
    // Command is the desired command (executed in a Bash shell) to be executed.
    Command string `json:"command,omitempty"`
}

// AtStatus defines the observed state of At
// +k8s:openapi-gen=true
type AtStatus struct {
    // Phase represents the state of the schedule: until the command is executed
    // it is PENDING, afterwards it is DONE.
    Phase string `json:"phase,omitempty"`
}

在at_controller.go文件z红我们实现了三个阶段的状态转换，PENDING到RUNNING到DONE。

注意

controller-runtime是另一个属于SIG API Machinery的项目，这个库的目标在于以Go语言包的形式提供一套通用的低级API功能来构建控制器。有关详细信息，请参阅第4章。

由于Kubebuilder和Operator SDK都使用了controller-runtime，Reconcile()函数实际上是相同的：

func (r *ReconcileAt) Reconcile(request reconcile.Request) (reconcile.Result, error) {
    the-same-as-for-kubebuilder
}

当CR example-at被创建后，我们会看到本地operator有以下输出：

$ OPERATOR_NAME=cnatop operator-sdk up local --namespace "cnat"
INFO[0000] Running the operator locally.
INFO[0000] Using namespace cnat.
...
{"level":"info","ts":1555044934.023597,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044934.023713,"logger":"controller_at",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044934.0237482,"logger":"controller_at",
  "msg":"Checking schedule","namespace":"cnat","at":
  "example-at","Target":"2019-04-12T04:56:00Z"}
{"level":"info","ts":1555044934.02382,"logger":"controller_at",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 04:56:00 +0000 UTC with a diff of 25.976236s"}
{"level":"info","ts":1555044934.148148,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044934.148224,"logger":"controller_at",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044934.148243,"logger":"controller_at",
  "msg":"Checking schedule","namespace":"cnat","at":"example-at",
  "Target":"2019-04-12T04:56:00Z"}
{"level":"info","ts":1555044934.1482902,"logger":"controller_at",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 04:56:00 +0000 UTC with a diff of 25.85174s"}
{"level":"info","ts":1555044944.1504588,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044944.150568,"logger":"controller_at",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044944.150599,"logger":"controller_at",
  "msg":"Checking schedule","namespace":"cnat","at":"example-at",
  "Target":"2019-04-12T04:56:00Z"}
{"level":"info","ts":1555044944.150663,"logger":"controller_at",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 04:56:00 +0000 UTC with a diff of 15.84938s"}
{"level":"info","ts":1555044954.385175,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044954.3852649,"logger":"controller_at",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044954.385288,"logger":"controller_at",
  "msg":"Checking schedule","namespace":"cnat","at":"example-at",
  "Target":"2019-04-12T04:56:00Z"}
{"level":"info","ts":1555044954.38534,"logger":"controller_at",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 04:56:00 +0000 UTC with a diff of 5.614691s"}
{"level":"info","ts":1555044964.518383,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044964.5184839,"logger":"controller_at",
  "msg":"Phase: PENDING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044964.518566,"logger":"controller_at",
  "msg":"Checking schedule","namespace":"cnat","at":"example-at",
  "Target":"2019-04-12T04:56:00Z"}
{"level":"info","ts":1555044964.5186381,"logger":"controller_at",
  "msg":"Schedule parsing done","namespace":"cnat","at":"example-at",
  "Result":"2019-04-12 04:56:00 +0000 UTC with a diff of -4.518596s"}
{"level":"info","ts":1555044964.5186849,"logger":"controller_at",
  "msg":"It's time!","namespace":"cnat","at":"example-at",
  "Ready to execute":"echo YAY"}
{"level":"info","ts":1555044964.642559,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044964.642622,"logger":"controller_at",
  "msg":"Phase: RUNNING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044964.911037,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044964.9111192,"logger":"controller_at",
  "msg":"Phase: RUNNING","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044966.038684,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044966.038771,"logger":"controller_at",
  "msg":"Phase: DONE","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044966.708663,"logger":"controller_at",
  "msg":"=== Reconciling At","namespace":"cnat","at":"example-at"}
{"level":"info","ts":1555044966.708749,"logger":"controller_at",
  "msg":"Phase: DONE","namespace":"cnat","at":"example-at"}
...

在这里你可以看到operator的三个状态：直到时间戳1555044964.518566一直在PENDING，然后RUNNING，然后DONE。

要验证自定义控制器的功能并检查操作结果，请输入：

$ kubectl get at,pods
NAME                                                  AGE
at.cnat.programming-kubernetes.info/example-at        23m

NAME                 READY   STATUS        RESTARTS   AGE
pod/example-at-pod   0/1     Completed     0          46s

$ kubectl logs example-at-pod
YAY

以上我们完成自定义控制器的开发后，并在本地执行。您可能希望将operator构建成镜像，在Kubernetes部署。可以使用以下命令生成容器镜像：

$ operator-sdk build $REGISTRY/PROJECT/IMAGE

这里是有关Operator SDK及其示例的更多资源：

Toader Sebastian在BanzaiCloud上发表的“Kubernetes Operator SDK完整指南”
Rob Szumski的博客文章“Building a Kubernetes Operator for Prometheus and Thanos”
来自Cloudark on ITNEXT的“Kubernetes Operator Development Guidelines for Improved Usability”

为了总结本章，我们来看一些编写自定义控制器和operator的替代方法。

其他方法

除了我们讨论过的方法之外，您可以看看以下项目，库和工具：

Metacontroller
Metacontroller的基本思想是为您提供状态和变化的声明性规范，使用JSON接口，基于级别触发的协调循环。也就是说，您将收到描述观察状态的JSON并返回描述所需状态的JSON。这对于在Python或JavaScript等动态脚本语言中快速开发自动化特别有用。除了简单的控制器，Metacontroller还允许您将API组合成更高级别的抽象 - 例如，BlueGreenDeployment。
KUDO
类似对于Metacontroller，KUDO提供了一种声明性方法来构建Kubernetes operator，涵盖整个应用程序生命周期。简而言之，是Mesosphere将Apache Mesos框架经验迁移到了Kubernetes。KUDO易于使用，几乎不需要编码; 实质上，您必须指定的是Kubernetes 配置清单的集合，其中包含了何时执行等内置逻辑。
Rook操作工具包
这个是一个实现operator的通用库。它起源于Rook Operator，现在已被分拆成一个独立项目。
ericchiang / K8S
这是一个由Eric Chiang精简的Go客户端，使用Kubernetesprotobuf协议。它的行为类似于官方的Kubernetes client-go，但只导入两个外部依赖项。虽然它有一些限制(例如，在集群访问配置方面)它是一个简单易用的Go包。
kutil
AppsCode通过kutil提供Kubernetes client-go附加组件。
基于CLI客户端的方法
一个客户端方法，主要用于实验和测试，是以编程方式使用用kubectl（例如，kubecuddler库）。

注意

虽然在本书中主要介绍了使用Go编程语言编写operator，但您也可以使用其他语言编写。两个值得关注的例子是Flant创建的Shell-operator，它使您能够在shell脚本中编写operator，以及Zalando创建的Kopf（Kubernetes运算符框架），一个Python框架和库。

正如本章开头所提到的，operator领域正在迅速发展，越来越多的参与者以代码和最佳实践的形式分享他们的知识，因此请关注这里的新工具。请一定要看看网上资源和论坛，比如#kubernetes-operators，#kubebuilder和#client-go-docs等Kubernetes Slack 上的频道，学习新的方法、讨论问题，当遇到困难时得到帮助。

未来方向

目前operator开发仍然被认为将是最受欢迎和广泛使用的方式。在Kubernetes生态中，在CR和控制器方面，有几个特殊兴趣小组。最主要的是SIG API Machinery，目前它拥有CR和控制器，负责Kubebuilder项目。operator SDK已经努力在与Kubebuilder API的对齐，因此你也会看到很多重合的地方。

总结

在本章中，我们了解了不同的工具，可以帮助您更有效地编写自定义控制器和operator。过去，遵循sample-controller是唯一的选择，但是现在可以使用Kubebuilder和Operator SDK，可以让您专注于自定义控制器的业务逻辑而不需要太多的关注于框架代码。另外一点幸运的是，这两个工具共享了很多API和代码，因此从一个工具转移到另一个工具也会比较方便。

现在，让我们看看如何发布我们的劳动成果 - 即如何打包和发布我们本章编写的控制器。

1我们只在这里展示相关部分; 函数本身有很多其他的样板代码，我们并不关心它们。

上一页第5章自动生成代码下一页第7章控制器管理

最后更新于4年前

这有帮助吗？