kubernetes(k8s) CRD

时间:2022-06-02 bigyong 人气:0

CustomResourceDefinition简介：

在 Kubernetes 中一切都可视为资源，Kubernetes 1.7 之后增加了对 CRD 自定义资源二次开发能力来扩展 Kubernetes API，通过 CRD 我们可以向 Kubernetes API 中增加新资源类型，而不需要修改 Kubernetes 源码来创建自定义的 API server，该功能大大提高了 Kubernetes 的扩展能力。

当你创建一个新的CustomResourceDefinition (CRD)时，Kubernetes API服务器将为你指定的每个版本创建一个新的RESTful资源路径，我们可以根据该api路径来创建一些我们自己定义的类型资源。CRD可以是命名空间的，也可以是集群范围的，由CRD的作用域(scpoe)字段中所指定的，与现有的内置对象一样，删除名称空间将删除该名称空间中的所有自定义对象。customresourcedefinition本身没有名称空间，所有名称空间都可以使用。

目前扩展Kubernetes API的常用方式有3种:

使用CRD(CustomResourceDefinitions)自定义资源类型
开发自定义的APIServer并聚合至主API Server
及定制扩展API Server源码。这其中，CRD最为易用但限制颇多，自定义API Server更富于弹性但代码工作量偏大，而仅在必须添加新的核心类型才能确保专用的Kberneves集群功能正常，才应该定制系统源码

CRD-->CRT-->CR

其中CRD与CRT一般由开发或服务供应商提供
CRD只是定义一个类型Kind，但实际把kind运行起来CR需要有Controller来对资源进行控制，所有只有定义CRD定义没有并没有实际意义，当然也可以通过定义现在kind来运行，比如deployment 通过定义 RC来运行

配置规范

apiVersion: apiextensions.k8s.io/v1 #API群组和版本
kind: CustomResourceDefinition #资源类别
metadata:
  -name <string> #资源名称
spec:
  conversion <object> #定义不同版本间的格式转换方式
    strategy <string># 不同版本间的自定义资源转换策略，有None和webhook两种取值
    webhook <0bject>#如何调用用于进行格式转换的webhook
  group <string>#资源所属的API群组
  names <object># 自定义资源的类型，即该CRD创建资源规范时使用的kind
    categories <[]string>#资源所属的类别编目，例如"kubectl get all"中的all
    kind <string> #kind名称,必选字段
    listKind <string> #资源列表名称，默认为"`kind`List"
    plural <string>  #复数，用于API路径`/apis/<group>/<version>/. . ./<plural>'
    shortNames <[string>#该资源的kind的缩写格式
    singular <string>#资源kind的单数形式，必须使用全小写字母，默认为小写的kind名称
  preserveUnknownFields <boolean> #预留的非知名字段，kind等都是知名的预留字段
  scope <string> #作用域，可用值为Cluster和Namespaced
  versions <[]object>#版本号定义
    additionalPrinterColumns <[]0bject> #需要返回的额外信息
    name <string>  #形如vM[alphaN|betaN]格式的版本名称，例如v1或vlalpha2等
    schema <object> #该资源的数据格式(schema)定义，必选字段
      openAPIV3Schema <object> #用于校验字段的schema对象，格式请参考相关手册
    served <boolean> #是否允许通过RESTful API调度该版本，必选字段
    storage <boolean> #将自定义资源存储于etcd中时是不是使用该版本
    subresources <0bject>#子资源定义
      scale <0bject># 启用scale子资源，通过autoscaling/v1.Scale发送负荷
      status <map[string]># 启用status子资源，为资源生成/status端点

可以查看之前部署Calico创建的自定义CRD

[root@k8s-master ~]# kubectl api-resources      #查看所有资源类型
NAME                              SHORTNAMES   APIGROUP                       NAMESPACED   KIND
...  
bgpconfigurations                              crd.projectcalico.org          false        BGPConfiguration
bgppeers                                       crd.projectcalico.org          false        BGPPeer
blockaffinities                                crd.projectcalico.org          false        BlockAffinity
clusterinformations                            crd.projectcalico.org          false        ClusterInformation
felixconfigurations                            crd.projectcalico.org          false        FelixConfiguration
globalnetworkpolicies                          crd.projectcalico.org          false        GlobalNetworkPolicy
globalnetworksets                              crd.projectcalico.org          false        GlobalNetworkSet
hostendpoints                                  crd.projectcalico.org          false        HostEndpoint
ipamblocks                                     crd.projectcalico.org          false        IPAMBlock
ipamconfigs                                    crd.projectcalico.org          false        IPAMConfig
ipamhandles                                    crd.projectcalico.org          false        IPAMHandle
ippools                                        crd.projectcalico.org          false        IPPool
kubecontrollersconfigurations                  crd.projectcalico.org          false        KubeControllersConfiguration
networkpolicies                                crd.projectcalico.org          true         NetworkPolicy
networksets                                    crd.projectcalico.org          true         NetworkSet

查看calico的yaml文件可以看到里面很多CRD的定义

[root@k8s-master plugin]# vim calico.yaml   
...
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: ippools.crd.projectcalico.org
spec:
......
...
[root@k8s-master plugin]# kubectl get CustomResourceDefinition
NAME                                                  CREATED AT
bgpconfigurations.crd.projectcalico.org               2021-08-29T14:33:24Z
bgppeers.crd.projectcalico.org                        2021-08-29T14:33:24Z
blockaffinities.crd.projectcalico.org                 2021-08-29T14:33:24Z
clusterinformations.crd.projectcalico.org             2021-08-29T14:33:24Z
felixconfigurations.crd.projectcalico.org             2021-08-29T14:33:24Z
globalnetworkpolicies.crd.projectcalico.org           2021-08-29T14:33:24Z
globalnetworksets.crd.projectcalico.org               2021-08-29T14:33:24Z
hostendpoints.crd.projectcalico.org                   2021-08-29T14:33:24Z
ipamblocks.crd.projectcalico.org                      2021-08-29T14:33:24Z
ipamconfigs.crd.projectcalico.org                     2021-08-29T14:33:24Z
ipamhandles.crd.projectcalico.org                     2021-08-29T14:33:24Z
ippools.crd.projectcalico.org                         2021-08-29T14:33:24Z
kubecontrollersconfigurations.crd.projectcalico.org   2021-08-29T14:33:24Z
networkpolicies.crd.projectcalico.org                 2021-08-29T14:33:24Z
networksets.crd.projectcalico.org                     2021-08-29T14:33:25Z

示例1: 创建自定义CRD

[root@k8s-master crd]# cat crd-v1-user.yaml 
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: users.auth.ilinux.io
spec:
  group: auth.ilinux.io
  names:
    kind: User
    plural: users
    singular: user
    shortNames:
    - u
  scope: Namespaced  #名称空间级别
  versions:
  - served: true
    storage: true
    name: v1alpha1  #版本号
    schema:
      openAPIV3Schema:
        type: object    #对字段做限制 
        properties:
          spec:
            type: object
            properties:
              userID:
                type: integer  #整形
                minimum: 1
                maximum: 65535
              groups :
                type: array   #列表
                items:
                  type: string
              email:
                type: string
              password:
                type: string
                format: password
            required: ["userID","groups"]
[root@k8s-master crd]# kubectl apply -f crd-v1-user.yaml 
[root@k8s-master crd]# kubectl api-resources
NAME                              SHORTNAMES   APIGROUP                       NAMESPACED   KIND
bindings                                                                      true         Binding
...
users                             u            auth.ilinux.io                 true         User

创造自定义CRD类型

[root@k8s-master crd]# cat user-cr-demo.yaml 
apiVersion: auth.ilinux.io/v1alpha1
kind: User
metadata:
  name: admin
  namespace: default
spec:
  userID: 1
  email: test@test.com
  groups:
  - superusers
  - adminstrators
  password: ikubernetes.io
[root@k8s-master crd]# kubectl apply -f user-cr-demo.yaml 
user.auth.ilinux.io/admin created
[root@k8s-master crd]# kubectl get User
NAME    AGE
admin   14s
[root@k8s-master ~]# kubectl describe User admin
Name:         admin
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  auth.ilinux.io/v1alpha1
Kind:         User
Metadata:
  Creation Timestamp:  2021-09-10T14:51:53Z
  Generation:          1
  Managed Fields:
    API Version:  auth.ilinux.io/v1alpha1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:email:
        f:groups:
        f:password:
        f:userID:
    Manager:         kubectl-client-side-apply
    Operation:       Update
    Time:            2021-09-10T14:51:53Z
  Resource Version:  2583010
  Self Link:         /apis/auth.ilinux.io/v1alpha1/namespaces/default/users/admin
  UID:               5af89454-e067-4f30-83b7-cc2ad82e3526
Spec:
  Email:  test@test.com
  Groups:
    superusers
    adminstrators
  Password:  ikubernetes.io
  User ID:   1
Events:      <none>

以上定义的kind资源没Controller并不能运行成实际对象，Controller的开发需要开发来完成

示例2: etcd Operator 部署 (该项目已不在维护)

Operator 项目地址:

https://github.com/coreos/etcd-operator/blob/master/doc/user/install_guide.md

https://github.com/coreos/etcd-operator

https://github.com/operator-framework/awesome-operators

先安装RBAC 再安装etcd operator 再部署创建etcd集群

[root@k8s-master etcd-operator]# example/rbac/create_role.sh
Creating role with ROLE_NAME=etcd-operator, NAMESPACE=default
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole
clusterrole.rbac.authorization.k8s.io/etcd-operator created
Creating role binding with ROLE_NAME=etcd-operator, ROLE_BINDING_NAME=etcd-operator, NAMESPACE=default
Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBinding
clusterrolebinding.rbac.authorization.k8s.io/etcd-operator created
[root@k8s-master etcd-operator]# kubectl create -f example/deployment.yaml
error: unable to recognize "example/deployment.yaml": no matches for kind "Deployment" in version "extensions/v1beta1"
#deployment版本太老修改example/deployment.yaml
[root@k8s-master etcd-operator]# cat example/deployment.yaml
apiVersion: apps/v1  #版本
kind: Deployment
metadata:
  name: etcd-operator
spec:
  replicas: 1
  selector:   #添加字段
    matchLabels:
      name: etcd-operator
  template:
    metadata:
      labels:
        name: etcd-operator
    spec:
      containers:
      - name: etcd-operator
        image: quay.io/coreos/etcd-operator:v0.9.4
        command:
        - etcd-operator
        # Uncomment to act for resources in all namespaces. More information in doc/user/clusterwide.md
        #- -cluster-wide
        env:
        - name: MY_POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
[root@k8s-master etcd-operator]# kubectl create -f example/deployment.yaml
deployment.apps/etcd-operator created
[root@k8s-master etcd-operator]# 
[root@k8s-master etcd-operator]# kubectl api-resources
...
etcdclusters                      etcd         etcd.database.coreos.com       true         EtcdCluster

部署创建etcd集群

[root@k8s-master etcd-operator]# cat example/example-etcd-cluster.yaml
apiVersion: "etcd.database.coreos.com/v1beta2"
kind: "EtcdCluster"
metadata:
  name: "example-etcd-cluster"
  ## Adding this annotation make this cluster managed by clusterwide operators
  ## namespaced operators ignore it
  # annotations:
  #   etcd.database.coreos.com/scope: clusterwide
spec:
  size: 3  #集群数理
  version: "3.2.13"
[root@k8s-master etcd-operator]# kubectl apply -f  example/example-etcd-cluster.yaml
etcdcluster.etcd.database.coreos.com/example-etcd-cluster created
[root@k8s-master etcd-operator]# kubectl get pod -o wide
NAME                              READY   STATUS    RESTARTS   AGE    IP              NODE        NOMINATED NODE   READINESS GATES
etcd-operator-646cbffdb6-brbn6    1/1     Running   0          12m    192.168.51.58   k8s-node3   <none>           <none>
example-etcd-cluster-nc8pdgjrjr   1/1     Running   0          3m3s   192.168.51.59   k8s-node3   <none>           <none>
- 后面在加一个SVC就可以使用了

加载全部内容