环境

rocky9.4 安装 Kubernetes v1.31.1 运行时 cri-o

k8s metallb bgp 配置，路由器routerOS

网络    cidr
主机网络    10.0.0.0/24,fd00::/64
podip    10.244.0.0/16,2000::/112
serviceip    10.96.0.0/12,3000::/112
loadbalanceip    10.0.1.0/24,fd22::/64

部署

首先主机网卡需要有IPv6地址，网关，dns，确保主机IPv6是正常的

开启内核参数

vim /etc/sysctl.d/k8s.conf

# 添加下面的参数
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.lo.disable_ipv6 = 0
net.ipv6.conf.all.forwarding = 1


sysctl --system 

修改kube-apiserver

vim /etc/kubernetes/manifests/kube-apiserver.yaml

--service-cluster-ip-range=10.96.0.0/12,3000::/112

修改kube-controller-manager

vim /etc/kubernetes/manifests/kube-controller-manager.yaml

--cluster-cidr=10.244.0.0/16,2000::/112
--service-cluster-ip-range=10.96.0.0/12,3000::/112
--node-cidr-mask-size-ipv4=24
--node-cidr-mask-size-ipv6=120

修改kube-proxy

kubectl edit cm -n kube-system kube-proxy

clusterCIDR: 10.244.0.0/16,2000::/64

重启kube-proxy容器

kubectl get pod -n kube-system | grep kube-proxy | awk '{print \$1}' | xargs kubectl delete pod -n kube-system

calico 配置双栈

编辑之前的custom-resources.yaml文件

vim custom-resources.yaml

# This section includes base Calico installation configuration.
# For more information, see: https://docs.tigera.io/calico/latest/reference/installation/api#operator.tigera.io/v1.Installation
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
  name: default
spec:
  # Configures Calico networking.
  calicoNetwork:
    ipPools:
    - name: default-ipv4-ippool
      blockSize: 26
      cidr: 10.244.0.0/16
      encapsulation: VXLANCrossSubnet
      natOutgoing: Enabled
      nodeSelector: all()
    - name: default-ipv6-ippool
      blockSize: 122
      cidr: 2000::/112
      encapsulation: None
      natOutgoing: Enabled
      nodeSelector: all()

---

# This section configures the Calico API server.
# For more information, see: https://docs.tigera.io/calico/latest/reference/installation/api#operator.tigera.io/v1.APIServer
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
  name: default
spec: {}

部署替换一下

kubectl replace -f custom-resources.yaml -n calico-system

metallb 添加v6

我的metallb是通过helm安装的，用的bgp，只需要修改IP pool

vim /opt/k8s/metallb/IPAddressPool.yaml

apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb
spec:
  addresses:
  - 10.0.1.1-10.0.1.100 # 可分配的 IP 地址
  - fd22::1-fd22::100

kubectl -n metallb replace -f IPAddressPool.yaml

验证

可以自己创建pod和svc验证，我就直接改已经存在的应用

例如：

[root@master01 ~]# kubectl -n ddns-go get svc
NAME           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)     AGE
ddns-ddns-go   ClusterIP   10.105.68.146   <none>        10168/TCP   5d23h
[root@master01 ~]# kubectl -n ddns-go get pod
NAME                            READY   STATUS    RESTARTS   AGE
ddns-ddns-go-6848678867-wwrwk   1/1     Running   1          6h57m
[root@master01 ~]# kubectl -n ddns-go delete pod ddns-ddns-go-6848678867-wwrwk

删除pod让他自己重建，然后查看信息就发现有v6的podip

[root@master01 ~]# kubectl -n ddns-go describe pod ddns-ddns-go-6848678867-wwrwk

Status:           Running
IP:               10.244.26.42
IPs:
  IP:           10.244.26.42
  IP:           2000::1a13

service需要修改如下：

ipFamilyPolicy: PreferDualStack
ipFamilies:
  - IPv4
  - IPv6

通过describe查看svc

kubectl -n ddns-go describe svc ddns-ddns-go 

IP Family Policy:         PreferDualStack
IP Families:              IPv4,IPv6
IP:                       10.105.68.146
IPs:                      10.105.68.146,3000::264a
Port:                     main  10168/TCP
TargetPort:               9876/TCP
Endpoints:                10.244.26.42:9876,[2000::1a13]:9876

ref:

calico

Elijah's Blog