配置 -------- flannel从etcd读取其配置。 默认情况下，它将从中读取配置/coreos.com/network/config（可以使用进行覆盖--etcd-prefix）。 使用该etcdctl实用程序在etcd中设置值。 config的值是具有以下键的JSON key： * Network（字符串）：CIDR格式的IPv4网络，可用于整个flannel网络。（这是唯一的强制key）。例如使用10.244.0.0/16 * SubnetLen（整数）：分配给每个主机的子网的大小。除非Network配置为小于/ 24，否则默认为24（即/ 24），在这种情况下，它比网络小1。 * SubnetMin（字符串）：子网分配应从其开始的IP范围的开始。默认为的第一个子网Network。 * SubnetMax（字符串）：子网分配应以IP地址结尾的范围。默认为的最后一个子网Network。 * Backend（dictionary）：要使用的后端类型以及该后端的特定配置。有三种方式，vxlan，host-gw和udp。默认为udp后端。 生成的网络配置 :: [root@zhang1 ~]# cat /run/flannel/subnet.env FLANNEL_NETWORK=10.244.0.0/16 FLANNEL_SUBNET=10.244.0.1/24 FLANNEL_MTU=1450 FLANNEL_IPMASQ=true 查看app的pod :: [root@zhang1 ~]# kubectl get po --namespace kube-system -l app=flannel -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-flannel-ds-8b5hq 1/1 Running 1 23h 192.168.103.87 node2 kube-flannel-ds-tzn6x 1/1 Running 1 23h 192.168.102.235 node1 kube-flannel-ds-zql4n 1/1 Running 2 23h 192.168.101.180 zhang1 使用标签选择器进行选择。 查看flannel在k8s集群的配置，它是一个configmap :: [root@zhang1 ~]# kubectl get configmap kube-flannel-cfg -n kube-system -o yaml apiVersion: v1 data: cni-conf.json: | { "name": "cbr0", "cniVersion": "0.3.1", "plugins": [ { "type": "flannel", "delegate": { "hairpinMode": true, "isDefaultGateway": true } }, { "type": "portmap", "capabilities": { "portMappings": true } } ] } net-conf.json: | { "Network": "10.244.0.0/16", "Backend": { "Type": "vxlan" } } kind: ConfigMap metadata: annotations: kubectl.kubernetes.io/last-applied-configuration: | {"apiVersion":"v1","data":{"cni-conf.json":"{\n \"name\": \"cbr0\",\n \"cniVersion\": \"0.3.1\",\n \"plugins\": [\n {\n \"type\": \"flannel\",\n \"delegate\": {\n \"hairpinMode\": true,\n \"isDefaultGateway\": true\n }\n },\n {\n \"type\": \"portmap\",\n \"capabilities\": {\n \"portMappings\": true\n }\n }\n ]\n}\n","net-conf.json":"{\n \"Network\": \"10.244.0.0/16\",\n \"Backend\": {\n \"Type\": \"vxlan\"\n }\n}\n"},"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app":"flannel","tier":"node"},"name":"kube-flannel-cfg","namespace":"kube-system"}} creationTimestamp: "2020-11-02T02:23:30Z" labels: app: flannel tier: node managedFields: - apiVersion: v1 fieldsType: FieldsV1 fieldsV1: f:data: .: {} f:cni-conf.json: {} f:net-conf.json: {} f:metadata: f:annotations: .: {} f:kubectl.kubernetes.io/last-applied-configuration: {} f:labels: .: {} f:app: {} f:tier: {} manager: kubectl-client-side-apply operation: Update time: "2020-11-02T02:23:30Z" name: kube-flannel-cfg namespace: kube-system resourceVersion: "24672" selfLink: /api/v1/namespaces/kube-system/configmaps/kube-flannel-cfg uid: cf48059d-2e8c-4e26-bdd3-ca69b1b88dde 路由Directrouting ------------------------ 在创建ConfigMap时，配置为"Directrouting": true， 表示节点在同一个交换机下使用该配置，可以提高传输性能。 [root@node2 ~]# ip route default via 192.168.100.254 dev enp0s8 proto dhcp metric 100 10.244.0.0/24 via 192.168.101.180 dev enp0s8 10.244.1.0/24 via 192.168.102.235 dev enp0s8 10.244.2.0/24 dev cni0 proto kernel scope link src 10.244.2.1 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 192.168.100.0/22 dev enp0s8 proto kernel scope link src 192.168.103.87 metric 100 如果去掉该配置， 则路由为: [root@node2 ~]# ip route default via 192.168.100.254 dev enp0s8 proto dhcp metric 100 10.244.0.0/24 via 10.244.0.0 dev flannel.1 onlink 10.244.1.0/24 via 10.244.1.0 dev flannel.1 onlink 10.244.2.0/24 dev cni0 proto kernel scope link src 10.244.2.1 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 192.168.100.0/22 dev enp0s8 proto kernel scope link src 192.168.103.87 metric 100 两者区别为，直接路由时，使用其实际物理网卡enp0s8。而间接路由，则使用虚拟的flannel.1网卡，报文使用VxLan进行封装。 直接路由不使用VxLan封装，性能会比使用Vxlan封装要好。直接路由这种情况两个Node节点不能有路由器， 因为报文直接使用了其原始docker 的IP地址，没有进行转换。 查看flannel的配置 ------------------------ 查看flannel的配置, 首先进入flannel的docker容器 :: kubectl exec kube-flannel-ds-w8g89 -it sh --namespace kube-system 然后通过以下命令查看 :: /etc/kube-flannel # ls cni-conf.json net-conf.json /etc/kube-flannel # cat * { "name": "cbr0", "cniVersion": "0.3.1", "plugins": [ { "type": "flannel", "delegate": { "hairpinMode": true, "isDefaultGateway": true } }, { "type": "portmap", "capabilities": { "portMappings": true } } ] } { "Network": "10.244.0.0/16", "Backend": { "Type": "vxlan", "Directrouting": true } } 查看flannel的log方法 --------------------- 通过logs命令来查看，需要指定名字空间和pod名称。 :: [root@zhang1 ~]# kubectl logs --namespace kube-system kube-flannel-ds-8b5hq I1103 01:44:26.155115 1 main.go:518] Determining IP address of default interface I1103 01:44:26.155854 1 main.go:531] Using interface with name enp0s8 and address 192.168.103.87 I1103 01:44:26.155999 1 main.go:548] Defaulting external address to interface address (192.168.103.87) W1103 01:44:26.156047 1 client_config.go:517] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. I1103 01:44:26.350028 1 kube.go:119] Waiting 10m0s for node controller to sync I1103 01:44:26.350091 1 kube.go:306] Starting kube subnet manager I1103 01:44:27.350746 1 kube.go:126] Node controller sync successful I1103 01:44:27.350793 1 main.go:246] Created subnet manager: Kubernetes Subnet Manager - node2 I1103 01:44:27.350806 1 main.go:249] Installing signal handlers I1103 01:44:27.350909 1 main.go:390] Found network config - Backend type: vxlan I1103 01:44:27.351042 1 vxlan.go:121] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false I1103 01:44:27.467240 1 main.go:355] Current network or subnet (10.244.0.0/16, 10.244.2.0/24) is not equal to previous one (0.0.0.0/0, 0.0.0.0/0), trying to recycle old iptables rules I1103 01:44:27.756982 1 iptables.go:167] Deleting iptables rule: -s 0.0.0.0/0 -d 0.0.0.0/0 -j RETURN I1103 01:44:27.762153 1 iptables.go:167] Deleting iptables rule: -s 0.0.0.0/0 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully I1103 01:44:27.850654 1 iptables.go:167] Deleting iptables rule: ! -s 0.0.0.0/0 -d 0.0.0.0/0 -j RETURN I1103 01:44:27.853937 1 iptables.go:167] Deleting iptables rule: ! -s 0.0.0.0/0 -d 0.0.0.0/0 -j MASQUERADE --random-fully I1103 01:44:27.857760 1 main.go:305] Setting up masking rules I1103 01:44:27.860323 1 main.go:313] Changing default FORWARD chain policy to ACCEPT I1103 01:44:27.860495 1 main.go:321] Wrote subnet file to /run/flannel/subnet.env I1103 01:44:27.860514 1 main.go:325] Running backend. I1103 01:44:27.860534 1 main.go:343] Waiting for all goroutines to exit I1103 01:44:27.860586 1 vxlan_network.go:60] watching for new subnet leases I1103 01:44:27.871682 1 iptables.go:145] Some iptables rules are missing; deleting and recreating rules I1103 01:44:27.953204 1 iptables.go:145] Some iptables rules are missing; deleting and recreating rules I1103 01:44:27.953228 1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 -j ACCEPT I1103 01:44:27.956196 1 iptables.go:167] Deleting iptables rule: -d 10.244.0.0/16 -j ACCEPT I1103 01:44:27.958522 1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 -j ACCEPT I1103 01:44:27.961947 1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN I1103 01:44:27.964259 1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully I1103 01:44:28.060345 1 iptables.go:155] Adding iptables rule: -d 10.244.0.0/16 -j ACCEPT I1103 01:44:28.065458 1 iptables.go:167] Deleting iptables rule: ! -s 10.244.0.0/16 -d 10.244.2.0/24 -j RETURN I1103 01:44:28.068364 1 iptables.go:167] Deleting iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE --random-fully I1103 01:44:28.150471 1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN I1103 01:44:28.156341 1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully I1103 01:44:28.161135 1 iptables.go:155] Adding iptables rule: ! -s 10.244.0.0/16 -d 10.244.2.0/24 -j RETURN I1103 01:44:28.252901 1 iptables.go:155] Adding iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE --random-fully