配置
flannel从etcd读取其配置。
默认情况下，它将从中读取配置/coreos.com/network/config（可以使用进行覆盖–etcd-prefix）。
使用该etcdctl实用程序在etcd中设置值。
config的值是具有以下键的JSON key：
Network（字符串）：CIDR格式的IPv4网络，可用于整个flannel网络。（这是唯一的强制key）。例如使用10.244.0.0/16
SubnetLen（整数）：分配给每个主机的子网的大小。除非Network配置为小于/ 24，否则默认为24（即/ 24），在这种情况下，它比网络小1。
SubnetMin（字符串）：子网分配应从其开始的IP范围的开始。默认为的第一个子网Network。
SubnetMax（字符串）：子网分配应以IP地址结尾的范围。默认为的最后一个子网Network。
Backend（dictionary）：要使用的后端类型以及该后端的特定配置。有三种方式，vxlan，host-gw和udp。默认为udp后端。
生成的网络配置
[root@zhang1 ~]# cat /run/flannel/subnet.env
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.0.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true
查看app的pod
[root@zhang1 ~]# kubectl get po --namespace kube-system -l app=flannel -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-flannel-ds-8b5hq 1/1 Running 1 23h 192.168.103.87 node2 <none> <none>
kube-flannel-ds-tzn6x 1/1 Running 1 23h 192.168.102.235 node1 <none> <none>
kube-flannel-ds-zql4n 1/1 Running 2 23h 192.168.101.180 zhang1 <none> <none>
使用标签选择器进行选择。
查看flannel在k8s集群的配置，它是一个configmap
[root@zhang1 ~]# kubectl get configmap kube-flannel-cfg -n kube-system -o yaml
apiVersion: v1
data:
cni-conf.json: |
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
net-conf.json: |
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan"
}
}
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{"cni-conf.json":"{\n \"name\": \"cbr0\",\n \"cniVersion\": \"0.3.1\",\n \"plugins\": [\n {\n \"type\": \"flannel\",\n \"delegate\": {\n \"hairpinMode\": true,\n \"isDefaultGateway\": true\n }\n },\n {\n \"type\": \"portmap\",\n \"capabilities\": {\n \"portMappings\": true\n }\n }\n ]\n}\n","net-conf.json":"{\n \"Network\": \"10.244.0.0/16\",\n \"Backend\": {\n \"Type\": \"vxlan\"\n }\n}\n"},"kind":"ConfigMap","metadata":{"annotations":{},"labels":{"app":"flannel","tier":"node"},"name":"kube-flannel-cfg","namespace":"kube-system"}}
creationTimestamp: "2020-11-02T02:23:30Z"
labels:
app: flannel
tier: node
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
f:data:
.: {}
f:cni-conf.json: {}
f:net-conf.json: {}
f:metadata:
f:annotations:
.: {}
f:kubectl.kubernetes.io/last-applied-configuration: {}
f:labels:
.: {}
f:app: {}
f:tier: {}
manager: kubectl-client-side-apply
operation: Update
time: "2020-11-02T02:23:30Z"
name: kube-flannel-cfg
namespace: kube-system
resourceVersion: "24672"
selfLink: /api/v1/namespaces/kube-system/configmaps/kube-flannel-cfg
uid: cf48059d-2e8c-4e26-bdd3-ca69b1b88dde
路由Directrouting
在创建ConfigMap时，配置为”Directrouting”: true， 表示节点在同一个交换机下使用该配置，可以提高传输性能。
[root@node2 ~]# ip route default via 192.168.100.254 dev enp0s8 proto dhcp metric 100 10.244.0.0/24 via 192.168.101.180 dev enp0s8 10.244.1.0/24 via 192.168.102.235 dev enp0s8 10.244.2.0/24 dev cni0 proto kernel scope link src 10.244.2.1 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 192.168.100.0/22 dev enp0s8 proto kernel scope link src 192.168.103.87 metric 100
如果去掉该配置， 则路由为:
[root@node2 ~]# ip route default via 192.168.100.254 dev enp0s8 proto dhcp metric 100 10.244.0.0/24 via 10.244.0.0 dev flannel.1 onlink 10.244.1.0/24 via 10.244.1.0 dev flannel.1 onlink 10.244.2.0/24 dev cni0 proto kernel scope link src 10.244.2.1 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 192.168.100.0/22 dev enp0s8 proto kernel scope link src 192.168.103.87 metric 100
两者区别为，直接路由时，使用其实际物理网卡enp0s8。而间接路由，则使用虚拟的flannel.1网卡，报文使用VxLan进行封装。
直接路由不使用VxLan封装，性能会比使用Vxlan封装要好。直接路由这种情况两个Node节点不能有路由器， 因为报文直接使用了其原始docker 的IP地址，没有进行转换。
查看flannel的配置
查看flannel的配置, 首先进入flannel的docker容器
kubectl exec kube-flannel-ds-w8g89 -it sh --namespace kube-system
然后通过以下命令查看
/etc/kube-flannel # ls
cni-conf.json net-conf.json
/etc/kube-flannel # cat *
{
"name": "cbr0",
"cniVersion": "0.3.1",
"plugins": [
{
"type": "flannel",
"delegate": {
"hairpinMode": true,
"isDefaultGateway": true
}
},
{
"type": "portmap",
"capabilities": {
"portMappings": true
}
}
]
}
{
"Network": "10.244.0.0/16",
"Backend": {
"Type": "vxlan",
"Directrouting": true
}
}
查看flannel的log方法
通过logs命令来查看，需要指定名字空间和pod名称。
[root@zhang1 ~]# kubectl logs --namespace kube-system kube-flannel-ds-8b5hq
I1103 01:44:26.155115 1 main.go:518] Determining IP address of default interface
I1103 01:44:26.155854 1 main.go:531] Using interface with name enp0s8 and address 192.168.103.87
I1103 01:44:26.155999 1 main.go:548] Defaulting external address to interface address (192.168.103.87)
W1103 01:44:26.156047 1 client_config.go:517] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work.
I1103 01:44:26.350028 1 kube.go:119] Waiting 10m0s for node controller to sync
I1103 01:44:26.350091 1 kube.go:306] Starting kube subnet manager
I1103 01:44:27.350746 1 kube.go:126] Node controller sync successful
I1103 01:44:27.350793 1 main.go:246] Created subnet manager: Kubernetes Subnet Manager - node2
I1103 01:44:27.350806 1 main.go:249] Installing signal handlers
I1103 01:44:27.350909 1 main.go:390] Found network config - Backend type: vxlan
I1103 01:44:27.351042 1 vxlan.go:121] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
I1103 01:44:27.467240 1 main.go:355] Current network or subnet (10.244.0.0/16, 10.244.2.0/24) is not equal to previous one (0.0.0.0/0, 0.0.0.0/0), trying to recycle old iptables rules
I1103 01:44:27.756982 1 iptables.go:167] Deleting iptables rule: -s 0.0.0.0/0 -d 0.0.0.0/0 -j RETURN
I1103 01:44:27.762153 1 iptables.go:167] Deleting iptables rule: -s 0.0.0.0/0 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I1103 01:44:27.850654 1 iptables.go:167] Deleting iptables rule: ! -s 0.0.0.0/0 -d 0.0.0.0/0 -j RETURN
I1103 01:44:27.853937 1 iptables.go:167] Deleting iptables rule: ! -s 0.0.0.0/0 -d 0.0.0.0/0 -j MASQUERADE --random-fully
I1103 01:44:27.857760 1 main.go:305] Setting up masking rules
I1103 01:44:27.860323 1 main.go:313] Changing default FORWARD chain policy to ACCEPT
I1103 01:44:27.860495 1 main.go:321] Wrote subnet file to /run/flannel/subnet.env
I1103 01:44:27.860514 1 main.go:325] Running backend.
I1103 01:44:27.860534 1 main.go:343] Waiting for all goroutines to exit
I1103 01:44:27.860586 1 vxlan_network.go:60] watching for new subnet leases
I1103 01:44:27.871682 1 iptables.go:145] Some iptables rules are missing; deleting and recreating rules
I1103 01:44:27.953204 1 iptables.go:145] Some iptables rules are missing; deleting and recreating rules
I1103 01:44:27.953228 1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 -j ACCEPT
I1103 01:44:27.956196 1 iptables.go:167] Deleting iptables rule: -d 10.244.0.0/16 -j ACCEPT
I1103 01:44:27.958522 1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 -j ACCEPT
I1103 01:44:27.961947 1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN
I1103 01:44:27.964259 1 iptables.go:167] Deleting iptables rule: -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I1103 01:44:28.060345 1 iptables.go:155] Adding iptables rule: -d 10.244.0.0/16 -j ACCEPT
I1103 01:44:28.065458 1 iptables.go:167] Deleting iptables rule: ! -s 10.244.0.0/16 -d 10.244.2.0/24 -j RETURN
I1103 01:44:28.068364 1 iptables.go:167] Deleting iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE --random-fully
I1103 01:44:28.150471 1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN
I1103 01:44:28.156341 1 iptables.go:155] Adding iptables rule: -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE --random-fully
I1103 01:44:28.161135 1 iptables.go:155] Adding iptables rule: ! -s 10.244.0.0/16 -d 10.244.2.0/24 -j RETURN
I1103 01:44:28.252901 1 iptables.go:155] Adding iptables rule: ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE --random-fully