More work on the kubecon slides

Author: kristenjacobs

slides/201810-voxxeddays/slides.key

@@ -1,16 +1,22 @@
 # Container Networking Talk Notes
 
-* Motivation. Why am I doing this?
-  Some time back I looked into updating the networking layer in the Oracle managed 
-  networking service from using Flannel (an overlay network) to a solution which utilises
-  the native networking features of the Oracle cloud (secondary VNICs + IPs). However, once
-  I started digging in, I quickly found that I didnt understand the current solution...
-   
-* Prerequsites. Here I am aiming to describe *container* networking from scratch. 
-  However, some networking concepts such as L2 vs. L3, subnets, CIDR ranges are assumed.
-  However, I'll try my best to briefly describe these as we go..
-
-* No expert though, i.e. at the end of this talk you'll know everything I know about container networking!
+* I work in the Oracle cloud infrastruture group, more specifically on
+  Kubernetes related stuff, and some time back I was given the task of looking
+  into updating the networking layer in the Oracle managed Kubernetes service
+  from using Flannel (an overlay network) to a solution which utilises the native
+  networking features of the Oracle cloud (secondary VNICs + IPs). Dont worry if
+  you dont know what Flannel is, or know what an overlay network is, as that is
+  the point of this talk!  However, once I started digging in, I quickly found
+  that I didn't understand how Flannle worked, and as it seemed a little wrong to
+  replace one thing with another solution, if you dont understand how the
+  original worked, I started digging deeper, and then relaised that I dont
+  understand networking in general!  Long story short, big rabitt hole, learnt
+  some stuff, and most importantly found that I really enjoyed this, so I thought
+  I would write a talk and come and spread the networking love!
+  
+* So, I'm Kris, and in the next 30 minutes or so, I'm going to attempt to explain
+  how a contaniner on one computer on the internet, can talk to a container on 
+  another computer, somewhere else on the internet.
 
 ## Slide: The aim
 
@@ -19,34 +25,29 @@
     * No NAT'ing going on.
     * Host can talk to containers, and vice versa.
 
-* Contrast this with the default docker approach (RHS diagram).
-    * i.e. Only containers on a node have unique IP addresses.
-    * Processes inside containers accessed via port mapping (IP tables). 
+* Note: We are not covering the default docker model here, where
+  containers on different nodes can have the same IPs.
 
 ## Slide: The plan
 
-* Summarise the 4 steps.
+* Going to work our way toward the general case in 4 steps.
+
+* Foreach, we will explain the model via a diagram. Show some code, run the code, 
+  then test what we have created. 
 
-* Summarise the demo setup, i.e. using pre-prepared/up vagrant environments.
+* Each step we be created using vagrant based VMs.
+
+* Summarise the 4 steps.
 
 ## Slide: Single network namespace diagram
 
 * Describe the outer box (the node). Could be a physical machine, or a VM as in this case.
 
 * Describe containers vs namespaces:
-    * What is a container:
-         * Cgroups: What a process can do. E.g:
-             * Restrict memory
-             * Restrict CPU
-             * Restrict network bandwidth
-
-         * Apparmour/secconf/capabilities: Security layer. E.g:
-             * Restrict the system calls the contained process has access to.
-
-         * Namespaces: What a process can see. E.g.
-             * Mount namespace: Controls which parts of the file system the contained process can see.
-             * Process namespace: Controls which other processes the contained process can see.
-             * Network namespace: See below...
+    * Containers use a bunch of different linux mechanisms to isolate the processes running inside,
+      both in terms of system calls, available resources, what it can see, i.e. filesystems, other processes, etc.
+      However, from a network connectivity point of view, the only mechanism that matters here is the network
+      namespace, so from now on, whenever I say container, what I really mean is network namespace.
 
     * What is a network namespace:
         * It's own network stack containing: 
@@ -56,9 +57,13 @@
         * When created, it is empty, i.e. no interfaces, routing or IP tables rules.
 
 * Describe VETH pair: Ethernet cable with NIC on each end.
-* Describe the relevant routing from/to the network namespace, including the 
-  types of each of the routing rules used here. Note The 'aha' monment, when I worked out
-  the possible types of routing rules => Key takeaway, understanding these is key! 
+
+* Describe the relevant routing from/to the network namespace:
+    * Directly connected route from the host to the network namespace.
+    * Default route out of the network namespace.
+
+* Note The 'aha' monment, when I worked out the possible types of routing rules 
+  => Understanding these was for me the key to understanding networking in general.
 
 ## Code: Single network namespace setup.sh
 
@@ -71,48 +76,29 @@
 
 ```
 ./setup.sh
-# The interfaces + routes inside the network namespace
+# The interfaces inside the network namespace
 sudo ip netns exec con ip a
-sudo ip netns exec con ip r
-# The interfaces + routes on the node
-ip a
-ip r
 # Pings the network namespace from the node
 ping 176.16.0.1
-# Pings the node from the network namespace
-sudo ip netns exec con ping 10.0.0.10
 ```
 
 * What is actually responding to the pings in these cases, as there is no process running 
-  inside the namespace who can respond in this case?
-  Do a quick dive into ICMP here. It is a layer 3(.5?) protocol, i.e. we have 
-  an ICMP header inside of the IP packet, which defines a bunch of bits used in managing
-  IP packetes. e.g.
-     - Reporting that TTL has expired - More on this later...
-     - Reporting that we need to fragment, but the DF bit is set - again, more on this later.
-     - Bits for echo request and echo response (A.K.A ping).
-  Therefore, in this case, it is the network stack in the kernel that is reponding to the 
-  ICMP echo request packet, with a ICMP echo request packet.
-
-For a more realistic example, We can run one (or more) real process in the network namespace 
-(e.g. the python file server), and can the curl this from the node:
+  inside the namespace who can respond in this case? It is the kernel network stack 
+  inside of the network namespace that is responding to these IMCP echo requests, with 
+  an ICMP echo request packet.
 
-```
-# Runs the python file server in the background on port 8000, inside the network namespace
-sudo ip netns exec con python3 -m http.server 8000 &
-# Curls the python file server from the node
-curl 172.16.0.1:8000
-```
+* For a more realistic example, We would run one (or more) real process in the network namespace. 
+  However, for the purposes of testing connectivity, pinging is enough.
 
-Note: you can run multiple processes inside a network namespace, which roughly corresponds to a Kubernetes pod.
+* Note: you can run multiple processes inside a network namespace, which is what happens inside Kubernetes pods.
 
 ## Slide: Diagram of multiple network namespaces on the same node
 
-* Describe the Linux bridge:
-    * A single L2 broadcast domain, much like a switch, implemented in the kernel.
+* Describe the Linux bridge: A single L2 broadcast domain, a virtual ethernet switch, implemented in the kernel.
 * The bridge now has its own subnet.
 * The bridge also has its own IP: Allows access from the outside.
 * Describe the route for the subnet.
+* Note: This corresponds to the default docker0 bridge.
 
 ## Code: Multiple network namespace setup.sh
 
@@ -125,26 +111,16 @@ Note: you can run multiple processes inside a network namespace, which roughly c
 
 ```
 ./setup.sh
-# The interfaces + routes inside a network namespace
-sudo ip netns exec con1 ip a
-sudo ip netns exec con1 ip r
-# The interfaces + routes on the node
+# The interfaces on the node
 ip a
-ip r
 # Pings between the network namespaces
 sudo ip netns exec con1 ping 172.16.0.3
 # Pings the node from the network namespace
 sudo ip netns exec con1 ping 10.0.0.10
 ```
 
-* When we ping between the network namespaces:
-    * Highlight the TTL. Should be the default value, thus no routing is going on here!
-    * Describe what the TTL is, and what happens when the TTL reaches zero.
-    * Describe how the TTL is used, e.g. in the implementation of traceroute.
-* When we ping network namespace from node:
-    * Highlight the TTL. Should be the same.
-* Mention that currently we cant get external traffic to the namespaces, as we are not fowarding IP packets. 
-  However, we will set this up in the next example.
+* Highlight the TTL. Should be the default value, thus no routing is going on here!
+* Describe what the TTL is, and what happens when the TTL reaches zero.
 
 ## Slide: Diagram of multiple network namespaces on different nodes but same L2 network
 
@@ -161,34 +137,27 @@ sudo ip netns exec con1 ping 10.0.0.10
 * Talk through the *setup.sh*.
     * Describe the parts common to the previous step.
     * Describe the setup of the extra routes.
-* Explain the IP forwarding.
-    * What does this do/why is it needed: Turns your Linux box into a router.
-    * Is enabling this a security risk: Maybe, but it is required in this case!
+* Explain the IP forwarding: Turns your Linux box into a router, which is 
+  required in this case as the node has to forward the packets for any network 
+  namespaces that live on that node.
 
 ## Demo: Multi node
 
 On each node, run:
 
 ```
 ./setup.sh
-# The routes on the node
-ip r
 ```
 
-From 10.0.0.20:
-
-```
-# Captures ICMP packetes on the veth10 interface connected to the bridge
-sudo tcpdump -ni veth10 icmp
-```
-
-Then from 10.0.0.10:
+From 10.0.0.10:
 
 ```
+# The routes on the node
+ip r
 # Pings from a network namespaces on one node to one on the other node
 sudo ip netns exec con1 ping 172.16.1.2
-# Pings the same network namespace from the node
-ping 172.16.1.2
+# Pings from a network namespaces on one node to the other node
+sudo ip netns exec con1 ping 10.0.0.20
 ```
 
 * When we ping from a network namespaces to another network namespace across nodes:
@@ -223,20 +192,13 @@ ping 172.16.1.2
 * Talk through the *setup.sh*. 
     * Describe the parts common to the previous step.
     * We need packet forwarding enabled here. This allows the node to act as a router, i.e.
-      to accept and forward packets recieved, but no tdestined for, the IP of the node.
+      to accept and forward packets recieved, but not destined for, the IP of the node.
     * Now no extra routes, but contains the socat implementation of the overlay.
 * Describe *socat* in general. It creates 2 bidirectional bytestreams, and transfers data between them.
 * Describe how *socat* is being used here. 
-* Describe how this is similar to a VPN: How could we construct a virtual network
-  between 2 hosts using socat (creating a VN!). For example, start the VN 'server' on the destination
-  network using a tun device and the UDP tunnel. Start the VN 'client', with the same setup. Connect the UDP
-  tunnel, then assign the tun device an address on the desination network. Just add encryption to this, and you'll
-  have your very own VPN!
 * Note the MTU settings, what is going on here? We reduce the MTU of the tun0 
   device as this allows for the 8 bytes UDP header that will be added, thus ensuring that 
   fragmentation does not occur.
-* Describe the scheme that is used to ensure that the kernel chooses packet sizes that dont cause fragmentation
-  (using the DF bit in the IP packets, and the 'Fragmentation required' ICMP response.
 * Reverse packet filtering:
     * What is this: Discards incoming packets from interfaces where they shouldn't be.
     * It's purpose: A security feature to stop IP spoofed packets from being propagated.
@@ -245,9 +207,6 @@ ping 172.16.1.2
       tunnel. However, the response will not (as it is destined for the node), thus the response
       will emerge on a different interface to which the request packet went. Therefore, the kernel 
       consider this suspicious, unless we tell it that all is ok.
-    * Is it OK to turn this off? Again, maybe, as this is primarily for DOS attacks, thus aimed at 
-      routers on the public internet. The alternative is to ensure that packets from network 
-      namespaces to remote nodes also go via the overlay (which would involve src based routing!)
 
 ## Demo: Overlay network
 
@@ -257,20 +216,13 @@ On each node, run:
 ./setup.sh
 ```
 
-From 10.0.0.20:
-
-```
-# Captures ICMP packetes on the veth20 interface connected to the bridge
-sudo tcpdump -ni veth10 icmp
-```
-
 From 10.0.0.10:
 
 ```
 # Ping from a network namespaces on one node to one on the other node
 sudo ip netns exec con1 ping 172.16.1.2
-# Pings the same network namespace from the node
-ping 172.16.1.2
+# Ping from a network namespaces on one node to the other node
+sudo ip netns exec con1 ping 10.0.0.20
 ```
 
 * When we ping from a network namespace to a network namespace across nodes: