k8s has multiple 3rd party plugins for ingress for advanced loadbalancing, e.g: Nginx, F5, etc.

Note: kubernets/ingress-nginx and nginxinc/kubernetes-ingress are different things. Here what we’re talking about all focusing on kubernets/ingress-nginx.

Now let’s take a look at ingress-nginx, see what are mandatory for it to work:

  1. ingress-nginx controller deployment or daemonset.
  2. Default backend svc along with its deployment.
  3. RBAC defined.
  4. properly configured configmap.
  5. use cloud loadbalancer(svc type loadbalancer for nginx controller) or external loadbalancer to mannually redirect traffic to ingress controller on NodePort.

So you would basically have traffic flow on ingress-nginx as: Internet -> Cloud LB|ingress-nginx controller svc -> real svc NodePort -> real pod exposed port you can also use NodePort instead: Internet -> FW NAT|ingress-nginx controller svc NodePort -> real svc NodePort -> real pod exposed port

The ingress process would be like:

  1. Ingress Controller responsible for redirecting traffic, so its Service will be the endpoint for user to connect;
  2. Default Backend is required, if nothing matches, traffic will hit it;
  3. Ingress defines redirecting rules. User hits Ingress Controller’s svc IP/Port, Ingress rule applies, if nothing matches it will be redirected to Default Backends.

Few places you can customize ingress-nginx:

  1. ingress-nginx controller deployment arg
  2. ingress-nginx configmap
  3. ingress annotation

Note:

  1. By default, ssl-redirect under configmap is true, which means all traffic hitting non https will be redirected to https, you can change it to false in configmap to disable this feature in a global range, or add annotation "nginx.ingress.kubernetes.io/ssl-redirect": "false" under individual ingress.
  2. Default backend can also be defined under controller deployment arg or ingress annotation.
  3. Multiple ingress controller to proxy different backend services is possbile by defining --ingress-class=nginx under args of controller deployment and recall it under ingress definition kubernetes.io/ingress.class: "nginx".
  4. The default helm deploy doesn’t have ingress-svc, sample should be like:
kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app: ingress-nginx
spec:
  externalTrafficPolicy: Local
  type: LoadBalancer
  selector:
    k8s-app: ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: http
  - name: https
    port: 443
    targetPort: https

Ingress Address

Ingress can read IP from k8s, either through nodeIP where nginx controller resides or through ExternalIP assigned by Loadbalancer. A trick is to use:

helm install --name nginx-ingress stable/nginx-ingress --set rbac.create=true --set controller.publishService.enabled=true

this way ingress will get correct exlb address.