Link Search Menu Expand Document

Ingress

Do you need to expose your services to the Internet? Or only internally? How is your DNS going to look like? Where do you get TLS certificates and how? How will that impact the security of your applications?

You should probably get answers to these (but not only) questions before you start planning the rest. Start with the big picture. Rancher is extremely easy to install so that’s going to be the least of your problems.

None of this is NetApp HCI or SolidFire specific, so you may plan without special considerations.

External

Use a network load balancer (software (Docker container or app on a VM, bare metal server) or hardware appliance) external to your Rancher VMs.

NGINX+ is a good choice for enterprise users. There’s a free version, too.

Internal

Internal networking is generally easier, but you still need control over internal DNS (over k8s.int.company.com for NLB and *.k8s.int.company.com for services, for example) and the ability to get (or issue) certificates signed by local CA. Self-signed certificates (and auto-created CA) are available but that doesn’t mean you should use them.

This post by NetApp’s partner Varonis has good tips about internal domain naming.

Default Networks on NetApp HCI

With VMware Virtual Standard Switch

TODO

With VMware Virtual Distributed Switch

These are created by NDE during deployment and should not be touched:

  • DVS_HCI_internal_mNode_Network
  • DVS_HCI_internal_Storage_Network
  • DVS_HCI_internal_vCenter_Network
  • DVS_HCI_vMotion
  • DVS_HCI_ISCSI_A
  • DVS_HCI_ISCSI_B

NetApp HCI has solution designs for VMware Private Cloud and other Kubenetes distributions which contain examples of network design for Kubernetes on NetApp HCI.

Rancher Management Cluster

This cluster must be able to communicate with at least the first three destinations in the list below:

  • vCenter of the VMware cluster in which it is installed
  • NetApp HCI Management Node (HCC services)
  • All Rancher Workload Clusters it manages (each may have a separate Management VLAN, but they could also share a common Kubernetes Cluster Management VLAN)
  • NetApp HCI Storage Management IP (i.e. SolidFire MVIP, if NetApp Trident is used for storage provisioning)
    • Also any other storage Management IP involved (ONTAP SVM Management IP, external NAS, etc.)
  • It may also need to be able to connect to a backup network, another cluster at the DR site, a gateway to connect to your Public Cloud provider, and so on

Rancher User Cluster(s)

These need to be able to communicate with:

  • The Rancher Management Cluster that manages it
  • Storage Network (Worker Nodes only, and only if these use iSCSI or NFS storage provisioned by NetApp Trident)
    • NFS could be ONTAP Select running on VMware or any ONTAP appliance reachable from Worker NodesA)
    • iSCSI could be NetApp HCI storage, NetApp E-Series (iSCSI), ONTAP (iSCSI)

Optionally these also may need to connect to other networks (for example to another cluster or service in a branch office or public cloud.)

DNS, NTP and other critical network services

As usual: make sure vSphere, HCC/mNode and Rancher requirements are satisfied.

If your services can’t work without Active Directory or outgoing Internet connectivity, take that into consideration as well.

Kubernetes Networking Options with Rancher

  • Flannel
  • Calico
  • Canal (Network Isolation option) - default
  • Weave

Network isolation may be preferred for K8s Projects exposed to the Internet.

The last three offer the option of modifying the MTU (CNI Plugin MTU Override). If your vSphere workload networks use MTU 9000, you can look into whether using non-default values helps (in terms of preformance, or avoid packet fragmentation, for example).

NetApp HCI enforces MTU 9000 on iSCSI networks while other networks can use MTU 1500. Rancher Worker VMs may use MTU 9000 on the networks for iSCSI, but they can also work with iSCSI interfaces set to VM defaults (MTU 1500, for example). Other interfaces may not be able to use jumbo frames if vSphere networking does not support them.

RancherOS could set MTU 9000 in cloud-init.yaml like so:

rancher:
  docker:
    extra_args: [--mtu, 9000]