Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 12 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
* [Deployment](#deployment)
* [User guide](#user-guide)
* [Usage of DANM's CNI](#usage-of-danms-cni)
* [Configuring DANM](#configuring-danm)
* [Network management](#network-management)
* [Generally supported DANM API features](#generally-supported-danm-api-features)
* [Naming container interfaces](#naming-container-interfaces)
Expand Down Expand Up @@ -90,7 +91,7 @@ CNI configurations are always transparent for Multus.

Whereas **DANM** is an E2E suite implementing one, big solution to a whole array of specific networking related features TelCo applications usually demand.
While Multus always delegates, DANM implements a lot of these functionalities on its own (but still being capable of delegating sub-tasks whenever it is configured to).
All components of DANM are mainly driven by and tightly coupled with a CRD-based proprietary Kubernetes API schema called DanmNet.
All components of DANM are mainly driven by and tightly coupled with a CRD-based Kubernetes network management API called DanmNet.
DANM actively interprets and mutates CNI configurations, even when it delegates network provisioning operations to other CNIs.

So, basically it is up to you to decide which solution is better suited to your needs. If you would like to remain close to the original networking ideals of Kubernetes, and only want to provision multiple network interfaces to your Pod, you might go with Multus.
Expand All @@ -100,7 +101,7 @@ If you are not afraid of going against the grain, and looking for a coupled E2E
### Prerequisites

As all the features of DANM are based on Kubernetes, you will need a Kubernetes cluster up-and running before you can use any components of the DANM suite. We suggest to use any of the automated Kubernetes installing solutions (kubeadm, minikube etc.) for a painless experience.
We currently test DANM with Kubernetes 1.11.X.
We currently test DANM with Kubernetes 1.14.X.
Compatibility with earlier than 1.9.X versions of Kubernetes is not officially supported.
Compatibility with newer versions of Kubernetes is not tested (theoretically it should work though, considering our project uses the official REST client generator created by the K8s community).

Expand Down Expand Up @@ -155,17 +156,11 @@ The method of deploying the whole DANM suite into a Kubernetes cluster is the fo
```
kubectl create -f integration/crds/
```
**2. Put the following CNI config file into the CNI configuration directory of all your kubelet nodes' (by default it is /etc/cni/net.d/):**
```
/ # cat /etc/cni/net.d/00-danm.conf
{
"name": "meta_cni",
"type": "danm",
"kubeconfig": "<PATH_TO_VALID_KUBECONFIG_FILE>"
}
```
**2. Put a valid CNI config file into the CNI configuration directory of all your kubelet nodes' (by default it is /etc/cni/net.d/) based on:**
[cniconfig](https://github.com/nokia/danm/tree/master/integration/cni_config/00-danm.conf)
The parameter "kubeconfig" is mandatory, and shall point to a valid kubeconfig file.
As kubelet considers the first .conf file in the configured directory as the valid CNI config of the cluster, it is generally a good idea to prefix the .conf file of any CNI metaplugin with "00".
Make sure to configure the optional DANM configuration parameters to match your environment!

**3. Copy the "danm" binary into the configured CNI plugin directory of all your kubelet nodes' (by default it is /opt/cni/bin/):**
```
Expand Down Expand Up @@ -204,6 +199,12 @@ We use Flannel for this purpose in our product. We also assume here the RBAC as
This section describes what features the DANM networking suite adds to a vanilla Kubernetes environment, and how can users utilize them.

### Usage of DANM's CNI
#### Configuring DANM
As DANM becomes more and more complex, we offer some level of control over the internal behaviour of how network provisioning is done.
Unless stated otherwise, DANM behaviour can be configured purely through its CNI configuration file.
The following configuration options are currently supported:
- cniDir: Users can define where should DANM search for the CNI config files for static delegates. Default value is /etc/cni/net.d
- namingScheme: if it is set to legacy container network interface names are set exactly to DanmNet.Spec.Options.container_prefix. Otherwise refer to [Naming container interfaces](#naming-container-interfaces) for details"
#### Network management
The DANM CNI is a full-fledged CNI metaplugin, capable of provisioning multiple network interfaces to a Pod, on-demand!
DANM can utilize any of the existing and already integrated CNI plugins to do so.
Expand Down
2 changes: 1 addition & 1 deletion crd/apis/danm/v1/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -128,4 +128,4 @@ type IpamConfig struct {
type IpamIp struct {
IpCidr string `json:"ipcidr"`
Version int `json:"version"`
}
}
6 changes: 3 additions & 3 deletions pkg/cnidel/cniconfs.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,11 @@ var (

//This function creates CNI configuration for all static-level backends
//The CNI binary matching with NetowrkType is invoked with the CNI config file matching with NetworkID parameter
func readCniConfigFile(netInfo *danmtypes.DanmNet) ([]byte, error) {
func readCniConfigFile(cniconfDir string, netInfo *danmtypes.DanmNet) ([]byte, error) {
cniConfig := netInfo.Spec.NetworkID
rawConfig, err := ioutil.ReadFile(cniConfigDir + "/" + cniConfig + ".conf")
rawConfig, err := ioutil.ReadFile(cniconfDir + "/" + cniConfig + ".conf")
if err != nil {
return nil, errors.New("Could not load CNI config file: " + cniConfig +".conf for plugin:" + netInfo.Spec.NetworkType)
return nil, errors.New("Could not load CNI config file: " + cniConfig +".conf for plugin:" + netInfo.Spec.NetworkType + " from directory:" + cniconfDir)
}
return rawConfig, nil
}
Expand Down
26 changes: 17 additions & 9 deletions pkg/cnidel/cnidel.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,21 @@ import (
"github.com/containernetworking/cni/pkg/types/current"
"github.com/containernetworking/cni/pkg/version"
"github.com/nokia/danm/pkg/danmep"
"github.com/nokia/danm/pkg/datastructs"
"github.com/nokia/danm/pkg/ipam"
danmtypes "github.com/nokia/danm/crd/apis/danm/v1"
danmclientset "github.com/nokia/danm/crd/client/clientset/versioned"
meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

const (
LegacyNamingScheme = "legacy"
)

var (
ipamType = "fakeipam"
defaultDataDir = "/var/lib/cni/networks"
flannelBridge = GetEnv("FLANNEL_BRIDGE", "cbr0")
cniConfigDir = GetEnv("CNI_CONF_DIR", "/etc/cni/net.d")
)

// IsDelegationRequired decides if the interface creation operations should be delegated to a 3rd party CNI, or can be handled by DANM
Expand All @@ -45,7 +49,7 @@ func IsDelegationRequired(danmClient danmclientset.Interface, nid, namespace str

// DelegateInterfaceSetup delegates K8s Pod network interface setup task to the input 3rd party CNI plugin
// Returns the CNI compatible result object, or an error if interface creation was unsuccessful, or if the 3rd party CNI config could not be loaded
func DelegateInterfaceSetup(danmClient danmclientset.Interface, netInfo *danmtypes.DanmNet, ep *danmtypes.DanmEp) (*current.Result,error) {
func DelegateInterfaceSetup(netConf *datastructs.NetConf, danmClient danmclientset.Interface, netInfo *danmtypes.DanmNet, ep *danmtypes.DanmEp) (*current.Result,error) {
var (
err error
ipamOptions danmtypes.IpamConfig
Expand All @@ -65,7 +69,7 @@ func DelegateInterfaceSetup(danmClient danmclientset.Interface, netInfo *danmtyp
return nil, errors.New("IPAM config creation failed for network:" + netInfo.ObjectMeta.Name + " with error:" + err.Error())
}
}
rawConfig, err := getCniPluginConfig(netInfo, ipamOptions, ep)
rawConfig, err := getCniPluginConfig(netConf, netInfo, ipamOptions, ep)
if err != nil {
freeDelegatedIps(danmClient, netInfo, ep.Spec.Iface.Address, ep.Spec.Iface.AddressIPv6)
return nil, err
Expand Down Expand Up @@ -128,14 +132,14 @@ func getCniIpamConfig(netinfo *danmtypes.DanmNet, ip4, ip6 string) (danmtypes.Ip
}, nil
}

func getCniPluginConfig(netInfo *danmtypes.DanmNet, ipamOptions danmtypes.IpamConfig, ep *danmtypes.DanmEp) ([]byte, error) {
func getCniPluginConfig(netConf *datastructs.NetConf, netInfo *danmtypes.DanmNet, ipamOptions danmtypes.IpamConfig, ep *danmtypes.DanmEp) ([]byte, error) {
cniType := strings.ToLower(netInfo.Spec.NetworkType)
for _, cni := range supportedNativeCnis {
if cni.BackendName == cniType {
return cni.readConfig(netInfo, ipamOptions, ep)
}
}
return readCniConfigFile(netInfo)
return readCniConfigFile(netConf.CniConfigDir, netInfo)
}

func execCniPlugin(cniType string, netInfo *danmtypes.DanmNet, rawConfig []byte, ep *danmtypes.DanmEp) (*current.Result,error) {
Expand Down Expand Up @@ -192,8 +196,8 @@ func setEpIfaceAddress(cniResult *current.Result, epIface *danmtypes.DanmEpIface

// DelegateInterfaceDelete delegates Ks8 Pod network interface delete task to the input 3rd party CNI plugin
// Returns an error if interface creation was unsuccessful, or if the 3rd party CNI config could not be loaded
func DelegateInterfaceDelete(danmClient danmclientset.Interface, netInfo *danmtypes.DanmNet, ep *danmtypes.DanmEp) error {
rawConfig, err := getCniPluginConfig(netInfo, danmtypes.IpamConfig{Type: ipamType}, ep)
func DelegateInterfaceDelete(netConf *datastructs.NetConf, danmClient danmclientset.Interface, netInfo *danmtypes.DanmNet, ep *danmtypes.DanmEp) error {
rawConfig, err := getCniPluginConfig(netConf, netInfo, danmtypes.IpamConfig{Type: ipamType}, ep)
if err != nil {
freeDelegatedIps(danmClient, netInfo, ep.Spec.Iface.Address, ep.Spec.Iface.AddressIPv6)
return err
Expand Down Expand Up @@ -260,13 +264,17 @@ func GetEnv(key, fallback string) string {
// CalculateIfaceName decides what should be the name of a container's interface.
// If a name is explicitly set in the related DanmNet API object, the NIC will be named accordingly.
// If a name is not explicitly set, then DANM will name the interface ethX where X=sequence number of the interface
func CalculateIfaceName(chosenName, defaultName string, sequenceId int) string {
// When legacy naming scheme is configured container_prefix behaves as the exact name of an interface, rather than its name suggest
func CalculateIfaceName(namingScheme, chosenName, defaultName string, sequenceId int) string {
//Kubelet expects the first interface to be literally named "eth0", so...
if sequenceId == 0 {
return "eth0"
}
if chosenName != "" {
return chosenName + strconv.Itoa(sequenceId)
if namingScheme != LegacyNamingScheme {
chosenName += strconv.Itoa(sequenceId)
}
return chosenName
}
return defaultName + strconv.Itoa(sequenceId)
}
12 changes: 12 additions & 0 deletions pkg/datastructs/datastructs.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
package datastructs

import (
"github.com/containernetworking/cni/pkg/types"
)

type NetConf struct {
types.NetConf
Kubeconfig string `json:"kubeconfig"`
CniConfigDir string `json:"cniDir"`
NamingScheme string `json:"namingScheme"`
}
Loading