#14 Investigate systemd-networkd
Closed None Opened 5 years ago by mattdm.

  • What: new service to do very basic network config
  • Where: remove the sysv network scripts, add config for this
  • Why: remove very kludgey shell scripts; try the new hotness
  • When: Before F21 alpha
  • Who: Janez Nemanic

Just to let everybody know I am still working on this ticket. Slowly I am making progress.

Cool, thanks. Any particular blockers? We can try it in rawhide when you're ready.

Report on my work.

First I had to learn a little bit about systemd. What is all about, how does it work etc. Then I focused on networking subsystem in Fedora and tried to understand how it works. After setting my little lab environment I started with hands on approach.

My test environment:
- host system: 32 bit Fedora 20
- hypervisor: VirtualBox
- guest system: 32 bit Fedora 20
- systemd release: rawhide

My experiment and current findings

I disabled Gnome's NetworkManager, NetworkManager-dispatcher and NetworkManager-wait-online and dropped network configuration file into /usr/lib/systemd/network. The content of this file is:

[Match]
Name=p*

[Network]
DHCP=yes

Systemd names network interface p2p1 on my guest Fedora. So that's why I used Name=p*. That's the most simple and basic network configuration with systemd-networkd. I think that DHCP is also default configuration in desktop Fedora 20. I restarted systemd-networkd service and I could ping www.google.com and do DNS look up with dig. Ping and dig work as you would expect on my guest and on my host system.

I did some research on scripts in /etc/sysconfig/network-scripts. All those scripts depend on ifcfg- files in the same directory. But systemd-networkd doesn't use ifcfg- files. I removed ifcfg- files and systemd-netword worked fine. I was also able to ping google. But if I tried to use for example ifup or ifdown script then that script failed. Because it couldn't source ifcfg-. I think so, I might be wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I think we also don't need to use /etc/rc.d/init.d/network. I removed that file and networking worked fine with systemd-networkd.

So to sum up. If systemd-networkd service is used it is possible in my opinion to remove scripts in /etc/sysconfig/network-scripts and /etc/rc.d/init.d/network script. Besides that it is possible to remove NetworkManager.

'''Thoughts? Suggestions? Ideas?'''

One more thing that's been bothering me for a few days now. How does systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces?

For the first always use /etc/systemd/system never /usr/lib/systemd <-- this is for packages and will get overwritten flushed out on updates secondly we will be migrating everything to network units when the time is right as well as splitting out and removing the remaining parts of the legacy sysv initscript if that work wont be halted by traditionalists ( initscript contain for example the chkconfig/service command ) within and outside Red Hat as well as RHEL 8 since there seems to be somekind of internal RH dispute going on determine how the future of network should look like in products which is being kept on need to know bases.

Anyway split probably will end up somewhere along these lines

initscript

Requires changes in the FPG

initscripts-doc
initscripts-man
initscripts-locale
initscripts-network

Split into it's own package outside initscript

netconsole
netreport
ppp
systemd-fedora

These changes along with other necessary once that might never be allowed to be implemented in Fedora due to RHEL but should be, will land in a core/baseOS copr repo which should emerge as soon as copr starts supporting groups in FAS ( we dont want to tie this to a single individual ) which the cloud community could use for testing and what not and at that same time it will also probably emerge a test repository which will contain the latest systemd release for GA releases as well.

Replying to [comment:3 janeznemanic]:

Report on my work.
[...]

I did some research on scripts in /etc/sysconfig/network-scripts. All those scripts depend on ifcfg- files in the same directory. But systemd-networkd doesn't use ifcfg- files. I removed ifcfg- files and systemd-netword worked fine. I was also able to ping google. But if I tried to use for example ifup or ifdown script then that script failed. Because it couldn't source ifcfg-. I think so, I might be wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I think we also don't need to use /etc/rc.d/init.d/network. I removed that file and networking worked fine with systemd-networkd.

So to sum up. If systemd-networkd service is used it is possible in my opinion to remove scripts in /etc/sysconfig/network-scripts and /etc/rc.d/init.d/network script. Besides that it is possible to remove NetworkManager.

I took a quick look at the files from initscripts that are likely networking related. It seems possible to remote these from initscripts core and (as Johannes suggested above) possibly break initscripts out into a new package. If there was a desire to add the missing functionality to networkd that is currently in initscripts then the new initscripts package could be slowly whittled away to nothing when ppp and netconsole support are added to networkd (that appears to be the only missing functionality).

/etc/NetworkManager
/etc/NetworkManager/dispatcher.d
/etc/NetworkManager/dispatcher.d/00-netreport
/etc/networks
/etc/ppp
/etc/ppp/ip-down
/etc/ppp/ip-down.ipv6to4
/etc/ppp/ip-up
/etc/ppp/ip-up.ipv6to4
/etc/ppp/ipv6-down
/etc/ppp/ipv6-up
/etc/ppp/peers
/etc/rc.d/init.d/netconsole
/etc/rc.d/init.d/network
/etc/sysconfig/netconsole
/etc/sysconfig/network-scripts
/etc/sysconfig/network-scripts/*
/usr/bin/ipcalc
/usr/lib/udev/rename_device
/usr/lib/udev/rules.d/60-net.rules
/usr/libexec/initscripts
/usr/libexec/initscripts/legacy-actions
/usr/sbin/ifdown
/usr/sbin/ifup
/usr/sbin/netreport
/usr/sbin/ppp-watch
/usr/sbin/usernetctl
/usr/share/doc/initscripts/changes.ipv6
/usr/share/doc/initscripts/ipv6-6to4.howto
/usr/share/doc/initscripts/ipv6-tunnel.howto
/usr/share/doc/initscripts/static-routes-ipv6
/usr/share/man/man1/genhostid.1.gz
/usr/share/man/man1/ipcalc.1.gz
/usr/share/man/man1/netreport.1.gz
/usr/share/man/man8/ifdown.8.gz
/usr/share/man/man8/ifup.8.gz
/usr/share/man/man8/ppp-watch.8.gz}}}

'''Thoughts? Suggestions? Ideas?'''

One more thing that's been bothering me for a few days now. How does systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces?

Systemd will name the device based on the bus/device/function of the interfaces, so there could be some differences in the name chosen depending on the hypervisor used (since each hypervisor may chose a difference bus/dev/function for a given emulated interface.

Replying to [comment:3 janeznemanic]:

But if I tried to use for example ifup or ifdown script then that script failed. Because it couldn't source ifcfg-*. I think so, I might be wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I think we also don't need to use /etc/rc.d/init.d/network. I removed that file and networking worked fine with systemd-networkd.

Sorry, I forgot to address this in my last post. As was mentioned above there are some who will want to preserve some of the ability to use ifup/ifdown in some manner. I know that controlling the network interfaces is a current TODO for networkd, but it would seem reasonable that the networkd unit can be stopped/started or restarted if one wanted to reset the network config, correct?

Replying to [comment:5 gospo]:

One more thing that's been bothering me for a few days now. How does systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces?

Systemd will name the device based on the bus/device/function of the interfaces, so there could be some differences in the name chosen depending on the hypervisor used (since each hypervisor may chose a difference bus/dev/function for a given emulated interface.

It's important to note that the Fedora Cloud image, we disable this feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a single device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one).

Also, note that we are not using NetworkManager -- we're using the traditional network initscripts. It would be nice to eventually move away from these, and I don't have a strong view over which one should win (once NetworkManager gains the ability to configure a dhcp interface and then stop running).

From my (cloud SIG) point of view, NetworkManager has the disadvantage of being dependency heavy (and the run-once mode is not yet done -- see https://bugzilla.redhat.com/show_bug.cgi?id=863515), but the significant advantage of parsing the existing config files.

If there were a generator for systemd-networkd which could read the basic values from the legacy files and create network units on the fly (akin to systemd-fstab-generator), that would level that advantage. (No need to implement the full syntax; just basic static and dhcp, plus log when something isn't understood and perhaps recommend NetworkManager for this cases.) And ifup/ifdown compatibility scripts would be nice (although I don't think we generally care for the cloud case)

Replying to [comment:7 mattdm]:

It's important to note that the Fedora Cloud image, we disable this feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a single device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one).

I was thinking about the fact that the slot-naming-rules could be modified so that the first NIC found was named em1, second em2, etc, but disabling the rules completely seems fine.

As Matt points out we lack any kind of ifcfg-generator we also lack tunnelling support etc so I suggest people wait until the networkd code in systemd stabilizes before proceeding any further with this

Replying to [comment:6 gospo]:

Replying to [comment:3 janeznemanic]:

But if I tried to use for example ifup or ifdown script then that script failed. Because it couldn't source ifcfg-*. I think so, I might be wrong. Since ifup and ifdown are links to /usr/sbin/{ifup,ifdown} it might be possible to remove /usr/sbin/{ifup,ifdown}. But I haven't tested that so I'm not sure. I think we also don't need to use /etc/rc.d/init.d/network. I removed that file and networking worked fine with systemd-networkd.

Sorry, I forgot to address this in my last post. As was mentioned above there are some who will want to preserve some of the ability to use ifup/ifdown in some manner. I know that controlling the network interfaces is a current TODO for networkd, but it would seem reasonable that the networkd unit can be stopped/started or restarted if one wanted to reset the network config, correct?

You can do ''systemctl {start,stop,restart} systemd-networkd.service'' to control networking.
I'm not fedora networking expert, but I think it is possible to use ip command instead of ifup/ifdown to bring interface up or down.

Replying to [comment:7 mattdm]:

Replying to [comment:5 gospo]:

One more thing that's been bothering me for a few days now. How does systemd name network interfaces when Fedora is run on top of different hypervisors? If xen or kvm is used how does systemd name network interfaces?

Systemd will name the device based on the bus/device/function of the interfaces, so there could be some differences in the name chosen depending on the hypervisor used (since each hypervisor may chose a difference bus/dev/function for a given emulated interface.

It's important to note that the Fedora Cloud image, we disable this feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a single device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one).

That's a good news. It makes things a lot less complicated.

Replying to [comment:10 johannbg]:

As Matt points out we lack any kind of ifcfg-generator we also lack tunnelling support etc so I suggest people wait until the networkd code in systemd stabilizes before proceeding any further with this

Let's start collecting a list of basic requirements. Just to brainstorm a bit

  • Single eth0 interface with DHCP
  • Possible other interfaces, also with DHCP (common in OpenNebula)
  • Static ipv4 configuration
  • ipv6?
  • Nice-to-have: ifcfg compatibility (basic config files, ifup/ifdown commands)

Some use cases for basic use and for more advanced functionality like tunneling would help make a prioritized list.

Replying to [comment:8 mattdm]:

Also, note that we are not using NetworkManager -- we're using the traditional network initscripts. It would be nice to eventually move away from these, and I don't have a strong view over which one should win (once NetworkManager gains the ability to configure a dhcp interface and then stop running).

From my (cloud SIG) point of view, NetworkManager has the disadvantage of being dependency heavy (and the run-once mode is not yet done -- see https://bugzilla.redhat.com/show_bug.cgi?id=863515), but the significant advantage of parsing the existing config files.

If there were a generator for systemd-networkd which could read the basic values from the legacy files and create network units on the fly (akin to systemd-fstab-generator), that would level that advantage. (No need to implement the full syntax; just basic static and dhcp, plus log when something isn't understood and perhaps recommend NetworkManager for this cases.) And ifup/ifdown compatibility scripts would be nice (although I don't think we generally care for the cloud case)

Found out today on mailing list that cloud image is not using NetworkManager?. To me the idea of having both NetworkManager? and systemd-networkd in the cloud image is a little bit strange since both are network managers. I assume that NetworkManager? is superior to systemd-networkd, but I think systemd-networkd does it's job good in virtualized environment. Anyway I'm not an expert so that's just my opinion.

Replying to [comment:9 gospo]:

Replying to [comment:7 mattdm]:

It's important to note that the Fedora Cloud image, we disable this feature (ln -s /dev/null /etc/udev/rules.d/80-net-name-slot.rules), because systemd's priority is consistency of naming when there are multiple interfaces on the same system. By and large, our concern is with consistency of a single device name whereever the image boots, so the network interface is traditional eth0 (or eth* if more than one).

I was thinking about the fact that the slot-naming-rules could be modified so that the first NIC found was named em1, second em2, etc, but disabling the rules completely seems fine.

I had same idea.

Replying to [comment:13 mattdm]:

Replying to [comment:10 johannbg]:

As Matt points out we lack any kind of ifcfg-generator we also lack tunnelling support etc so I suggest people wait until the networkd code in systemd stabilizes before proceeding any further with this

Let's start collecting a list of basic requirements. Just to brainstorm a bit

  • Single eth0 interface with DHCP
  • Possible other interfaces, also with DHCP (common in OpenNebula)
  • Static ipv4 configuration
  • ipv6?
  • Nice-to-have: ifcfg compatibility (basic config files, ifup/ifdown commands)

Some use cases for basic use and for more advanced functionality like tunneling would help make a prioritized list.

Which other interfaces do you have in mind? Bridge?

So what's the plan here? How do we proceed? What shall I do?

Replying to [comment:17 janeznemanic]:

So what's the plan here? How do we proceed? What shall I do?

It might be useful to put together a variant of the kickstart file used to produce the current cloud image which uses systemd-networkd for eth0 instead of initscripts. See https://git.fedorahosted.org/cgit/spin-kickstarts.git/tree/fedora-cloud-base.ks and basically search for "network" for parts that might need to be changed.

Then, you should be able to feed that kickstart file into ImageFactory (or appliance-creator) and get a qcow2 image we can test.

I suppose that someone should file an RFE for systemd to have a "generator" (that's a typical systemd construct) for handling legacy ifcfg files (as above), and to create the ifup/ifdown compatibility scripts. Most ideally, someone will contribute that as patches to systemd upstream, because code always speaks louder than wishes :) but, just explaining what we need is a good start.

Ok, I will try to put together kickstart file and create test image.
Systemd generator is a good idea. We should keep ifup/ifdown scripts. I think they simplify things. It is possible to do the same with ip utility and related tools, but that's more complicated.
I was wondering which process is currently responsible for creating and modifying ifcfg-* files? I tried to find out that by myself but I can't find the answer.
Can someone explain to me what are scripts in /etc/ppp/ used for?
Are scripts in /etc/sysconfig/network-scripts/ the same in desktop Fedora and in cloud Fedora?

Here is the kickstart file. I added few lines at the end of file. Have a look at it and let me know if something is wrong.

Can't we just -initscripts (maybe you also need to list -dhclient, not sure which one is pulling in which) in %packages? Also ^services --enabled should not list network anymore, but systemd-networkd instead (I don't see where you active it and it seems to be disabled by default). Furthermore, there's a few things in %post that can now be removed completely ("Network fixes", writing if ifcfg-eth0).

Please be patient guys, I'm doing this for the first time. I published kickstart file here to get feedback, to discover what I'm doing wrong and how to improve that. I'll fix what's wrong.

Replying to [comment:21 red]:

Can't we just -initscripts (maybe you also need to list -dhclient, not sure which one is pulling in which) in %packages? Also ^services --enabled should not list network anymore, but systemd-networkd instead (I don't see where you active it and it seems to be disabled by default). Furthermore, there's a few things in %post that can now be removed completely ("Network fixes", writing if ifcfg-eth0).

I'm not sure if we can simply remove initscripts. There are files in initscripts used by udev, systemd and some scripts other than network-scripts.
I left lines that deal with writing of ifcfg- in the %post section, because as I understand we are planning to keep those files. Since we are planning to keep /etc/sysconfig/network-scripts and current network scripts depend on ifcfg- files. But if not needed those lines can be easily removed.

Replying to [comment:23 janeznemanic]:

I'm not sure if we can simply remove initscripts. There are files in initscripts used by udev, systemd and some scripts other than network-scripts.

Let's try it and see what breaks. On my test cloud instance, dhclient is the only thing that currently requires initscripts. If anything else requires it but doesn't say so, those packages should be fixed to either not have the requirement or have an explicit dependency.

I haven't looked into it myself, but it's my understanding that systemd has or will have its own dhcp client, which raises some concerns about compatibility testing but is also interesting on its own because on our current cloud image, dhclient is by far the largest single memory consumer. If systemd has a more lightweight dhcp client, that's a valuable plus in itself.

I left lines that deal with writing of ifcfg- in the %post section, because as I understand we are planning to keep those files. Since we are planning to keep /etc/sysconfig/network-scripts and current network scripts depend on ifcfg- files. But if not needed those lines can be easily removed.

Yeah, I think we do want that. These files are used by a lot of software, including, for example, Amazon's ec2-net-utils. And they're the common language between initscripts and NetworkManager (and hopefully systemd)

Replying to [comment:24 mattdm]:

I haven't looked into it myself, but it's my understanding that systemd has or will have its own dhcp client, which raises some concerns about compatibility testing but is also interesting on its own because on our current cloud image, dhclient is by far the largest single memory consumer. If systemd has a more lightweight dhcp client, that's a valuable plus in itself.

Before I've written comment:22, I removed both initscripts and dhclient from a F20 image that I upgraded to Rawhide, and it worked just fine with systemd-networkd (in a non-cloud VM, i.e. there also was no cloud-init).

Replying to [comment:25 red]:

Replying to [comment:24 mattdm]:

I haven't looked into it myself, but it's my understanding that systemd has or will have its own dhcp client, which raises some concerns about compatibility testing but is also interesting on its own because on our current cloud image, dhclient is by far the largest single memory consumer. If systemd has a more lightweight dhcp client, that's a valuable plus in itself.

Before I've written comment:22, I removed both initscripts and dhclient from a F20 image that I upgraded to Rawhide, and it worked just fine with systemd-networkd (in a non-cloud VM, i.e. there also was no cloud-init).

Let us try to remove initscripts and dhclient. Then we'll see what happens.

Systemd-networkd enabled, initscripts and dhclient removed. Why do we use rsyslog and not systemd's journald?
Let me try and build image with koji.

Replying to [comment:27 janeznemanic]:

Systemd-networkd enabled, initscripts and dhclient removed. Why do we use rsyslog and not systemd's journald?

Because cloud-init pulls it in. See https://fedorahosted.org/cloud/ticket/48 (I think this is another "help wanted" situation.)

RFE for the generator: https://bugzilla.redhat.com/show_bug.cgi?id=1090090

I didn't yet file one for ifup/ifdown scripts. That's probably best done as a new "ifupdown-scripts" package with an explicit conflicts with initscripts, and I'm really not quite clear on how it would work exactly (since networkd isn't really designed around fine control over individual interfaces).

Replying to [comment:8 mattdm]:

If there were a generator for systemd-networkd which could read the basic values from the legacy files and create network units on the fly (akin to systemd-fstab-generator), that would level that advantage. (No need to implement the full syntax; just basic static and dhcp, plus log when something isn't understood and perhaps recommend NetworkManager for this cases.) And ifup/ifdown compatibility scripts would be nice (although I don't think we generally care for the cloud case)

Sorry, but I miss why we would want to use the legacy files used by initscipts for networkd? If we are concerned about the ability to migrate from networkd to NM we should probably consider a plug-in for NM to read networkd files.

Replying to [comment:30 gospo]:

Sorry, but I miss why we would want to use the legacy files used by initscipts for networkd? If we are concerned about the ability to migrate from networkd to NM we should probably consider a plug-in for NM to read networkd files.

Many people are used to these files being the Fedora (and Red Hat distribution family) way to configure network interfaces. Their Puppet modules and Chef recipies work this way. In fact, the upstream code includes modules that understand this format natively. Tools like ec2-net-utils do too. All of our existing documentation talks about these files. And, it's already a common language between initscripts and Network Manager.

I don't think we would be doing anyone any favors by making Fedora a special case. In the future, if systemd-networkd becomes common and native tool support is widespread, it'll be easy to transition people.

In the future

Replying to [comment:31 mattdm]:

Replying to [comment:30 gospo]:

Sorry, but I miss why we would want to use the legacy files used by initscipts for networkd? If we are concerned about the ability to migrate from networkd to NM we should probably consider a plug-in for NM to read networkd files.

Many people are used to these files being the Fedora (and Red Hat distribution family) way to configure network interfaces. Their Puppet modules and Chef recipies work this way. In fact, the upstream code includes modules that understand this format natively. Tools like ec2-net-utils do too. All of our existing documentation talks about these files. And, it's already a common language between initscripts and Network Manager.

I don't think we would be doing anyone any favors by making Fedora a special case. In the future, if systemd-networkd becomes common and native tool support is widespread, it'll be easy to transition people.

Ah yes, I agree, then. Thanks for the explanation.

Help needed with koji. I'd like to build qcow2 cloud image. I think I found the right command, but that's where I get really stuck. So the command is:

koji image-build [options] <name> <version> <target> <install-tree-url> <arch> [<arch>...]

  1. options

The way I see it I need to use these options:
- format=qcow2
- kickstart=path to ks file
- no-wait

  1. What are the values of these arguments:

<name> = ??? (perhaps Fedora-cloud)
<version> = ??? (probably F21)
<target> = ??? (no idea here)
<install-tree-url> = ??? (no idea at all)
<arch> = ??? (probably i386 or x86_64)

Could someone give an example how to use koji image-build and a short explanation? So that I can build the above image.

janeznemanic - you need special permissions to build images in koji. We can check with release engineering about getting you permission to do at least scratch builds in this way.

Also, FESCo wants me to let you know that there is some skepticism with doing this change for f21, and that ifcfg compatibility is probably a must-have.

@mattdm - So I guess there is nothing else to do at the moment then to wait for generator.

This has been on my mind for some time now. I have been wondering if all the scripts currently in /etc/sysconfig/network-scripts are needed in cloud environment? For example do we need if{up,down}-{isdn,bnep} scripts in cloud image?

So, the version of NetworkManager in Fedora 22 has some exciting improvements -- it can do a "configure and exit" mode, is much more modular (no mandatory bluetooth or model support), and uses the same lightweight dhcp client from systemd. Since that currently already understands ifcfg files, that's kind of my first inclination for F22.

I have tested this on a running instance, will submit the F23 change page tomorrow. Just wondering if this should be a contained change or the system wide change?

Replying to [comment:40 kushal]:

I have tested this on a running instance, will submit the F23 change page tomorrow. Just wondering if this should be a contained change or the system wide change?

If we are talking about moving exclusively to systemd-networkd I don't think it makes sense as a systemwide change. There are a lot of benefits that NetworkManager gives for other use cases that make sense. systemd-networkd makes sense for cloud I think. The one use case that we had for systemd-networkd not behaving was cases where people plop down their own network files in the image (in /etc/sysconfig/network-scripts). Ideally we'd like to have systemd-networkd handle this and there was discussion of doing this (see https://bugzilla.redhat.com/show_bug.cgi?id=1194623).

While networkd is certainly suited to simple network setups like those of most clouds, we need to keep the consequences of using it within the broader context of Fedora in mind. Every other Fedora build, including Atomic, uses !NetworkManager to address the issues this feature aims to address, so choosing to use something else incurs costs that we should acknowledge:

  1. Network configuration will not be compatible with Fedora Workstation. As that is Fedora's development environment, this conflicts with the [https://fedoraproject.org/wiki/Cloud/Cloud_PRD Cloud PRD]'s goal of avoiding compatibility issues or surprises from differences in the environment. Perhaps we could make this change in concert with Workstation.
  2. Network configuration will not be compatible with Fedora Server. The cloudtoserver script could handle migration from networkd to NM, or the Server SIG could simply acknowledge that some fraction of Fedora Server users will wind up using networkd.
  3. Network configuration will not be compatible with Fedora Atomic. This may not actually be a big deal, but shipping two cloud products that use incompatible means of configuring networking is somewhat surprising.
  4. This adds another tree of potential errors and breakage. Using the same technology across multiple products benefits all of Fedora because they can share the testing burden. This is unfamiliar code; we will need additional test cases to cover it and it will take some time and effort for people to spin up on it so they can debug it and provide feedback more useful than, "It doesn't work."
  5. This adds another tree of documentation since networkd can't use NM configuration files.

We can address nearly all of these issues with a systemd [https://bugzilla.redhat.com/show_bug.cgi?id=1090090 generator] that reads the ifcfg-* files that NM uses. People won't be able to use nmcli, but ifcfg files are still well-understood and what we configure NM to emit anyway.

As for the additional testing burden, we probably just need to get QA buy-in. Has anyone from QA chimed in on this feature yet?

To be clear, I'm not against using networkd here, but the feature page is quite light on details like these that ought to be acknowledged.

NetworkManager has come a long way since I filed this, with a lot of work on paring down for minimal use cases.

Maybe a crazy idea, but perhaps we could do a systemd-networkd vs NetworkManager showdown for the F23 Alpha, and then decide on one for F23 Beta based on results.

In discussions about this we often bring up the need to have a set of requirements on network that the cloud images (and cloud environments) have. I have sent an email to the list [1] to solicit input and roshi has created a wiki page [2] for us to add information to.

[1] - https://lists.fedoraproject.org/pipermail/cloud/2015-August/005670.html
[2] - https://fedoraproject.org/wiki/Cloud/Network-Requirements

Kushal and I talked about this proposal in detail at Flock for the cloud images. While it's great for server devices that aren't going to be moving around often, it's not ideal for laptops or desktops that change their physical location frequently. NetworkManager makes a lot more sense there.

I spoke with zbyszek about improving the documentation around systemd-networkd as well. Much of the documentation is very focused and doesn't speak to full use cases where you're creating multiple .netdev and .network (and perhaps .link) files.

With that said, I've been using systemd-networkd instead of NetworkManager on 10-20 production cloud instances, over 250 hypervisors, and several other physical systems since Fedora 21. Other than the bonding shenanigans (fixed in systemd 216), I haven't seen any issues.

As a side note, just to add the mhayden's update. He's written a lot of great blog posts about using systemd-networkd in a number of use cases.

https://major.io/?s=networkd

Also, action item from 2015-09-23 Cloud SIG meeting is to request mhayden to do a formal write up of the comparison between NetworkManager and systemd-nspawn with different use cases and/or performance metrics (where applicable) since he appears to be the most experienced in the topic space.

https://twitter.com/TheMaxamillion/status/646737193839423488

https://meetbot.fedoraproject.org/fedora-meeting-1/2015-09-23/cloud_wg.2015-09-23-17.00.html
http://meetbot.fedoraproject.org/fedora-meeting-1/2015-09-23/cloud_wg.2015-09-23-17.00.txt

I'm happy to do this, but I'd like to understand exactly what I need to convey.

So we're talking about systemd-networkd vs. NetworkManager for containers and virtual machine guests and how they compare in:

  • Ease of use
  • Resource usage
  • Managing network interfaces once they're up/configured

(Feel free to add more bullets)

Should this be a wiki post? Blog post? Fedora docs?

FWIW, I'm putting together a talk proposal for RH Summit 2016 to teach/demo systemd-networkd. If anyone has any feedback on that, I'm all ears. ;)

During the Cloud WG meeting on 2015-01-06 it was decided that we come back to this ticket during Fedora 25 time.

Closing this ticket as part of trac clean up process. If you want to reopen, please reopen it after we move to pagure.io as atomic-wg.

Login to comment on this ticket.

Metadata