As readers of my previous article on DNS for desktop Linux will be able to attest, DNS management by systemd is complex. However, by investing time in understanding its complexity, we can create nuanced DNS resolution behaviors for specialized use cases.
To pull it back up, this installment will begin by detailing how systemd handles route queries. From there, I’ll describe how to configure DNS for each connection. Finally, I will reflect on why it is so difficult to get simple, consistent and actionable information on this topic.
Everything in its proper domain
Last time I casually mentioned that in the configuration of systemd-resolved “~.” is a special route-only domain that specifies the “default” link (or global) to use for DNS resolution. Let’s go into details.
In systemd-resolved’s split DNS, there are two ways of determining which queries go to which links/globally: lookup domains and routing-only domains. Both search and redirect-only domains are set to a domain string as a value (eg “linuxinsider.com”). Each link/global can have only one domain configured, either a route-only domain or a lookup domain (not both).
When a domain is submitted for a query, it is compared against all search and routing-only domains that have configured (ie, non-empty) values.
If (only) one of the configured search/route-only domain values is found within the queried domain, it is considered a “match” and the query is sent to the connection with the corresponding search/route-only domain.
If it is multiple search/route-only domains match, the query is sent to link/global whose search/route-only domain is the longest.
If none search domain match/route only, all connections and globally sending DNS queries. Since the whole point of DNS is that it is a distributed but consistent system, they you should they all give you the same answer, but it still drags unnecessary time and computing resources into the query.
Search domains and routes only are almost identical in functionality. In my research, I didn’t get as clear an answer about the difference as I had hoped. As the Fedora Magazine article and the Gnome Foundation blog post show, and the man page resolved with systemd confirms, the main difference is in the handling of single-label domains.
With search domains, if you search for a domain with a single tag without the “.” characters in it, the query is executed with the added value of the search domain. For example, if we have a search domain for “example.com” configured on a connection, and we query “mail”, the query sent will go to that connection and query as “mail.example.com”. By default, we can say that single-tag queries always try to match all configured search domains, and by definition, single-tag queries will not themselves match route-only or search-only domains because they have at least one ” .” in them.
Another significant difference is how search domains and only routes are configured. All configured per-link domains are search domains by default. Adding a “~” in front of a domain makes it a routing-only domain.
This brings us to our friend “~.”. As alluded to in the previous section, “~.” matches any query that does not match any other route-only domain or search domain. Thinking about it, it makes sense. This is the functional default because all domains (except single-label ones) will always match since they all have a “.” in them. But this domain match will always be shorter than any other possible route/search-only domain. So that’s the default value.
Query order and return conditions are even more complex than this. If you’re dying to know everything, open the “systemd-resolved” tutorial page linked above. But this is really all we need to understand systemd-resolved to a practical degree.
Reassembling all the split parts
We are now ready to understand the split DNS behavior of systemd-resolved.
To adapt the example from the Gnome Foundation piece, let’s take a common scenario of connecting to a business VPN. In Linux, VPNs (among other network configurations) can create a virtual network interface. Your computer treats it like any other network interface (eg a wireless card), but it’s virtual (ie software, not hardware). Furthermore, systematically solved also treats this virtual network interface as a connection. Therefore, it can have its own DNS configuration.
On a business VPN, you may need to query work-related domains against a DNS server on the network over the VPN tunnel. Well-implemented Linux VPN software will be designed to make the correct systemd-resolved API calls to configure its systemd-resolved connection to any route-only or lookup domain, as well as any DNS servers it needs.
As such, let’s say the VPN connection has “ecorp.com” as its search domain, and let’s say this is in addition to your wireless card having “~.” route-only domain. The result is that any domains you query that contain “ecorp.com” will go to the DNS servers set up on the VPN connection. All other DNS queries would go to the wireless servers.
So what happens if we leave out the “~.” outside our wireless connection? Domain queries containing “ecorp.com” or queries with a single tag will work the same as before. However, any domains that do not match the VPN connection search domain will be queried all connections/global. If you don’t want a corporate VPN to see where you’re browsing, it’s not desirable.
Missing link files
Although we established in the previous part of this series that if all you want to do is override the DNS servers provided by your Access Point (AP), changing the global configuration of systemd-resolved is the best. However, you may encounter situations where you need to adjust DNS behavior on a particular connection.
To do this, we turn to systemd-networkd, another daemon that controls network behavior. systemd-networkd manages all your network devices according to its own default settings, but you can give it custom instructions. If systemd-networkd finds files in the /etc/systemd/network directory with the correct filename conventions and file contents, it will do as you tell it.
Apart from your preferred DNS servers, all you need to know is the name of your network interface. To view the interface, run ip connection. For most desktop Linux installations, the only interfaces you’re likely to find are “lo” (“loopback”, self-reference), a wireless interface, and perhaps an Ethernet interface.
Interface name in hand, create a file for your interface (as superuser). /etc/systemd/network with a file name ending in “.network”. Set the content to the bottom, with some minor adjustments.
[Match]
Name=interface
[Network]
DHCP=yes
DNS=server1
DNS=server2
Domains=~.
[DHCPv4]
UseDNS=no
[DHCPv6]
UseDNS=no
Replace “interface” with the name you got ip connection, and “server1” and “server2” with the IP addresses of any DNS server you want. You must declare one server, but you can have a maximum of three.
There are a few things I want to shed some light on.
First, enabling “DHCP” is very important. DHCP is enabled by default if do not have a “.network” file for the interface. But when you create this file, it is disabled unless you enable it. This is important because the AP uses DHCP to give your computer an IP address on its network. Without it, your computer is functionally invisible.
Second, disabling “UseDNS” is equally important. This is a switch that determines whether your computer accepts the DNS servers given to it by the AP (“yes”) or not (“no”). Since our goal is to customize DNS, we need to tell the AP, “No thanks, I brought my own servers.”
The “DNS” and “Domains” items work like systemd-resolved.
After saving the file, reload systemd-networkd and systemd-resolved to reconfigure them. On my Linux Mint system, I noticed that systemd-networkd is not enabled by default. You can check if it is enabled by running it systemctl status systemd-networkd. If you find that it is disabled, run these commands with superuser privileges.
systemctl enable systemd-networkd
systemctl run systemd-networkd
systemctl restart systemd-solved
Otherwise, run them (also as superuser).
systemctl restart systemd-networkd
systemctl restart systemd-solved
The procedure for verifying your settings is the same as changing systemd-resolved’s global settings, so follow the steps in my previous article.
Looking for answers in the wrong places
My biggest motivation for writing these articles, more than teaching DNS configuration, was to illustrate the importance of verifying information and doing things the right way. There is a lot of bad advice on the subject of customizing Linux DNS. Here are a few examples I’ve come across.
Overwrite /etc/resolv.conf and then make the file immutable using Linux file permissions. Brute force methods like this are never wise. Since systemd-resolved is still running, your system is wasting resources writing to an immutable file. This may not work either because many Linux distributions forward queries to systemd-resolved via D-Bus rather than sending queries to the raw stub listener via /etc/resolv.conf.
Disable systemd-resolved to make /etc/resolv.conf static file again and then write your changes to it. As much as I’d love to revive the old ways, systemd is too embedded to go back. Moreover, important system components rely on systemd-resolved, so they might break if you turn it off.
Install dnsmasq for DNS caching. This is unnecessary because systemd-resolved already caches DNS responses. Why take up disk space and duplicate resources? I can understand that the average Linux user doesn’t know this, but can someone give this advice without knowing it’s redundant?
Final address
With this exercise behind me, I took away two lessons worth emphasizing.
First, if you’re looking for answers online, check them against the manual. Even studying the sources I cited, which were judged to be credible, I more compared them with the manual. Man pages are ground truth and come pre-installed, so why not check them out?
Second, the only way to really confirm your work is to read the record. I have problems with systemd, but one thing it does well is logging. Since different systemd components run as different system users, you can run journalctl and specify the user whose records you want.
I hope I did my job and I’m not the only one who learned something from this.
Suggest a topic
Is there a tutorial you’d like to see?
Email me your ideas and I’ll consider them for a future column.
And use the Reader Comments feature below to give your input!