DNS over HTTPS, DNS over TLS explained: Encrypting DNS traffic

Being the backbone of the internet, the Domain Name System (DNS) protocol has undergone a series of improvements and enhancements over the past few years. The lack of stringent protections in the original DNS specification and discovery of security weaknesses over time, such as the decade-old Kaminsky bug, gave birth to the Domain Name System Security Extensions (DNSSEC) in 2010.

DNSSEC was created to build cryptographic protections through digital signatures so that the DNS clients around the world could authoritatively verify that a DNS response was coming from an authoritative DNS server and that the response wasn’t altered in transit. 

Well then, some of you may wonder if DNSSEC can adequately provide security, what is the need for DNS over HTTPS and DNS over TLS?

DNSSEC only ensures the authenticity of the DNS responses and data integrity but does not ensure privacy. Protocols like DNS over HTTPS (DoH) or DNS over TLS (DoT) provide end-to-end encryption, therefore guaranteeing data confidentiality. In other words, your DNS traffic now benefits from the same end-to-end encryption as your web traffic to and from HTTPS sites.

What is DNS over HTTPS?

By default, the DNS protocol functions over the User Datagram Protocol (UDP), a transport layer protocol, although DNS could also be made to run over Transmission Control Protocol (TCP). DoH transmits DNS messages encrypted over HTTPS as opposed to the faster UDP. Since HTTPS is the HTTP protocol running over TLS (Transport Layer Security), DoH, in effect, is DNS over HTTP over TLS.

With DoH, both the DNS queries and DNS responses are transmitted over HTTPS and use port 443, making the traffic virtually indistinguishable from any other HTTPS web traffic. For example, using Google’s DoH service, you could try resolving the CSOOnline.com domain yourself right from your web browser. The manner of typing an HTTPS URL in your browser to resolve domain name over HTTPS feels akin to visiting any regular website which uses SSL/TLS.

The DNS response returned by Google shows CSO’s server IP address (A records), all packed neatly in the JSON format.

Ax Sharma

While the fact that you are using Google’s DoH service may become known to a network administrator, assuming a corporate man-in-the-niddle (MitM) proxy is not present between you and Google’s DoH, no one would be able to ascertain what domain you were trying to look up (CSO Online) or the response to your DNS query (the JSON result). Therefore, using DoH guarantees both your privacy (data confidentiality) and the integrity of the information received—i.e., DNS response was not tampered with in transit. 

Delivering DNS over an end-to-end encryption channel can also become problematic. For example, attackers have previously abused DNS over HTTPS services to hide malicious traffic.

The attackers would resolve their malicious domain over Google’s DoH, or any DoH provider for that matter. The encrypted response returned would contain TXT records for the attacker-controlled domain with encoded malicious payload that could then be parsed by malware. Essentially, this is how threat actors can abuse secure DNS protocols for facilitating their command-and-control (C2) activities. Because DoH providers have legitimate business use cases, it would be challenging to simply block traffic inbound or outbound traffic between your corporate networks and DoH providers.

What is DNS over TLS?

DoT encrypts DNS queries over the TLS protocol (at the transport layer), rather than HTTPS that lives on the application layer. Unlike DoH, DoT skips one layer in between, the application-level HTTPS.

Basically, DoT encrypts DNS UDP requests and responses over TLS and ensures that these messages are not altered during the course of their transmission. DoT uses a different port altogether: 853—separate from the ports used by HTTPS (port 443) or plain old DNS (port 53). Like with DoH, DoT traffic benefits from end-to-end encryption because the communication between a DNS client and resolver takes place over TLS.

Which DNS protocol is better?

Whether DoH is better than DoT up for debate. Network administrators may express a slight preference toward DoT as it gives them more flexibility when it comes to monitoring DNS queries. This can be especially useful when security professionals want to block malicious DNS traffic and indicators of compromise (IOCs) from their network.

DoH offers a heightened privacy for the end user as the user’s DNS queries are now mingled alongside other HTTPS traffic, and the network administrator now can’t ascertain what domains are being resolved or the DNS responses being returned. This, unfortunately, also means it is much harder for network administrators to block DoH without impacting legitimate business communications.

For example, enterprise firewalls can easily be configured to add a policy that universally filters traffic travelling over port 853 (for blocking DoT), but filtering port 443 (for DoH) is simply not a practical option as it would block most legitimate web traffic.

Another point to note, DoT is very slightly more lightweight as it lives on the transport layer, whereas DoH has HTTPS living on the application layer. The fewer layers involved are bound to result in a smaller size of a DoT packet and perhaps a small performance boost (lower latency) when compared against DoH.

The battle does not stop at DoT vs. DoH. Further developments in the space by network infrastructure companies like Cloudflare have also extended DNS suite to include newer “fun” protocols, such as, DNS over Twitter, DNS over Tor, DNS over Telegram or DNS over email.

Cloudflare offers an Onion service that allows visitors to its customers’ sites to use the Tor network. Cloudflare’s resolver 1.1.1.1 supports both DoH and DoT and is available through its Onion service. “We call it DNS-over-Tor. We also operate a Twitterbot that listens for specifically formatted Tweets to @1111Resolver, converts them into DNS queries, resolves the queries with 1.1.1.1, and Tweets back the result,” says Nick Sullivan, head of research at Cloudflare.

The newly devised protocols, a lot of which use encrypted channels like HTTPS or SOCKS (Tor) have yet again introduced more options for the end user but could pose more challenges for security professionals when it comes to filtering out traffic. For example, blocking DNS over Twitter, could mean blocking Twitter altogether, unless a more sophisticated workaround is adopted. But using a workaround such as an enterprise MitM proxy nullifies many protections offered by DoH with regards to confidentiality and user privacy.

Therefore, using any DNS protocol, whether DoT or DoH comes down to your organization’s needs and what defines an acceptable tradeoff between user privacy and reasonable network monitoring.