Ian J MacIntosh.com

DNS for Web Developers

Imagine you go to your office, open up your web browser, and navigate to 172.217.172.197 to check your email. Then you visit 140.82.112.4 to review some code changes your coworkers have proposed. Then you open your favorite chat client and connect to 84.17.44.180. If you haven’t figured it out yet: this sucks. This is a world without domain names.

Weʼre more accustomed to typing in gmail.com, github.com, and chat.freenode.net, but just like you need your friendʼs phone number to call them, your web browser needs to know the IP address of the server to request pages from. Fortunately, we have a system for finding the IP address for a given domain name that works so seamlessly that we almost never have to think about it.

That also means I used to have no clue how any of it actually works, so whenever someone would talk to me about changing DNS settings, Iʼd get a sense of dread; “itʼs like a phone book for the internet” doesnʼt help me understand how record caching affects recursive lookups.

If youʼve ever had that feeling, I want to help you get rid of it. Maybe youʼre a web developer working for a company that provides a supporting team of network engineers to maintain everything below the application layer, or maybe you’re in a position where you’re expected to know literally everything from color theory to algorithms. Either way, you should have a basic understanding of how DNS lets your site’s visitors connect to your app.

⌨️ Locate the address record for www.ianjmacintosh.com

Letʼs say I want to visit www.ianjmacintosh.com. For my browser to see it, my browser needs to know the IP address to request it from. First, my browser will check to see if it already has a cached record of www.ianjmacintosh.comʼs IP address. This record is called an address record, or an A record. If I donʼt have an A record on hand, my browser requests one from my DNS resolver. DNS resolvers provide clients with whatever records they request, and every major ISP provides one. Iʼm going to demonstrate how to use a command line tool (dig, short for Domain Information Groper) to manually request records from Googleʼs public DNS resolver and other name servers.

dig performs DNS lookups and displays the answers that are returned. Most DNS administrators use dig to troubleshoot DNS problems because of its flexibility, ease of use and clarity of output. It comes pre-installed on most Mac OS X and Linux systems, but Windows users who want to use it need to install it manually. If youʼre reading this on another OS (like Android or iOS) you can try to follow along with a web-based dig client like this random one I found, but youʼll probably get more out of this article by reading for now and running the commands later at a computer with a local install of dig.

Try your first dig query:

dig @8.8.8.8 www.ianjmacintosh.com A +short

This invocation queries 8.8.8.8 (the IP address for Googleʼs public DNS resolver) for www.ianjmacintosh.comʼs A (address) record. The +short query option asks dig to provide a terse answer.

That terse answer looks like this:

54.207.147.214
18.230.52.212

Those are the IP addresses that Googleʼs DNS resolver said I should reach out to if I want www.ianjmacintosh.com.

Even if you stop reading now, at least you saw how to quickly get the IP address associated with a domain name.

DNS Trees

Imagine you have a path on your local filesystem: /Users/alice/projects/next-great-american-novel. The next-great-american-novel directory is in the projects directory, which is in the alice directory, which is in the Users directory, which is in the root (/) directory. Computer scientists call this abstract idea a tree.

The relationship between com, ianjmacintosh, and www is the same, even though itʼs written the right-to-left instead of left-to-right. Instead of traversing directories on our local filesystem, we query DNS servers on remote systems. These servers are called name servers.

To get the A record for www.ianjmacintosh.com, the DNS resolver asks a name server it already trusts about com, which points it to another name server that knows all about ianjmacintosh.com, which points it to another name server that knows all about www.ianjmacintosh.com.

Actually, thatʼs not entirely accurate; DNS resolvers and name servers sometimes have records cached locally and donʼt even need to make a request. If a name server has a cached record for ianjmacintosh.com, itʼll start from there instead of going up to the top to locate the record for www.ianjmacintosh.com or pozo.ianjmacintosh.com or another subdomain of ianjmacintosh.com.

Also, sometimes the name server makes the appropriate next query on the requesterʼs behalf, saving the requester the trouble. That next server in turn may do the same, saving the requesting name server from needing to make an additional request. This can keep on going on and on forever, and is called a recursive search.

Normally this is useful because it helps distribute the work required to locate records, but in our case we donʼt want to distribute that work! We want to do it all ourselves. So weʼre going to use dig to tell the name servers we want the A record for www.ianjmacintosh.com, but we do not desire recursion.

⌨️ Query 8.8.8.8 for www.ianjmacintosh.comʼs address (A) record with the +norecurse query option

Hereʼs the syntax weʼre going to use with dig throughout this article:

dig [@server] [name] [type] [queryopt...]

Iʼll use a keyboard emoji (⌨️) to indicate the queries Iʼd like you to perform using dig. Iʼll describe them in plain English, your exercise will be to invoke the actual commands.

💁🏻‍♂️ Trying to keep up with how all this works mentally without running commands is difficult and silly. If you can, please go to your command line to follow along and get the most out of this article

Try to compose a command to query 8.8.8.8 for www.ianjmacintosh.comʼs address record with the +norecurse query option on your own. Click to expand this block to see the invocation I used 👀 
dig @8.8.8.8 www.ianjmacintosh.com A +norecurse

These arguments and options are:

  • 8.8.8.8 is the server to query: Googleʼs public DNS resolver. I put an @ on the front because thatʼs what digʼs documentation (man dig) prescribes. If you omit this [@server] argument, dig will consult /etc/resolv.conf and query the name servers listed there.
  • www.ianjmacintosh.com is the name of the resource record to look up.
  • A is the type of record weʼre querying for -- we are looking for an “A” record (an address record). If you omit this [type] argument, dig will perform a lookup for an A record anyway, but I always prefer to be as verbose and explicit as possible when showing examples.
  • +norecurse is the query option to toggle recursion (and iteration) off. In short: this tell servers NOT to do any extra work for us. If you omit this [queryopt], the server youʼre querying may eliminate the need to do additional exercises to find our resource. Since weʼre trying to get practice, thatʼs bad!
  • NOTE: You may have noticed I omitted the +short option. For the rest of this article, Iʼll work with verbose reports.

Putting it all together: this command instructs dig to query 8.8.8.8 for a record named www.ianjmacintosh.com with the type of A, and asks the server not to search recursively.

Read the response and notice you got an error

dig will return a report like this one:

; <<>> DiG 9.10.6 <<>> @8.8.8.8 www.ianjmacintosh.com A +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 25344
;; flags: qr ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;www.ianjmacintosh.com.		IN	A

;; Query time: 40 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Nov 24 15:42:43 -03 2020
;; MSG SIZE  rcvd: 50

Instead of trying to understand all that, jump to the header section (->> HEADER <<-), and see it reads status: SERVFAIL. Thatʼs short for “server failure” and means the name server was unable to process this query due to a problem with the name server. You didnʼt give a bad query, but it couldnʼt give you a real answer. If it did provide an answer, itʼd show a big section with a headline of ANSWER SECTION thatʼd have our requested record in it. It also explicitly says ANSWER: 0 on the line below SERVFAIL.

In short, this report shows us Googleʼs public DNS resolver doesnʼt have the address record for www.ianjmacintosh.com. If you ran this command again and added the +short option, youʼd get an empty response.

The reason we got an answer when we ran dig before was because we didnʼt turn off recursion, and Googleʼs public DNS resolver did some extra work for us. That extra work started with going to what are called “root name servers.”

The top-level domain com is part of . (pronounced “root”), just like www.ianjmacintosh.com is part of ianjmacintosh.com, and like ianjmacintosh.com is part of com. To explicitly show the root of a domain name, DNS often uses a trailing dot when showing domain names, as in: www.ianjmacintosh.com. This is called a “fully qualified domain name” (FQDN), and that trailing . carries the same meaning as the the leading / in an absolute path (such as /Users/alice/projects/next-great-american-novel): root.

Every search needs to start somewhere, and so our search for the www.ianjmacintosh.comʼs address record needs to start at the root. Weʼre looking for name server records, not an address record -- weʼre not trying to open . in our browser. For this reason, weʼll need to change our [type] argument from A (for “address”) to NS (for “name server”)

⌨️ Query 8.8.8.8 for .ʼs name server (NS) record with +norecurse

Same deal as before: try to write the command on your own first, then click to expand this block to see my invocation 🧙🏻‍♂️ 
dig @8.8.8.8 . NS +norecurse

These arguments and options are:

  • 8.8.8.8 is Googleʼs public DNS resolver
  • . is the root of all domain names
  • NS is short for “name server”
  • +norecurse is still preventing the server from doing our exercises for us

Putting it all together: this command instructs dig to query 8.8.8.8 for a record named . with the type of NS, and asks the server not to search recursively.

Locate a root server from the response report

; <<>> DiG 9.10.6 <<>> @8.8.8.8 . NS +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 51781
;; flags: qr ra ad; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;.				IN	NS

;; ANSWER SECTION:
.			86755	IN	NS	a.root-servers.net.
.			86755	IN	NS	b.root-servers.net.
.			86755	IN	NS	c.root-servers.net.
.			86755	IN	NS	d.root-servers.net.
.			86755	IN	NS	e.root-servers.net.
.			86755	IN	NS	f.root-servers.net.
.			86755	IN	NS	g.root-servers.net.
.			86755	IN	NS	h.root-servers.net.
.			86755	IN	NS	i.root-servers.net.
.			86755	IN	NS	j.root-servers.net.
.			86755	IN	NS	k.root-servers.net.
.			86755	IN	NS	l.root-servers.net.
.			86755	IN	NS	m.root-servers.net.

;; Query time: 34 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Nov 25 17:39:56 -03 2020
;; MSG SIZE  rcvd: 239

Wow! Thatʼs more like it! 🎉

First off, check out the header where it says status: NOERROR. No error! And on the next line, where it said ANSWER: 0 before, now it says ANSWER: 13. We got a proper ANSWER SECTION with 13 different root name servers, any of which should be able to answer a query for the name server record for com.

Pick one at random and use it your next query.

By the way, instead of using an IP address for the server to send your next query to, you can use a domain name. dig will quietly ask your DNS resolver for its IP address in the background before sending your query. Instead of 8.8.8.8, you can use dns.google. Instead of 198.41.0.4 you can use a.root-servers.net.

⌨️ Query a DNS root server for comʼs NS record (with +norecurse)

Try to write your own query first, then click here to see what I came up with 🐕‍🦺 
dig @a.root-servers.net com NS +norecurse

These arguments and options are:

  • a.root-servers.net is the server weʼre asking for a name server record for the top-level domain com.
  • com is the name of the record weʼre asking for
  • NS is short for “name server”
  • +norecurse keeps a.root-servers.net from doing a recursive search

Get a com name server from the report

; <<>> DiG 9.10.6 <<>> @a.root-servers.net com NS +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 16916
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 13, ADDITIONAL: 27

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;com.				IN	NS

;; AUTHORITY SECTION:
com.			172800	IN	NS	e.gtld-servers.net.
com.			172800	IN	NS	b.gtld-servers.net.
com.			172800	IN	NS	j.gtld-servers.net.
com.			172800	IN	NS	m.gtld-servers.net.
com.			172800	IN	NS	i.gtld-servers.net.
com.			172800	IN	NS	f.gtld-servers.net.
com.			172800	IN	NS	a.gtld-servers.net.
com.			172800	IN	NS	g.gtld-servers.net.
com.			172800	IN	NS	h.gtld-servers.net.
com.			172800	IN	NS	l.gtld-servers.net.
com.			172800	IN	NS	k.gtld-servers.net.
com.			172800	IN	NS	c.gtld-servers.net.
com.			172800	IN	NS	d.gtld-servers.net.

;; ADDITIONAL SECTION:
e.gtld-servers.net.	172800	IN	A	192.12.94.30
e.gtld-servers.net.	172800	IN	AAAA	2001:502:1ca1::30
b.gtld-servers.net.	172800	IN	A	192.33.14.30
b.gtld-servers.net.	172800	IN	AAAA	2001:503:231d::2:30
j.gtld-servers.net.	172800	IN	A	192.48.79.30
j.gtld-servers.net.	172800	IN	AAAA	2001:502:7094::30
m.gtld-servers.net.	172800	IN	A	192.55.83.30
m.gtld-servers.net.	172800	IN	AAAA	2001:501:b1f9::30
i.gtld-servers.net.	172800	IN	A	192.43.172.30
i.gtld-servers.net.	172800	IN	AAAA	2001:503:39c1::30
f.gtld-servers.net.	172800	IN	A	192.35.51.30
f.gtld-servers.net.	172800	IN	AAAA	2001:503:d414::30
a.gtld-servers.net.	172800	IN	A	192.5.6.30
a.gtld-servers.net.	172800	IN	AAAA	2001:503:a83e::2:30
g.gtld-servers.net.	172800	IN	A	192.42.93.30
g.gtld-servers.net.	172800	IN	AAAA	2001:503:eea3::30
h.gtld-servers.net.	172800	IN	A	192.54.112.30
h.gtld-servers.net.	172800	IN	AAAA	2001:502:8cc::30
l.gtld-servers.net.	172800	IN	A	192.41.162.30
l.gtld-servers.net.	172800	IN	AAAA	2001:500:d937::30
k.gtld-servers.net.	172800	IN	A	192.52.178.30
k.gtld-servers.net.	172800	IN	AAAA	2001:503:d2d::30
c.gtld-servers.net.	172800	IN	A	192.26.92.30
c.gtld-servers.net.	172800	IN	AAAA	2001:503:83eb::30
d.gtld-servers.net.	172800	IN	A	192.31.80.30
d.gtld-servers.net.	172800	IN	AAAA	2001:500:856e::30

;; Query time: 401 msec
;; SERVER: 198.41.0.4#53(198.41.0.4)
;; WHEN: Wed Nov 25 18:12:00 -03 2020
;; MSG SIZE  rcvd: 828

Jiminy Cricket! Thatʼs a lot of info! We received 13 authorities to choose from for com records, and an additional section containing all their IP addresses.

All of these 13 name servers have records for every registered domain under com. Really!

⌨️ Ask a com name server for ianjmacintosh.comʼs name server record (+norecurse like usual)

Try to write the command, then look at my version 🦮 
dig @192.31.80.30 ianjmacintosh.com NS  +norecurse
  • 192.31.80.30 is the server weʼre asking for a name server record for the top-level domain com.
    • I could have just as easily used d.gtld-servers.net instead and let dig find its IP address in the background, but since I already had the IP address in front of me, I used that instead.
  • ianjmacintosh.com is the name of the record weʼre asking for
  • NS is short for “name server”
  • +norecurse keeps 192.31.80.30 from doing a recursive search

Get an ianjmacintosh.com name server from the report

; <<>> DiG 9.10.6 <<>> @192.31.80.30 ianjmacintosh.com NS +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 30145
;; flags: qr; QUERY: 1, ANSWER: 0, AUTHORITY: 4, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;ianjmacintosh.com.		IN	NS

;; AUTHORITY SECTION:
ianjmacintosh.com.	172800	IN	NS	dns1.p03.nsone.net.
ianjmacintosh.com.	172800	IN	NS	dns2.p03.nsone.net.
ianjmacintosh.com.	172800	IN	NS	dns3.p03.nsone.net.
ianjmacintosh.com.	172800	IN	NS	dns4.p03.nsone.net.

;; Query time: 411 msec
;; SERVER: 192.31.80.30#53(192.31.80.30)
;; WHEN: Wed Nov 25 18:37:01 -03 2020
;; MSG SIZE  rcvd: 135

Any of these four name servers could give me additional record information for www.ianjmacintosh.com. Choose one for your next query, weʼre almost there!

⌨️ Query an ianjmacintosh.com name server for www.ianjmacintosh.comʼs A record (+norecurse)

Try to write the command on your own, then click to expand when you want to see my invocation ✨ 
dig @dns1.p03.nsone.net www.ianjmacintosh.com A  +norecurse
  • dns1.p03.nsone.net is the server weʼre asking for a name server record for the top-level domain com.
    • You can see I got lazy again and am letting dig find the IP address for dns1.p03.nsone.net
  • www.ianjmacintosh.com is the name of the record weʼre asking for
  • A is short for “address”
  • +norecurse keeps dns1.p03.nsone.net from doing a recursive search -- not that it would have to

💰 Locate the IP address for www.ianjmacintosh.com

; <<>> DiG 9.10.6 <<>> @dns1.p03.nsone.net www.ianjmacintosh.com A +norecurse
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 25003
;; flags: qr aa; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;www.ianjmacintosh.com.		IN	A

;; ANSWER SECTION:
www.ianjmacintosh.com.	20	IN	A	18.230.52.212
www.ianjmacintosh.com.	20	IN	A	54.207.147.214

;; Query time: 31 msec
;; SERVER: 198.51.44.3#53(198.51.44.3)
;; WHEN: Wed Nov 25 18:40:39 -03 2020
;; MSG SIZE  rcvd: 82

You did it! You got the IP addresses for the servers hosting www.ianjmacintosh.com. Right there in the answer section, youʼll see the same IP addresses we saw in the report from our first dig: 18.230.52.212 & 54.207.147.214

For fun, you can run dig with the +trace query option to watch dig iterate over all these steps automatically:

dig www.ianjmacintosh.com +trace

Conclusion

Wow, that was a lot. Your computer does all this automatically behind the scenes whenever you try to go to a website. Congratulations, you now know dramatically more than most people about DNS.

Thereʼs plenty more to learn, but this is an excellent starting point. I plan on writing follow-up articles to explore some more DNS mysteries, including what a CNAME is, how you can update your records, and how to handle some common misconfigurations.

If you just skimmed this article, didnʼt run any commands, and want to feel more confident with DNS anyway, thatʼs fine! I hope youʼll come back to this article and walk through the steps when you can. Until then, hereʼs some quick hits:

Additional Information

Why are domain trees written right-to-left (subdomain.domain.com), then paths written left-to-right (/users/~imacintosh/articles/dns-for-web-developers)?

Good question. I havenʼt found a satisfying answer. Common explanations on Q&A forums point to how email addresses (user@host) work the same way of going more specific to less specific. Jon Postelʼs notes from a 1982 meeting of the “Network Working Group” suggest the group sought to make decisions that would provide the fewest implementation challenges. This seems to be one such decision, but I donʼt see the reasons why.

For what itʼs worth, Tim Berners-Lee mentioned in an interview if he could go back and do things differently, he would have put domains in com.domain.subdomain order. 🤷🏻‍♂️

So basically the entire internet is running on the backs of 13 different root DNS servers?

Nope! Those are 13 different IP addresses, but the servers responding to requests for them are much more numerous; there were over 1300 when I wrote this, and theyʼre pretty well distributed around the world. The root servers homepage has a neat map that shows where they all are. If you want to understand how multiple systems can share one IP address, you can look up “anycast.”

If 18.230.52.212 is the IP address for the server hosting www.ianjmacintosh.com, why do I get a weird error message instead of this website when I visit it?

The server at 18.230.52.212 serves up lots of websites, not just mine. That server needs to know which website youʼre asking for. When you visit www.ianjmacintosh.com in your browser, it tells the server at 18.230.52.212 to serve up www.ianjmacintosh.com. When you visit 18.230.52.212, it may not know what you want.