Posted on 2008-02-10 20:00:00
Tags: Networking, DNS, dnstracer
The first, alphabetically sorted, DNS root servers has been assigned an IPv6 address. It is not the first one, relatively speaking: f.root-servers.net, h.root-servers.net, j.root-servers.net, k.root-servers.net and m.root-servers.net had one before a.root-servers.net. So what?
Since it's the first letter in the alphabet, programs will use the a.root-servers.net as their first source to get information from the DNS system. So does dnstracer, one of my tools to gather information about possible issues with the DNS system.
To start dnstracer, you can give it an initial DNS server where it should start its queries with regarding to a certain domain. To prevent you from having to enter a.root-servers.net, you can just give the string ., which will be replaced internally with a.root-servers.net. That part works fine.
Dnstracer also has an option to disable IPv6 queries during the diagnostics phase. That part also works fine.
What didn't work fine was the part which did do the initial DNS server, the a.root-servers.net and the option to disable IPv6 queries: It didn't disable the IPv6 query for the initial DNS server. The result? The initial request for a.root-servers.net always returned the IPv6 address, even if you disabled the IPv6 queries. And since 95+% of the popuplation of this planet still doesn't have access to an IPv6 network, the tool didn't work anymore.
Well, it worked if you used b.root-servers.net instead of ., but it needed to fixed properly too.
The fixed version can be found at http://www.mavetju.org/download/dnstracer-1.9.tar.gz, the FreeBSD port is updated.
Posted on 2007-06-13 17:00:00
Tags: Rant, DNS
Over the past years, I've created a nice hierarchy in DNS to keep my insanity under control. For example, for the POP server we have (pop.barnet) which points with a CNAME to pop2.barnet which point with a CNAME to the dbmail2.barnet jail which point with an A record to the IP addresses of the machine:
So if the machine fails, or the dbmail jail doesn't work anymore, or the dbmail-pop3 program is broken, all we have to do is one little changes in the hierarchy and it is all working again, without disrupting the real operation of the machine.pop 60 IN CNAME pop2 pop2 IN CNAME dbmail2 dbmail2 IN A 202.83.178.99
Since earlier this month we don't have one, but two POP servers! And of course the easiest solution would be: Let pop.barnet be a CNAME to both pop1.barnet and pop2.barnet.
And there starts the trouble:pop 60 IN CNAME pop1 pop 60 IN CNAME pop2 pop1 IN CNAME dbmail1 pop2 IN CNAME dbmail2 dbmail1 IN A 202.83.178.88 dbmail2 IN A 202.83.178.99
Well, I'm (!)@*#()!@*#'d. This is not allowed... Now I have, because it can't be done any different, reintroduced A records for the services....Jun 13 16:17:24 ns0 named[3106]: dns_master_load: .db/barnet.com.au:203: pop.barnet.com.au: multiple RRs of singleton type
pop 60 IN A 202.83.178.88 ; pop1 pop 60 IN A 202.83.178.99 ; pop2 pop1 IN CNAME dbmail1 pop2 IN CNAME dbmail2 dbmail1 IN A 202.83.178.88 dbmail2 IN A 202.83.178.99
Posted on 2006-12-11 09:04:18, modified on 2006-12-11 11:42:03
Tags: FreeBSD, DNS
I have been plagued by this error in some of my jails for a long time:
$ dig foo.bar ;; reply from unexpected source: 202.83.178.125#53, expected 127.0.0.1#53
telnet itself works fine, it's just that dig and friends give this strange error. What is also strange is that the tcpdump output doesn't reflect the settings in /etc/resolv.conf:
$ cat /etc/resolv.conf search barnet.com.au server 202.83.176.1
And the tcpdump output:
# tcpdump -ni lo0 port 53 11:28:45.204241 IP 202.83.178.125.57276 > 202.83.178.125.53: 15750 A? www.ibm.com. (29) 11:28:45.218305 IP 202.83.178.125.53 > 202.83.178.125.57276: 15750- 0/4/4 (203)
Of course this was a configuration issue. In /etc/resolv.conf, the right statement is nameserver, not server. But the resolver in the jail looked through the /etc/resolv.conf of the host which contains 127.0.0.1.
But the final question is: Where does it get 202.83.178.125 from? It is the IP address of the jail, to which 127.0.0.1 silently gets translated to. But the resolver still expects it to come from the 127.0.0.1 number, and is for that reason throwing the warning.
Posted on 2006-11-14 14:01:40, modified on 2006-11-14 14:20:18
Tags: Networking, DNS
I was looking for a program to see if an IP address was tagged in one of the spam black lists on the internet. I saw dns/rbllookup, which did the basic stuff.
But boy, it was a little bit outdated. Last update was 2003. It contained a lot of blacklists which were shut down ages ago, and it didn't have a proper configuration file, and it didn't print the TXT records.
Anyway, four hours later and a lot of internal redesign, it now supports
It's faster. The 700 RBLs in the Moensted list are done, with standard options, in 110 seconds, and with 500 requests at once it's handled in 35 seconds.
It is available as dns/rbllookup-ng.
Posted on 2006-06-16 11:01:52, modified on 2006-06-16 11:12:42
Tags: Networking, Rant, DNS
One of our users complained that the LawLink website (http://www.lawlink.nsw.gov.au) was very slow. I checked our traffic report webpage, and it looked fine. But why didn't it work for him? The problem lies in DNS:
[~] edwin@k7>dig lawlink.nsw.gov.au ns ;; ANSWER SECTION: lawlink.nsw.gov.au. 80018 IN NS ns.magna.com.au. lawlink.nsw.gov.au. 80018 IN NS kettle.magna.com.au. ;; ADDITIONAL SECTION: ns.magna.com.au. 79883 IN A 203.111.0.10 kettle.magna.com.au. 79887 IN A 203.111.0.13
Looks fine... FIrst nameserver
[~] edwin@k7>dig @ns1.lawlink.nsw.gov.au www.lawlink.nsw.gov.au a ;; ANSWER SECTION: www.lawlink.nsw.gov.au. 0 IN A 203.3.176.80
Besides a TTL of 0 which is very strange, this one works fine. Next one!
[~] edwin@k7>dig @ns2.lawlink.nsw.gov.au www.lawlink.nsw.gov.au a ;; connection timed out; no servers could be reached
Unreachable! Now it starts to make sense.
Due to the TTL of 0, which means that the answer never gets cached, and half of the advertised DNS servers unreachable, it will take some time to get an answer for the hostname www.lawlink.nsw.gov.au.
Typical case of having your domains hosted by somebody who has zero clue about how DNS works. Way to go Magna Data!
Posted on 2006-02-15 13:22:52, modified on 2006-02-15 13:55:21
Tags: Networking, DNS, DDOS
Dear Script kiddies, Blackmailers and other thugs on the internet,
Please stop abusing my computer as a reflector for your 'greater plans' on the Internet.
Edwin
13:05
I get a phone call via my VoIP phone. Halfway the call, the call, it just drops dead and I see the phone rebooting. Funny, not something I see often since I moved from wireless ADSL to just-use-an-ethernet-ADSL.
13:06My VPN connection is... getting... very... sluggish. Yes, sluggish is the word. Trafshow to the rescue!
13:06Wonder why there is so much DNS traffic going on:
From Address To Address Prot Bytes CPS 63.214.168.62..15796 192.168.1.1..53 udp 48193 11632 192.168.1.1..53 63.214.168.62..15796 udp 488276 65655
A general WTF comes up in my mind. Anyway, now that I know it's DNS traffic, let's see what it is.
13:07# tcpdump -s 1500 -ni sk0 port 53 13:07:17.035118 IP 63.214.168.62.20435 > 192.168.1.1.53: 15043+ [1au] ANY ANY? x.p.ctrc.cc. (40) 13:07:17.035258 IP 192.168.1.1.53 > 63.214.168.62.20435: 15043- 1/1/2 TXT[|domain] 13:07:17.176355 IP 63.214.168.62.15879 > 192.168.1.1.53: 13909+ [1au] ANY ANY? x.p.ctrc.cc. (40) 13:07:17.176515 IP 192.168.1.1.53 > 63.214.168.62.20435: 13909- 1/1/2 TXT[|domain] 13:07:17.225230 IP 208.222.0.82.9761 > 192.168.1.1.53: 24263+ [1au] ANY ANY? x.p.ctrc.cc. (40) 13:07:17.225398 IP 192.168.1.1.53 > 208.222.0.82.9761: 24263- 1/1/2 TXT[|domain]
Somebody is asking my nameserver for x.p.ctrc.cc. Why me? And why do I give answers (and why is 1500 bytes not enough to hold the answer?
First things first:13:09# ipfw -a l ipfw add 50 deny udp from 63.214.168.62 to me dst-port 53 ipfw add 51 deny udp from 208.222.0.82 to me dst-port 53
Why does my nameserver actually answer this request? I mean, I'm not authoritative and I have disabled recursion and I have... oh wait... This new machine still has a virgin named running.
acl nobody {
none;
};
acl everybody {
any;
};
acl we {
192.168.0.0/16;
127.0.0.0/8;
};
options {
allow-recursion {
we;
};
};I forgot, the payload. Why wasn't 1500 bytes enough to hold the answer?
$ dig x.p.ctrc.cc any ;; Truncated, retrying in TCP mode. ; <<>> DiG 9.3.1 <<>> x.p.ctrc.cc any ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55082 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ;x.p.ctrc.cc. IN ANY ;; ANSWER SECTION: x.p.ctrc.cc. 81842 IN TXT "............................... ................................................................................ ................................................................................ ................................................................" "............. ................................................................................ ................................................................................ ................................................................................ .." "........................................................................... ................................................................................ [...] ............................................................................" ". ................................................................................ ........" ;; AUTHORITY SECTION: p.ctrc.cc. 81842 IN NS 321blowjob.com. ;; ADDITIONAL SECTION: 321blowjob.com. 27224 IN A 66.98.217.195 ;; Query time: 12 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Wed Feb 15 13:30:05 2006 ;; MSG SIZE rcvd: 4015
That's 4015 bytes of data to be sent out for every request. No wonder my VoIP phone dropped out.
Moral of the storyKeep your configurations secure. Do not allow SMTP relaying, do not allow DNS recursion. There are people out there who don't play nice.
13:50Weblog finished :-)
Posted on 2005-03-24 10:41:11, modified on 2006-01-09 16:29:23
Tags: Networking, Printers, DNS
We have the Kyocera KM-4035 network printer/scanner. Beautiful machine, it can copy, print and scan. It accepts print jobs from the network, and it can send scanned pictures as PDF to your mailbox.
Well, most of the time. Sometimes it refuses to send emails. Why?
To scan, you need to press the scan button. And sometimes, it just says "SMTP server could not be found". Very annoying. And what was more annoying was that the problem was not easily reproducable, it was actually very hard to figure it out.
To make a long story short, the problem lies in the DNS request of the scanner:
12:54:30.879447 10.200.5.11.1024 > 10.200.5.1.53: 19311 A? smtp.banco.net.au. (47) 0x0000 4500 004b 0a59 0000 ff11 91ad 0ac8 050b E..K.Y.......... 0x0010 0ac8 0501 0400 0035 0037 28ad 4b6f 0000 .......5.7(.Ko.. 0x0020 0001 0000 0000 0000 0473 6d74 7005 6261 .........smtp.ba 0x0030 6e63 6f03 6e65 7402 6175 0000 0100 0100 nco.net.au...... 0x0040 0000 0000 0000 0000 0000 00 ...........
At offset 0x001c the DNS header starts: 0x4b6f (=19311) for the identification, 0x0000 for the flags, 0x0001/0x0000/0x0000/0x000 for the number of requests/answers/authority/additional resource records and the question: who knows the A record for smtp.banco.net.au.
The DNS server for that LAN, at 10.200.5.1, is a caching-only forwarding name server. It does know where to ask for others, but itself isn't authoritative for any domains. It will give answer to questions of which the answers are cached, or to questions which have the RD (Recursion Desired) flag set. The RD flag is normally set for DNS request from simple clients (PCs, network equipment etc). If the RD flag is not set, it indicates that the device (most likely a DNS server) asking the question is smart enough to know how to handle answers with referrals.
So the scanner sends a question without the RD flag.
12:54:30.879929 10.200.5.1.53 > 10.200.5.11.1024: 19311 3/2/2 CNAME smtp.barnet.com.au., CNAME mail2.barnet.com.au., A 202.83.176.13 (169)
12:51:51.747207 10.200.5.1.53 > 10.200.5.11.1024: 27028 0/13/13 (454)
How can it be resolved?
The model of the printer/scanner is: KM-4035 Network Scanner
The scanner firmware is: KM-4035 Ver2.62.8
The network firmware is: NS-30 Ver1.3.00
Kyocera has been informed.
Posted on 2005-03-12 22:53:41, modified on 2006-01-09 16:29:23
Tags: Voice over IP, Cisco, DNS
The Cisco 7970 phones have a nifty feature: IP Phone Services. With it, you can access services on the internet (for example the stock value of CSCO). I have been asked to make some nifty features, but up to now it's no luck for me! Read on...
An IP Phone Service is defined as an URL, which returns an XML file with the commands in it. All very simple stuff.
For example, http://1.2.3.4/test.xml would return an XML file. This works.
But, we're living in the 21st century and we use hostnames these days. So, I changed it to http://xml.example.org/test.xml. No fish. Not even an TCP session towards the webserver. Why?
15:43:25.727288 10.192.15.229.1177 > 10.192.0.2.53: 48+ Type1907 (Class 29802)?. (33) [tos 0x60] 0x0000 4560 003d 1186 0000 3e11 4564 0ac0 0fe5 E`.=....>.Ed.... 0x0010 0ac0 0002 0499 0035 0029 0000 0030 0100 .......5.)...0.. 0x0020 0001 0000 0000 0000 0007 7374 6a61 6d65 ..........stjame 0x0030 7303 6e65 7402 6175 0000 0100 01 s.net.au.....
This is why. Don't ask me why the phone asks for A record of stjames.net.au, but it is asking it wrong: At offset 0x0028, the value 00 is there by mistake, it shouldn't have been there in the first place.
My name server happily refuses the query, and the Cisco 7970 returns "Host not found". Let's hope that Cisco can do something about it :-/
Note: Please note that this problem has been fixed in version 6.0.3.
Posted on 2003-04-01 21:10:55, modified on 2006-01-09 16:29:20
Tags: Networking, DNS, dnstracer
This article regarding dnstracer was published in the SAGE-AU Advice Volume 9 Number 1 (March 2003) and The Journal of AUUG Inc. Volume 24 number 1 (March 2003)
The Domain Name Server system is a globally replicated and distributed database which primary translate hostnames (www.sage-au.org.au) into IP addresses (66.216.68.159), route mail (@sage-au.org.au) to mailhubs (sagemx.sage-au.org.au) and converts IP addresses (66.216.68.159) into hostnames (platypus.instaweb.com.au). Without it, we would have to use remember the IP addresses of the servers we want to connect to (telnet 131.155.132.36 4000) and it would be very hard to send emails as easy as it goes today (mcvax!moskvax!kremvax!chernenko).
Normally you don't have to worry about DNS, you just get the settings for the nameserver you have to use via PPP when dialing into an ISP or via DHCP when connecting to a LAN at a company. They make sure that their nameservers know where to get the rest of their data, which are initially the root-nameservers.
The root-nameservers are the 13 (13 logical, but physical more) most important nameservers on the internet. They know where the rest of the DNS servers can be found.
Furthermore you have master and slave servers for a domain: the data for a domain is only manually changed at the master, the slaves transfer the data via the internal DNS mechanics.
If you're requesting the IP address of www.sage-au.org.au your nameserver will ask one of the root-servers for it. It will reply that it doesn't know it, but that the answer can be found at the DNS servers for .au and supplies a list with them and their IP addresses (The list is known as Authority Data, the IP addresses are known as Additional Data). Your server will ask the question again at one of the servers responsible for .au and get a similair answer: it doesn't know it, but it hands you a list of servers for .org.au and their IP addresses. This goes on until you're at the servers which are responsible for sage-au.org.au, in which case you get the IP address of www.sage-au.org.au (Answer Data).
If you're requesting the IP address of www.sage-au.org.au your Your server now caches the data for .au, .org.au, .sage-au.org.au and www.sage-au.org.au for a short time (the Time To Live) so that following requests for that data doesn't need to explore so much, it just can do a quick lookup of in it's own cache and returns the answer.
The DNS system is not really a SPOF, it is designed as a globally replicated and distributed database which means that if you can't reach one of the servers, you can try it at a different one. As there are 13 root-servers which know where to find the rest, there are 6 servers for the .au domain (6 logical with a total of at least 8 IP addresses), there are 9 servers for the .org.au domain and two servers for the sage-au.org.au domain. The location of the servers on the internet and replication is used to overcome connectivity problems. Regarding the network, there isn't much which can go wrong. Regarding the administrative side of it, that's where things go wrong.
When you register a new domain, you are asked what the nameservers are and if necessary also the IP addresses. Furthermore, these nameservers have to be configured to answer requests for that new domain and to exchange information between them. And actually data has to be served on that domain. Five places for things to go wrong!
At the time of writing, one of the domains of a nameserver for .org.au has expired (for people interested: optus.net has expired at December 16th 2002 and after half a month it still hasn't been re-registrered). That means that the IP address of the nameserver audns01.syd.optus.net can't be found and that this server will never be queried (after all, if you don't know an IP address you can't connect to it)
Changing the IP address of a nameserver is a pain and often it will be forgotten on one or two machines (Remember that switch in the cupboard which got installed a long time ago? Yes, that one too has the IP address of the DNS server hardcoded). Or that the registrar makes it impossible to change the IP address of the nameserver via their website because of all kind of internal checks.
Lame servers are servers which are mentioned in the NS records for a domain but are not authoritative for that domain. This can happen because of a typo in the IP address or a change which has never been fully finished (new server added while it wasn't ready or old server data removed but never from the NS records).
Stealth servers are servers which are not mentioned in the NS records but are authoritative for that domains. For example servers which have been removed from the NS records but the configuration of the server never updated.
When data is changed on the master server, the slaves will have to transfer it from there. But sometimes they can't because the master has disabled it for some reason. In that case the data on the slaves will get more and more obsolete.
DNS server software has strange habbits and one of them is often that if you end a name without a dot, it will add the current domainname to it. So if you see a zonefile with www.sage-au.org.au.sage-au.org.au, you know that they forgot to end it with a dot at the end.
Remember the traceroute(8) utility? It shows the path an IP packet takes when you send it to its destination IP address. Remember ntptrace(8)? It shows the path of NTP servers which your NTP client is syncing on. Dnstracer is something similair, it shows you where a DNS server will go for its information. So if you want to know the path to www.sage-au.org.au:
[~] edwin@k7>dnstracer -s . -o www.sage-au.org.au Tracing to www.sage-au.org.au via A.ROOT-SERVERS.NET, timeout 15 seconds A.ROOT-SERVERS.NET [.] (198.41.0.4) |\___ SEC3.APNIC.NET [au] (202.12.28.140) | |\___ ns3.melbourneit.com [org.au] (203.27.227.10) | | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) Got authoritative answer | | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) Got authoritative answer | |\___ ns3.ausregistry.net [org.au] (203.18.56.43) | | |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | | \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | |\___ ns2.ausregistry.net [org.au] (203.18.56.42) | | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | |\___ ns1.ausregistry.net [org.au] (203.18.56.41) | | |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | | \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | |\___ audns01.syd.optus.net [org.au] (No IP address) | |\___ au2ld.csiro.au [org.au] (130.116.2.21) | | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | |\___ dns1.telstra.net [org.au] (203.50.5.200) | | |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | | \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | |\___ box2.aunic.net [org.au] (203.202.150.20) | | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | \___ ns4.ausregistry.net [org.au] (210.8.15.253) | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) |\___ SEC1.APNIC.NET [au] (202.12.29.59) | |\___ au2ld.csiro.au [org.au] (130.116.2.21) (cached) | |\___ dns1.telstra.net [org.au] (203.50.5.200) (cached) | |\___ box2.aunic.net [org.au] (203.202.150.20) (cached) [...] ns1.sage-au.org.au (203.27.221.52) www.sage-au.org.au -> 66.216.68.159 ns2.sage-au.org.au (130.102.171.100) www.sage-au.org.au -> 66.216.68.159
Just like expected: the server goes to a root-server, the servers for the .au domain, the servers for the .org.au domain and the servers of the .sage-au.org.au domains. The answers received are printed at the end and they agree on it.
Sometimes it will go wrong, for example when a lame server is detected:
[~] edwin@k7>dnstracer -o -s RELAY-1.FTEL.CO.UK fataldimensions.nl.eu.org Tracing to fataldimensions.nl.eu.org via RELAY-1.FTEL.CO.UK, timeout 15 seconds RELAY-1.FTEL.CO.UK (192.65.220.24) |\___ ns.cistron.nl [nl.eu.org] (62.216.31.55) Got answer |\___ ns.lf.net [nl.eu.org] (212.9.160.1) Got answer |\___ ns.eu.org [nl.eu.org] (137.194.2.218) Lame server |\___ ns2.ispi.net [nl.eu.org] (206.131.193.15) Got authoritative answer |\___ ns.patriots.net [nl.eu.org] (206.131.200.40) Got authoritative answer \___ auth1.dns.elm.net [nl.eu.org] (81.17.34.251) Got authoritative answer [...]
The difference between "Got answer" and "Got authoritative answer" is that the first one can be a cached answer, while the second one is one from a server which admits that its responsible for that domain.
See http://www.mavetju.org/unix/dnstracer.php for more information about the dnstracer utility and how to obtain it. For FreeBSD and OpenBSD, it is in the ports-collection. For Linux, there is an RPM for it. Otherwise, just grab the tarball and compile it.