Posted on 2013-08-09 18:00:00
Tags: Rant, Networking, Riverbed
In the first thirteen years of my working life I have encountered a lot of different people in the field of networking. And for some reason they were all skilled, experienced and willing to learn. They understood their stuff, in case of issues a pointer to the right direction was enough to help them out.
In my experience at the Riverbed TAC I have encountered several new kind of networking people, although I wouldn't call them all "networking" people.
Did I miss a category? Most likely because I have repressed them, very deep...
No comments | Share on Facebook | Share on Twitter
Posted on 2013-07-08 08:00:00
Tags: FreeBSD, ARM, Networking
Recently I have obtained two Raspberry Pi units, just to see what can be done with them. So far I'm impressed, this little device is pretty powerful with regarding to video. Being a network guy, this doesn't interests me very much. But as a replacement of a 150 dollar Serial-over-TCP unit it would be very handy to get a serial console on devices it is required for.
First thing, this device runs with an ARM processor, something I haven't played with before on system administration level. First I had to build a FreeBSD ARM based kernel and world and that failed dramatically on my FreeBSD 9.1 i386 based machines. So VirtualBox to the rescue, and on an 64 bit machine it compiled a 10-Current world. My first cross-build was successful!
This disk for the Raspberry Pi is just a flash card, so I had to build an boot image (that was easy), download it to the host of the VirtualBox service (that was easy) and tell somehow OSX to not mount the disk anymore but also to keep it available for me to use dd on. That was tricky, but nothing Google couldn't tell me.
Then the final moment: Boot the Raspberry Pi with FreeBSD and... It worked. How undramatic. Ping the router, ping something on the Internet. All fine.
Except that the RTT for the ICMP packets didn't make sense:
64 bytes from 192.168.123.231: icmp_seq=22 ttl=64 time=1.321 ms 64 bytes from 192.168.123.231: icmp_seq=23 ttl=64 time=10.312 ms 64 bytes from 192.168.123.231: icmp_seq=24 ttl=64 time=9.328 ms 64 bytes from 192.168.123.231: icmp_seq=25 ttl=64 time=8.335 ms 64 bytes from 192.168.123.231: icmp_seq=26 ttl=64 time=7.411 ms 64 bytes from 192.168.123.231: icmp_seq=27 ttl=64 time=6.448 ms 64 bytes from 192.168.123.231: icmp_seq=28 ttl=64 time=5.497 ms 64 bytes from 192.168.123.231: icmp_seq=29 ttl=64 time=4.508 ms 64 bytes from 192.168.123.231: icmp_seq=30 ttl=64 time=3.540 ms 64 bytes from 192.168.123.231: icmp_seq=31 ttl=64 time=2.588 ms 64 bytes from 192.168.123.231: icmp_seq=32 ttl=64 time=1.635 ms 64 bytes from 192.168.123.231: icmp_seq=33 ttl=64 time=0.738 ms 64 bytes from 192.168.123.231: icmp_seq=34 ttl=64 time=9.770 ms 64 bytes from 192.168.123.231: icmp_seq=35 ttl=64 time=8.805 ms 64 bytes from 192.168.123.231: icmp_seq=36 ttl=64 time=7.833 ms 64 bytes from 192.168.123.231: icmp_seq=37 ttl=64 time=6.843 ms 64 bytes from 192.168.123.231: icmp_seq=38 ttl=64 time=5.869 ms
That is a beautiful sawtooth, but that doesn't make sense.
I tried various things, like setting the tick interval from 100 to 1000 Hz, but no luck yet. Together with Peter Jeremy we are looking for an answer, but being a non-kernel guy there is not much what I can do here except test patches.
Show comment | Share on Facebook | Share on Twitter
Posted on 2011-03-29 18:00:00
Tags: Networking, Riverbed
Even after working for Riverbed Technology for two and a half years now, I still have to come up with a bullet-proof analogy of how WAN optimization works.
Consider a network from one side of this planet to the other side: A round trip time of 300 milliseconds for 20 million meters. Of that 300 milliseconds, you have two factors: The speed of light to get from A to B, and serialization delay which happens at every hop and is related to the bandwidth and the size of the packet. One is constant, the other one is variable.
To move data in a stream, the throughput is limited to the smallest bandwidth in the path. Although you can move data faster on other parts, it all has to go through this one.
As an example, say you have a stream of 300 Mb and a smallest bandwidth of 2 Mbps. Without any protocol overhead, this will take 20 minutes to go through there. With a fourty byte protocol overhead and an MTU size of 1500, this will take 20 minutes and 33 seconds.
Now with WAN optimization. It consists of three parts: Optimization on TCP level, which has been ignored for now. Latency optimization on application specific protocols, which has been ignored for now. And data optimization, where the data is either compressed or only referred to. If the same or similar data gets transfered via two WAN optimizers twice, the first time you would get a relative small reduction factor, depending on the compressability of the data, while the second time you would get a large reduction factor because the data patterns is alreayd known on both devices.
If that 300 Mb is split into segments of 1024 bytes, making it 300 000 segments, and each segment has a 64 byte label, you end up with only about 19 Mb worth of labels.
Transfering that 19 Mb through a 2 Mbps link will be take 80 seconds, about 15 times faster.
Now back to the topic: A good analogy for WAN optimization.
Is it faster than light? It feels like it, but the Round Trip Time of the WAN is still the same. And the speed the packets go via is still the same.
Is it a "wormhole"? Wormhole-based paths which are shorter than a non-wormhole-based paths. The path travelled travelled for optimized traffic still has the same distance.
Is it comparable with ships, where goods are stored in containers (labels) and then transported in large bulk carriers? If the speed limit of other ships was limited to the speed limit of the bulk carriers, then it would be a good start.
Is it a train analogy, where passengers are cramped into carriages and efficiently transported across the rail network? It could be, except that on the railroad network everything is put into train carriages and transported efficiently on it. Comparing it with the French TGV and the Japanese bullet trains does not work neither, because the speed of the packets is still the same while these trains are way fast.
So, the analogy needs to use the same speed limits on the transport mechanism, and needs to give the impression that the delivery gets faster without changing the distance.
The best thing I come up with is transport of goods via large trucks instead of via small delivery vans: Goods are shipped via small delivery vans to a distribution point, stored into a single large truck which then uses the same transport infrastructure as small fast vans would have used if they would have transported their payload. Instead of a long convoy of small vans, you get one truck towards the distribution point which there gets reloaded into numerous small vans. The only thing which does not make sense yet is that small delivery vans are often 2 x 4 x 1.5 meters and big trucks are 3 x rather long x rather high, which gives the impression that size still matters while this isn't the case on WAN optimized traffic...
That is the problem if you work with magic :-)
No comments | Share on Facebook | Share on Twitter
Posted on 2011-03-17 18:00:00
Tags: Networking, Apple, Cisco
The "FreeBSD laptop as a Wireless Access Point for an iPhone" project I wrote earlier about has made me some followers, mostly they have no idea where the free internet connection comes from. But, based on the amount of download measured on it, they are enjoying it. One of the methods to determine how many people are on it is to use the output of the "arp -na" command: Every MAC address you see there is a mobile device which is associated with the wireless access-point you created.
One thing which you can do with that data is to match it against manufacturers. Very boring for non-networking techies... Don't read the rest :-)
MAC addresses consist of 12 hex-digits (48 bits) which are split in two parts: A six hex-digit (24 bits) prefix and a six hex-digit sequence number.
The MAC (or OUI as the IEEE calls it) prefix database can be found on the website of the IEEE at http://standards.ieee.org/regauth/oui/oui.txt. It contains at the moment of writing 14765 prefixes. The manuf(acturers) file from the Wireshark project can be found at http://anonsvn.wireshark.org/wireshark/trunk/manuf and contains 18321 prefixes plus a handful of shared prefixes.
Why is the one from the Wireshark project larger? Not really sure, but if you look at the registration costs it (US$ 1750 for a public registrered prefix OR US$ 1750 plus US$ 2100 per year for a private registered prefix) must be part of it. So it could be that the list from the Wireshark project has determined a bunch of the private ones. And unlike IP space which you can register in advance, you can't get a new prefix until you have certificated that you have used 95% of the sequence numbers.
Some statistics based on grep and cut and wc:
Number of prefixes Company 503 Cisco 122 Shenzhen 112 Motorola 109 Nokia Danmark A/S 84 Samsung Electronics 84 Apple 78 Intel Corp 59 Advanced (??) 61 Hewlett Packard 51 Private
"Advanced" could be a mistake in here, since it matches Advanced This and Advanced That. "Private" means a company who pays the US$ 2100 per year. Shenzhen has the same issue as "Advanced", it is a large bunch of companies in the Shenzhen city in China (near Hongkong). Apple was the company I didn't expect in the Top 10, but considering their iPhone / iPad success, it shouldn't surprise much.
Every prefix has 2 ** 24 entries in it, or 16 777 216 (about 16 million if you are conservate, or 17 million if you are optimistic), making there 1.4 billion Apple MAC addresses in the world. That number is not the number of Apple devices, since you need one per network interface: Ethernet, wireless or Bluetooth.
But the other number of Cisco is much more impressive: 8 438 939 648 MAC addresses. More than the next five in the list together.
Unfortunately the list of prefixes does not contain any assignment dates, it would have been interested to see what happened when LANs based on switching instead of hubs became the norm (and thus Cisco when Cisco started to sell their switches) and when mobile devices like the iPhone became popular, it would have boosted the allocation rate by Apple for sure.
MAC prefix exhaustion?
Unlike other technologies, and IPv4 comes in mind here, the MAC address prefix pool is pretty much unlimited but also only slowly being touched: There are 2 ** 22 or 4 194 304 prefixes. The number is 22, not 24 because two bits in the first byte of the prefix are used to determine if the MAC address is globally unique one or a special one. And right now, a good 35 years after the invention of Ethernet and Tokenring there are not even 19 thousand used.
The other causes are of the more strict rules the IEEE handles: You get a single prefix and don't get more until you have informed us officially that you have used 95% of them, and of course that you actually need to produce (and sell) something which uses a MAC address.
No comments | Share on Facebook | Share on Twitter
Posted on 2011-01-12 18:00:00
Tags: FreeBSD, 3G, Networking, Free Internet, iPhone
Recently I was on a holiday where the provider of my iPhone had no signal, but where the provider of my 3G modem for the laptop did have a signal. At least my glass was half-full!
In the past I have tried to setup Bluetooth between my laptop and my iPhone, and that resulted in a night of hard work and no effort. This time I tried a different approach: Instead of using Bluetooth for communication, I transformed the FreeBSD laptop into a wireless access point.
The command to change the wireless card from a normal client to a wireless access point are:
[~] edwin@lappie>cat wlan-iphone #!/bin/sh ifconfig wlan0 destroy ifconfig wlan0 create \ wlandev ath0 \ wlanmode hostap \ bssid \ authmode open \ ssid "My iPhone WiFi" ifconfig wlan0 up ifconfig wlan0 inet 10.0.0.1 netmask 255.255.255.0 sleep 1 sysctl -a net.inet.ip.forwarding=1 service isc-dhcpd restart
Notes:
The 3G connection is setup via ppp(8) and to enable NAT on the outgoing packets, you need to enter the following command or add it to the right label in your ppp.conf:
ppp ON lappie> dial Ppp ON lappie> PPp ON lappie> PPp ON lappie> Warning: 0.0.0.0/0: Change route failed: errno: No such process PPP ON lappie> nat enable yes PPP ON lappie>
And to make sure that the connected clients get their IP address, you should run the ISC DHCP server with for example the following configuration:
option domain-name "";
option domain-name-servers 8.8.8.8;
default-lease-time 150;
max-lease-time 300;
ddns-update-style none;
authoritative;
log-facility local7;
subnet 10.0.0.0 netmask 255.255.255.0 {
range 10.0.0.10 10.0.0.99;
option routers 10.0.0.1;
}
Notes:
Everything is working now, your glass is full again! :-)
Posted on 2009-04-27 21:00:00
Tags: Huawei E169, FreeBSD, Networking, 3G
The story so far... An upgrade to FreeBSD 7.2, X not working, something goes wrong in the IPCP layer, a non-working MacOS/X upgrade and in need of a firmware upgrade.
When I put the E169 in the first Windows machine I found, it kind of died a horrible way: it kept telling me that it had found new hardware and when I was lucky enough, I managed to run it before it decided that it had found new hardware again. Flashing it to a new firmware version was a little bit optimistic for that machine!
Today I took it the E169 modem with me to work and during lunch I put it in a Windows machine there. It installed without an itch, it connected to the Virgin network and it downloaded at a wopping speed of 150 kbps. Now why did it work there and how did it achieve it?
I found a program called Advanced Serial Port Monitor which kind of puts itself between the Windows COM port drive and the Windows kernel and displays what is being said between the client and the hardware.
The client program said "AT+CGMM?" and the modem said "E169". The client program said "AT^CARDLOCK?" and the modem said "^CARDLOCK: 2,10,0" (any idea?). The client program said "AT+CIMI" and the modem said "5050299062636465". And the client program said "AT+COPS?" and the modem said "+COPS: 1,0,"YES OPTUS",2". And a lot of other AT commands were issued, but unfortunately the client program didn't want to dial out while the monitor was running...
Unfortunately, none of the AT commands helped me from getting a proper CONNECT string (CONNECT 7200000 was the one I was looking for) and getting the IPCP part of the PPP initialisation working.
None of this helped. I kept looking and looking and suddenly I saw this one being executed instead of the "YES OPTUS" one: "AT+COPS=1,2,"50502",2". It didn't make much sense, but never tried is always miss. And I ended up with a CONNECT 7200000 and a full PPP handshake session. Bingo!
So, at the end, we ended up with the following configuration on the modem (just in case it differs from yours and it doesn't work):
and the following ppp.conf:ati Manufacturer: huawei Model: E169 Revision: 11.314.17.00.261 IMEI: 351029385479124 +GCAP: +CGSM,+DS,+ES at&v &C: 2; &D: 2; &E: 0; &F: 0; &S: 0; &W: 0; E: 1; L: 0; M: 0; Q: 0; V: 1; X: 0; Z: 0; \Q: 3; \S: 0; \V: 0; S0: 0; S2: 43; S3: 13; S4: 10; S5: 8; S6: 2; S7: 50; S8: 2; S9: 6; S10: 14; S11: 95; S30: 0; S103: 1; S104: 1; +FCLASS: 0; +ICF: 3,3; +IFC: 2,2; +IPR: 115200; +DR: 0; +DS: 0,0,2048,6; +WS46: 12; +CBST: 0,0,1; +CRLP: (61,61,48,6,0),(61,61,48,6,1),(240,240,52,6,2); +CV120: 1,1,1,0,0,0; +CHSN: 0,0,0,0; +CSSN: 0,0; +CREG: 0; +CGREG: 0; +CFUN:; +CSCS: "IRA"; +CSTA: 129; +CR: 0; +CRC: 0; +CMEE: 2; +CGDCONT: (1,"IP","VirginBroadband","0.0.0.0",0,0) ; +CGDSCONT: ; +CGTFT: ; +CGEQREQ: (1,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(2,2,0,0, 0,0,2,0,"0E0","0E0",3,0,0),(3,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(4,2,0,0,0,0,2,0, "0E0","0E0",3,0,0),(5,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(6,2,0,0,0,0,2,0,"0E0","0 E0",3,0,0),(7,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(8,2,0,0,0,0,2,0,"0E0","0E0",3,0, 0),(9,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(10,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(11, 2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(12,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(13,2,0,0, 0,0,2,0,"0E0","0E0",3,0,0),(14,2,0,0,0,0,2,0,"0E0","0E0",3,0,0),(15,2,0,0,0,0,2, 0,"0E0","0E0",3,0,0),(16,2,0,0,0,0,2,0,"0E0","0E0",3,0,0) ; +CGEQMIN: ; +CGQREQ: ; +CGQMIN: ; ; +CGEREP: 0,0; +CGCLASS: "B"; +CGSMS: 1; +CSMS: 0; +CMGF: 0; +CSAS: 0; +CRES: 0; +CSCA: "+61411990010",145; +CSMP: ,,0,0; +CSDH: 0; +CSCB: 0,"",""; +FDD: 0; +FAR: 0; +FCL: 0; +FIT: 0,0; +ES: ,,; +ESA: 0,,,,0,0,255,; +CMOD: 0; +CVHU: 1; ; +CPIN: , ; +CMEC: 0,0,0; +CKPD: 1,1; +CGATT: 1; +CGACT: 0; +CPBS: "SM"; +CPMS: "SM","SM","SM"; +CNMI: 0,0,0,0,0; +CMMS: 2; +FTS: 0; +FRS: 0; +FTH: 3; +FRH: 3; +FTM: 96; +FRM: 96; +CCUG: 0,0,0; +COPS: 1,0,""; +CUSD: 0; +CAOC: 1; +CCWA: 0; +CCLK: ""; +CLVL: 4; +CMUT: 0; +CPOL: 0,2,"",0,0,0; +CPLS: 0; +CTZR: 0; +CTZU: 0; +CLIP: 0; +COLP: 0; +CDIP: 0; +CLIR: 0; ^PORTSEL: 0; ^CPIN: , ; ^ATRECORD: 0; ^FREQLOCK: 12805096,13050984; ^CVOICE: 0; ^DDSETEX: 0
virgin:
set mru 1440
set device /dev/cuaU0.0
set speed 115200
set phone *99\#
set authname VirginBroadband
set authkey VirginBroadband
deny chap
disable deflate
disable pred1
disable vjcomp
disable mppe
enable dns
accept dns
set dial "ABORT BUSY ABORT NO\\sCARRIER TIMEOUT 5 \
\"\" AT OK-AT-OK ATE1Q0 OK \
AT+COPS=1,2,"50502",2 OK \
AT+CGDCONT=1,\\\"IP\\\",\\\"VirginBroadband\\\" OK \
\\dATDT\\T TIMEOUT 40 CONNECT"
set ifaddr 10.0.0.1/0 10.0.0.2/0 0.0.0.0 0.0.0.0
add default HISADDR
This only brings up the issue with X: Starting X with the E169 modem
plugged in did make it start the X graphical screen but it failed
to start the window manager and the mouse was unresponsive. Taking
out the modem resulted in going back to the console.
Starting X without the E169 modem plugged in started everything up
but after plugging in the modem the whole system became unresponsive.
Taking out the modem and everything worked again.
I got a hint from Callum on #bugs that I should try x11-servers/xorg-server without hald support. A recompile later and X and the mouse works with the modem plugged in, but the keyboard didn't work yet. Adding these four lines to my xorg.conf overcame the problem:
Section "ServerFlags"
Option "AllowEmptyInput" "False"
Option "AutoAddDevices" "False"
EndSectionShort version: The E169 modem works with FreeBSD 7.2 out of the box, Xorg 7.4 works with four extra lines in the config, the XFCE window manager works, everybody is happy!
Posted on 2009-04-25 19:00:00
Tags: Huawei E169, FreeBSD, Networking, 3G
Earlier this century I had a Huawei E220 USB 3G modem in my possession and believe it or not, after some fiddling and hacking I got it working under FreeBSD 6.x. (Un)fortunately I quit my job and had to give it back.
Overtime it got more and more difficult to find 3G provide who offered a E220 and I had given up on ever running 3G on my FreeBSD laptop again until I saw a commit of Poul-Henning Kamp which said "Make the ubsa(4) work with Huawei Exxx (tested with E169) 3G radio devices".
It took me still quiet a couple of months before I dared to go looking for a replacement and last week I found out that Virgin shop was the only 3G provider who could tell me from their head which brand and model USB modem they provided. And it was a Huawei E169! Life would be sweet again!
Luckily the commit by Poul wasn't too big and I was able to backport it to the 7.1 release running on my laptop and oh boy... It didn't really work yet. For some reason it did the same as my previous modem did before:
It still didn't finish the IPCP layer! I checked the APN, which is VirginBroadband. And I remembered what I did last time: I had to put it in a Windows machien before it worked. I still don't understand this, but okay.$ ppp virgin ppp> dial Ppp> PPp>
Stopping the PPP program caused a kernel panic. Rebooting the machine without stopping the PPP program caused a kernel panic. Starting X while the USB modem was in caused in a hanging X. Putting the USB modem in while X was running caused a hanging keyboard. Fsck was very upset with me, every time.
So, later I found a commit of Nick Hibma which said "Say hello to the u3g driver, implementing support for 3G modems." plus a bunch of follow-up commits with fixes. And, it was backported into RELENG_7! So I upgraded my laptop to 7.2 (RC1 or something) and rebooted.... It didn't recognize the USB modem at the first attempt, but when I loaded the u3g.ko kernel module it recognized both the modem and the disk part of it! No more patches, it is all vanilla 7.2.
But for some reason it did the same as my previous attempt did before:
That kind of euhm... sucks bigtime.$ ppp virgin ppp> dial Ppp> PPp>
I don't have a Windows machine, but I do have access a MacOSX machine which should be supported out of the box! The USB modem goes into the MacOSX machine, the software gets installed and I click on the "Connect" button. It says "Connecting" and it comes back with "Connection failed". Euhm...
Looking further, it seems that the program SetNetworkConfig gets called when you press the Connect button and it crashes. At least it gives me some kind of hint where to go next: I need to get newer software, either from Huawei E169 download page or from Virgin Broadband website. The installer of this puts the new software runs on the USB modem and for some reason it only works on a Windows machines.
So tomorrow I'm going to find me a Windows machine, put the USB modem in it, see if it works with Virgin 3G, then see if it then works on my FreeBSD machine, if not then upgrade the software first, see if it still works on the Windows machine and then see if it gets to work on the FreeBSD machine.
Lots of adventures are pending!
Posted on 2008-09-10 17:00:00, modified on 2008-09-11 12:00:00
Tags: Networking, FreeBSD, PXE, tftp
After being able to use PXE to boot up virtual machines in QEMU, I found an old computer with an old (1998 firmware) fxp ethernet card (Intel EtherExpress PRO/100 Ethernet) I thought "let's boot FreeBSD -current on it!"
That was easier said than done, because for some reason the ethernet card it was requesting a very strange path:
16:48:24.742828 IP 10.204.250.12.2071 > 10.204.250.2.69: 30 RRQ "pxebootM-^?" octet blksize 1456
0x0000: 4500 003a 0006 0000 1411 9d06 0acc fa0c E..:............
0x0010: 0acc fa02 0817 0045 0026 a4f9 0001 7078 .......E.&....px
0x0020: 6562 6f6f 74ff 006f 6374 6574 0062 6c6b eboot..octet.blk
0x0030: 7369 7a65 0031 3435 3600 size.1456.
Let's see if the DHCP answer is correct (with net/dhcpdump, also available from my website):
TIME: 2008-09-11 10:48:23.169
IP: 10.204.250.2 (00:0f:ea:2c:d5:18) > 255.255.255.255 (ff:ff:ff:ff:ff:ff)
OP: 2 (BOOTPREPLY)
HTYPE: 1 (Ethernet)
HLEN: 6
HOPS: 0
XID: b45ceb89
SECS: 1024
FLAGS: 7f80
CIADDR: 0.0.0.0
YIADDR: 10.204.250.12
SIADDR: 10.204.250.2
GIADDR: 0.0.0.0
CHADDR: 00:02:b3:5c:eb:89:00:00:00:00:00:00:00:00:00:00
SNAME: .
FNAME: pxeboot.
OPTION: 53 ( 1) DHCP message type 2 (DHCPOFFER)
OPTION: 54 ( 4) Server identifier 10.204.250.2
OPTION: 51 ( 4) IP address leasetime 600 (10m)
OPTION: 1 ( 4) Subnet mask 255.255.255.0
OPTION: 3 ( 4) Routers 10.204.250.1
OPTION: 60 ( 6) Vendor class identifier Mavvie
OPTION: 67 ( 7) Bootfile name pxeboot
So let's give it the file it wants. The shell I use nor the terminals I use actually make it possible for me to enter the ASCII character 255. So it's Perl to the rescue:
[/tftpboot] root@k7>perl -e 'link("pxeboot", "pxeboot\xff"); '
[/tftpboot] root@k7>ls -al
total 854
drwxr-xr-x 2 nobody wheel 512 Sep 10 11:05 .
drwxr-xr-x 21 root wheel 512 Aug 2 07:40 ..
-rw-r--r-- 2 root wheel 260097 Aug 27 21:20 pxeboot
-rw-r--r-- 2 root wheel 260097 Aug 27 21:20 pxeboot?
Did you see the blksize 1456? If you are using the net/freebsd-tftp port, you will send packets with that size instead of 512 bytes:
16:48:24.747635 IP 10.204.250.2.65265 > 10.204.250.12.2071: UDP, length 15 16:48:24.747821 IP 10.204.250.12.2071 > 10.204.250.2.65265: UDP, length 4 16:48:24.747961 IP 10.204.250.2.65265 > 10.204.250.12.2071: UDP, length 1460 16:48:24.748919 IP 10.204.250.12.2071 > 10.204.250.2.65265: UDP, length 4
Posted on 2008-09-06 11:00:00
Tags: Networking, Rant, Cisco
When Cisco Systems started, the world of networking was simple, there were routers and there were hubs. Routers connected to other routers and hubs, hubs connected to one router and computers. Each interface on the router was its own LAN, its own IP subnet (Unless you used the interface for SNA, DECNet, IPX, AppleTalk or briding only). And the configuration on the routers made sense:
interface serial0 ip address 192.168.1.1 255.255.255.0 ! interface ethernet0 ip address 192.168.2.1 255.255.255.0
Over time, hubs got replaced by switches. Coax cables got replaced by cat5 cables. Seperate routers and switches got integrated and people started to think in VLANs instead of router interfaces. And this is where the Cisco IOS syntax went wrong: They kept talking about router interfaces instead of LANs.
For example, to create a new VLAN an Extreme Networks switch/router or a Riverstone / Cabletron switch/router (does anybody remember them?), you create the VLAN (you give it a name, not just an index number) add the IP subnet to the VLAN, add a tag to the VLAN and add (finally!) the ports, tagged or untagged, to the VLAN. So you have a VLAN, and it has the VLAN tag and IP address properties, and it has one or more ports in it. Port specific properties (speed, duplex, label) are configured in the ports section.
As you can see, this is readable and this is logical.create vlan "backbone" configure vlan backbone tag 2 configure vlan backbone add ports 4 tagged configure vlan backbone add ports 5 untagged configure vlan backbone ipaddress 10.128.7.1/28 [...] configure ports 4 display-string fibre-to-dc1 configure ports 4 auto off speed 100 duplex full configure ports 5 display-string natgw
Now let's see how it goes on the Cisco switch/router. It calls both the physical and logical ports and the VLAN definitions "interfaces", so there is no hierarchical approach of obvious difference between them:
Let's see, vlan 2 is euhm... on ethernet0/2 and on ethernet0/1 (maybe on others too, I couldn't find it so fast in the configuration), ethernet0/2 is the access network so it is untagged but it sits in vlan 2 and ethernet0/1 is full-duplex and has vlan 2 on the trunk so it must be tagged.interface ethernet0/1 description fibre-to-dc1 switchport trunk encapsulation dot1q switchport trunk allowed vlan 2 switchport mode trunk duplex full spanning-tree portfast ! interface ethernet0/2 description natgw switchport mode access switchport access vlan 2 spanning-tree portfast ! interface vlan 2 description backbone ip address 10.128.7.1 255.255.255.240
So the definition of VLANs in the IOS Syntax has become more of a hack without hierarchical approach to the issue than a proper style of hierarchical definition of the VLANs, its properties and the ports in it. Instead of the above, it could have gotten its own section:
interface ethernet0/1 description fibre-to-dc1 duplex full spanning-tree portfast ! interface ethernet0/2 description natgw spanning-tree portfast ! vlan 2 description backbone ip address 10.128.7.1 255.255.255.240 untagged ethernet0/2 tagged ethernet0/1
Can this issue be resolved and the IOS Syntax replaced by a proper syntax in which you can define a VLAN and its properties readable and logically? Asking the question is answering it: Of course. But will it ever happen? I hope it, because the current syntax is very error-prone. But I doubt it, since it is there already for years and hundreds of thousands of devices do use this syntax. Having people to change all of these configurations isn't something Cisco would want to do.
Posted on 2008-09-04 16:00:00
Tags: Networking, Peering
There are several kind of internet providers. One of them are the edge providers, who have their own IP space, AS number, provide application hosting and network services for their customers and have one or two uplinks to the internet which get charged per gigabyte of data. And a port on the local internet exchange of course! The other kind are the big ones, who have multiple links with other big ISPs and sell transit to the little ones. Oh, and also provide application hosting and network services for their own customers, and sometimes they are even on the local internet exchange because of that!
Local internet exchanges are places where multiple local ISP come together (you pay for the speed and for the port on the internet exchange, not for the traffic) and agree to route traffic between destined for the other ISPs on the internet exchange directly via the internet exchange instead of via the uplinks to the big ISPs. To push this behaviour, the address ranges advertised to the local internet exchanges are often /24s while the address ranges advertised on the internet uplinks are often /21s and bigger.
So euhm... What is the issue? Nothing yet, but it is coming :-)
If you are an edge provider, what should you do? Take an uplink which is also on the local internet exchange, advertise your /24s to the local internet exchange and your big /21 to the uplink provider. Why? Because the uplink provider will advertise your /21 to the rest of the internet, while it will internally route it via the /24s to the local internet exchange. Free inbound traffic! And if your port on the local internet exchange is 100Mbps or 1000Mbps and the link towards your uplink provider is less than 100Mbps, you will have a nice extra speed increase with it too. Of course this is only for the inbound traffic, the outbound traffic to the internet still goes via the slow uplink, but downloading goes fast. And since you are an edge provider, most of the traffic will be inbound.
The thing you have to take care about is that you have to monitor, specially in the first months, that the bill of your uplink provider does reflect the real traffic going over their uplink. Some providers run their accounting systems based on all the IP traffic going through their edge routers and will bill you for the traffic even if it doesn't go over the physical wire. Check your terms and conditions to see what you can do about this Layer 8 behaviour.
So what can the big ISPs do about this? Design their network properly. Consider the three different services which are being provided: Users, Services, Transit. Users traffic and Services traffic can go via the local internet exchange, but Transit traffic shouldn't. The network should be designed to have three routing clouds (call them Autonomous Systems if you want), and the exchange of routes between the three clouds and the outside world should be regulated carefully to make sure that the Systems and Users clouds are only providing the big ISPs IP space to the transit cloud. That way the /21 is in the Transit cloud and the /24s are in the Users and Services clouds.
Is this possible? I think so. Even if the Service networks and User networks are scattered around the world, with IP-over-IP tunnels between them it will make it look like one contiguous network. Doing proper routing traffic exchange between the three routing clouds internally and between the individual routing clouds and the external networks and the network behaves the way it was meant to work.
Posted on 2008-09-02 15:00:00
Tags: FreeBSD, Networking
It's September, which means it is Spring in Australia! The semi-cold weather is gone, the birds are shrieking (still) and the trees still have leaves. And what better can you do in Spring than having a Spring Clean?
This years Spring Clean will be about the mirrors of the FreeBSD project. Over the years a huge amount of mirrors have been gathered, and most of are pretty good. Unfortunately, one or two are broken, not up-to-date, out-of-service, have disappeared or can't be found.
Earlier this month, I did the Spring for the Australian mirrors. DNS is now clean, FTP mirrors are all fine again except for ftp4.au.freebsd.org, CVSup now points to a working server, rsync is still not available and the ISO distribution looks fine. It wasn't even Spring yet, imagine that!
Without statistics you don't know anything, so I made a tool which checks for...
So, where to find these goodies? At http://www.mavetju.org/unix/freebsd-mirrors/ is the main overview. The score is calculated on the number of (correct) elements found in each test. It is possible to browse through the history of the statistics (as far as collected that is) and to see which items are changed between two dates.
I have applied for a hat to contact the mirror maintainers and ask them to fix the issues, but haven't heard anything yet.
Posted on 2008-05-13 23:00:00
Tags: IPv6, FreeBSD, Networking
Victory! Tonight I managed to get the nat6to4 daemon working. Remember what went wrong yesterday:
IPv6 packet does not go from the nat6to4 daemon into divert. What?!?!?Yes, that would have worked in one go if I actually had pushed the data in the right IPv6 socket instead of in the wrong IPv4 socket. It happens, specially when you are copying whole functions around.
As a demo:
That is pretty uninteresting for the naked eye, but the thing is that the Allegro-Software-RomPager doesn't support IPv6, it's mapped via the nat6to4 gateway.[~] edwin@freefall>telnet 2001:5c0:8fff:ffff::c3 80 Trying 2001:5c0:8fff:ffff::c3... Connected to 2001:5c0:8fff:ffff::c3. Escape character is '^]'. HEAD / HTTP/1.0 HTTP/1.1 200 OK Content-Type: text/xml; charset="utf-8" Date: Tue, 13 May 2008 14:10:23 GMT Expires: Thu, 26 Oct 1995 00:00:00 GMT Last-Modified: Tue, 13 May 2008 14:10:23 GMT Pragma: no-cache Content-Length: 32 Server: Allegro-Software-RomPager/4.34 Connection closed by foreign host.
So, my nat6to4 daemon works for mapping the TCP or UDP payload of basic IPv6 packets (single header, nothing fancy) onto IPv4 packets: Now I can make all basic services on my jails (webservers, LDAP servers, DNS servers, POP and IMAP servers) available via IPv6.
Now I have to put netinet6/ip6_divert.c into a shape so that it gets accepted by the FreeBSD project, right now there are too many things commented out because I didn't know what they are for. Yes, I feel like an apprentice magician left alone with a hand full of scrolls and asked to find out if they are interesting.
Patches for FreeBSD 6.3 are available from
http://people.freebsd.org/~edwin/freebsd63-ip6divert-20080513.patch.
The nat6to4d is available from
http://people.freebsd.org/~edwin/nat6to4d-20080513.c.
And the ipfw rules:
01005 606 245695 divert 8664 ip from any to 192.168.253.2 01006 452 33304 allow ipv6-icmp from any to me6 via tun1 01006 517 51742 divert 8666 ip6 from any to me6 via tun1 65535 79349 20349420 allow ip from any to any
Posted on 2008-05-12 10:00:00
Tags: IPv6, FreeBSD, Networking
A small update:
sockstat still doesn't show it though...nat6to4d 2578 root 3u IPv6 0xc1df8ec4 0t0 DIVERT *:8666 nat6to4d 2578 root 4u IPv4 0xc1dfaec4 0t0 DIVERT *:8664
struct sockaddr_div {
uint8_t div_len;
sa_family_t div_family; /* AF_INET / AF_INET6 */
in_port_t div_cookie; /* was: sin_port */
char div_iface[8];
struct in6_addr div6_addr; /* IPv6 address */
struct in_addr div4_addr; /* IPv4 address */
};So what works and what doesn't?
But that is an adventure for later when I have some spare time again... work and two kids, that doesn't leave much time adventures like this (except between 22:00 and 01:00 which is very bad for everybody)
Posted on 2008-05-06 10:00:00
Tags: IPv6, FreeBSD, Networking
This whole IPv6 Divert idea seems to be a little bit optimistic now. As all new technology (I wouldn't call IPv6 new technology, but still) not everything you have available now is supported in it.
So, how far did we get?
So, this is getting trickier and trickier for an userland guy like me to experiment with on a sunday afternoon... But I'm not going to give up! (yet)
On the other hand, an interesting paper by Dr. Goto (I'm not kidding) and friends available at IPV4/V6 NETWORK EMULATOR USING DIVERT SOCKET says:
I have asked him if he wanted to share his code but haven't heard anything back from him.[...] We have ported divert socket to IPv6 by adding about 1000 lines of C program code either on FreeBSD and Linux kernel for this research. [...]
Posted on 2008-05-03 09:00:00
Tags: IPv6, Networking
After two days of the IPv6 training workshop organized by APNIC, I think I'm ready for it. Mentally that is. I will guide BarNet safely into the 21st century! (Yes, I know that that century is already eight years old, but then IPv6 is already more than eight years old)
There are a couple of interesting things about IPv6: The first one is the absence of the checksum field in the IP header. Had an IPv4 header a checksum which had to be recalculated on every router it went through (that's no fun for highspeed routers I tell you), IPv6 packets don't have to do this anymore.
The second thing is the absence of the ARP protocol: It's gone now. It's over for it. Bye! It's now part of the ICMPv6 protocol: If you need to know the ethernet address of a host, you send an ICMP packet asking for it. I'm pretty sure that RARP (RFC903) is obsolete now.
The third one is the absence of subnet broadcast address: No more all 1's, it just doesn't exist. If you want to tell something to all hosts, use a local multicast.
Related to the third one is that the network troubleshooting trick to see how many hosts are alive by pinging everybody in the subnet, which is now an /64, is obsolete, because it will take seventeen days before you have complete the full sweep.
The famous IPv6 autoconfiguration... It is great for a simple network where hosts have everything (DNS, WINS, proxy etc) statically configurated, but I don't really believe in it for a properly managed network: DHCPv6 is the way to go. Still. I will have to figure out how it works: I saw that the ISC-DHCP port in the FreeBSD ports tree was very outdated and that there wasn't a 3.1 not 4.x version in it... I will have to take things in my own hands!
With regarding to our network equipment I'm not too worried: The routing Extreme Networks boxes and the Juniper boxes do support IPv6, the Cisco PoE switches should be fine. The FreeBSD, Linux and Windows 2K3 devices do support it. And PIPE networks does support it.
With the FreeBSD boxes there is a small problem right now though... In the past we carefully designed our network and services with jails and such, and jails support only one IP address. That's the whole idea behind a jail, nothing you can do about it. And that works great, oh, except for the fact that in dual stack mode you need two IP addresses: an IPv4 and IPv6 one. I know that there are patches around for FreeBSD 7.0 to support multiple IP addresses, but that doesn't help me yet because we have just migrated everything to 6.3...
So, how can provide IPv6 access to our services easily? The simplest
way is to make some kind of IPv6-to-IPv4 gateway, which translates
an IPv6 packet into an IPv4 packet and forwards that to our servers.
Confused? Here is an example: www.mavetju.org has an DNS A record
of 202.83.176.248. The IPv6 DNS AAAA record would be
2001:0DF0:0009::CA53:B0F8 (or 2001:DF0:0009::0202:0083:0176:0248
to prevent nasty dec-to-hex conversion errors). As you can see, the
last 32 bits is the IPv4 address, the first 48 bits is our network.
The IPv6-to-IPv4 gateway would carefully craft an IPv4 packet with
the right IPv4 address and send it of to my webserver. The answer
would be translated back into an IPv6 address.
Nothing difficult, this is just plain NAT. And it would expose all our services in one go to the IPv6 world, without anything to change on the services. Okay, you would lose the information of who it really was who asked it, but that is just a small price to pay until the full IPv6 service is in place. Let's do some hacking!
Posted on 2008-05-01 08:00:00, modified on 2008-07-14 16:00:00
Tags: Networking, FreeBSD, 3G
Thanks to my job, or cursed by my job sometimes, I have access to a 3G modem of Three Mobile. It works fine in areas where there is Three Mobile coverage, outside these areas were we are roaming (and probably on the Telstra 3G network) there are interesting issues with the IPCP phase (IP Confirguration Protocol) of PPP which make the PPP setup fail.
So what works and what doesn't? According to the PPP manual, this is what we should see:
The first two Ps become capitals, but the third one doesn't when we are roaming.ppp ON awfulhak> # No link has been established Ppp ON awfulhak> # We've connected & finished LCP PPp ON awfulhak> # We've authenticated PPP ON awfulhak> # We've agreed IP numbers
What do the logs say? The short version of a successful handshake is:
The full log is at the end of this writeup.IPCP: FSM: Using "deflink" as a transport IPCP: deflink: State change Initial --> Closed IPCP: deflink: LayerStart. IPCP: deflink: SendConfigReq(1) state = Closed IPCP: deflink: State change Closed --> Req-Sent IPCP: deflink: RecvConfigNak(1) state = Req-Sent IPCP: deflink: SendConfigReq(2) state = Req-Sent IPCP: deflink: RecvConfigNak(2) state = Req-Sent IPCP: deflink: SendConfigReq(3) state = Req-Sent IPCP: deflink: RecvConfigReq(0) state = Req-Sent IPCP: deflink: SendConfigNak(0) state = Req-Sent IPCP: deflink: RecvConfigRej(3) state = Req-Sent IPCP: deflink: SendConfigReq(4) state = Req-Sent IPCP: deflink: RecvConfigReq(1) state = Req-Sent IPCP: deflink: SendConfigAck(1) state = Req-Sent IPCP: deflink: State change Req-Sent --> Ack-Sent IPCP: deflink: RecvConfigNak(4) state = Ack-Sent IPCP: deflink: SendConfigReq(5) state = Ack-Sent IPCP: deflink: RecvConfigAck(5) state = Ack-Sent IPCP: deflink: State change Ack-Sent --> Opened
The short version of an unsuccessful handshake is:
The full log is at the end of this writeup.IPCP: FSM: Using "deflink" as a transport IPCP: deflink: State change Initial --> Closed IPCP: deflink: LayerStart. IPCP: deflink: SendConfigReq(1) state = Closed IPCP: deflink: State change Closed --> Req-Sent IPCP: deflink: RecvConfigNak(1) state = Req-Sent IPCP: deflink: SendConfigReq(2) state = Req-Sent IPCP: deflink: RecvConfigNak(2) state = Req-Sent IPCP: deflink: SendConfigReq(3) state = Req-Sent IPCP: deflink: RecvConfigNak(3) state = Req-Sent IPCP: deflink: SendConfigReq(4) state = Req-Sent IPCP: deflink: RecvConfigNak(4) state = Req-Sent IPCP: deflink: SendConfigReq(5) state = Req-Sent IPCP: deflink: RecvConfigNak(5) state = Req-Sent IPCP: deflink: SendConfigReq(6) state = Req-Sent IPCP: deflink: RecvConfigNak(6) state = Req-Sent IPCP: deflink: SendConfigReq(7) state = Req-Sent IPCP: deflink: RecvConfigNak(7) state = Req-Sent IPCP: deflink: SendConfigReq(8) state = Req-Sent IPCP: deflink: RecvConfigNak(8) state = Req-Sent IPCP: deflink: SendConfigReq(9) state = Req-Sent IPCP: deflink: RecvConfigNak(9) state = Req-Sent IPCP: deflink: SendConfigReq(10) state = Req-Sent IPCP: deflink: RecvConfigNak(10) state = Req-Sent IPCP: deflink: SendConfigReq(11) state = Req-Sent IPCP: deflink: SendTerminateReq(11) state = Req-Sent IPCP: deflink: State change Req-Sent --> Closing IPCP: deflink: RecvTerminateAck(11) state = Closing IPCP: deflink: LayerFinish. IPCP: deflink: State change Closing --> Closed IPCP: deflink: RecvConfigNak(11), dropped (expected 12)
If anybody has managed to setup a successful PPP connection on the 3G network of anybody but Three Mobile can send me some logs, I would be very happy to see if I can fix things.
Update According to John Marshall, sending a string similar to this:
could resolve the issue. Next time when I'm in the roaming-zone I'll try it!AT+CGDCONT=1,"IP","telstra.internet"
It worked, when I used "3netaccess" as the indentifier.
Here are the full logs:
This is the IPCP part of a successful handshake
ppp[1098]: tun0: Phase: bundle: Network ppp[1098]: tun0: IPCP: FSM: Using "deflink" as a transport ppp[1098]: tun0: IPCP: deflink: State change Initial --> Closed ppp[1098]: tun0: IPCP: deflink: LayerStart. ppp[1098]: tun0: IPCP: deflink: SendConfigReq(1) state = Closed ppp[1098]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1098]: tun0: IPCP: COMPPROTO[6] 16 VJ slots without slot compression ppp[1098]: tun0: IPCP: PRIDNS[6] 202.124.76.66 ppp[1098]: tun0: IPCP: SECDNS[6] 255.255.255.255 ppp[1098]: tun0: IPCP: deflink: State change Closed --> Req-Sent ppp[1098]: tun0: LCP: deflink: RecvProtocolRej(2) state = Opened ppp[1098]: tun0: LCP: deflink: -- Protocol 0x80fd (Compression Control Protocol) was rejected! ppp[1098]: tun0: CCP: deflink: State change Req-Sent --> Stopped ppp[1098]: tun0: IPCP: deflink: RecvConfigNak(1) state = Req-Sent ppp[1098]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1098]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1098]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1098]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1098]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1098]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1098]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1098]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1098]: tun0: IPCP: deflink: SendConfigReq(2) state = Req-Sent ppp[1098]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1098]: tun0: IPCP: COMPPROTO[6] 16 VJ slots without slot compression ppp[1098]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1098]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1098]: tun0: IPCP: deflink: RecvConfigNak(2) state = Req-Sent ppp[1098]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1098]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1098]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1098]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1098]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1098]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1098]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1098]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1098]: tun0: IPCP: deflink: SendConfigReq(3) state = Req-Sent ppp[1098]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1098]: tun0: IPCP: COMPPROTO[6] 16 VJ slots without slot compression ppp[1098]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1098]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1098]: tun0: IPCP: deflink: RecvConfigReq(0) state = Req-Sent ppp[1098]: tun0: IPCP: [EMPTY] ppp[1098]: tun0: IPCP: deflink: SendConfigNak(0) state = Req-Sent ppp[1098]: tun0: IPCP: IPADDR[6] 10.0.0.2 ppp[1098]: tun0: IPCP: deflink: RecvConfigRej(3) state = Req-Sent ppp[1098]: tun0: IPCP: COMPPROTO[6] 16 VJ slots without slot compression ppp[1098]: tun0: IPCP: deflink: SendConfigReq(4) state = Req-Sent ppp[1098]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1098]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1098]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1098]: tun0: IPCP: deflink: RecvConfigReq(1) state = Req-Sent ppp[1098]: tun0: IPCP: [EMPTY] ppp[1098]: tun0: IPCP: deflink: SendConfigAck(1) state = Req-Sent ppp[1098]: tun0: IPCP: [EMPTY] ppp[1098]: tun0: IPCP: deflink: State change Req-Sent --> Ack-Sent ppp[1098]: tun0: IPCP: deflink: RecvConfigNak(4) state = Ack-Sent ppp[1098]: tun0: IPCP: IPADDR[6] 119.11.8.120 ppp[1098]: tun0: IPCP: IPADDR[6] changing address: 0.0.0.0 --> 119.11.8.120 ppp[1098]: tun0: IPCP: PRIDNS[6] 202.124.68.130 ppp[1098]: tun0: IPCP: SECDNS[6] 202.124.76.66 ppp[1098]: tun0: IPCP: Primary nameserver set to 202.124.68.130 ppp[1098]: tun0: IPCP: Secondary nameserver set to 202.124.76.66 ppp[1098]: tun0: IPCP: deflink: SendConfigReq(5) state = Ack-Sent ppp[1098]: tun0: IPCP: IPADDR[6] 119.11.8.120 ppp[1098]: tun0: IPCP: PRIDNS[6] 202.124.68.130 ppp[1098]: tun0: IPCP: SECDNS[6] 202.124.76.66 ppp[1098]: tun0: IPCP: deflink: RecvConfigAck(5) state = Ack-Sent ppp[1098]: tun0: IPCP: IPADDR[6] 119.11.8.120 ppp[1098]: tun0: IPCP: PRIDNS[6] 202.124.68.130 ppp[1098]: tun0: IPCP: SECDNS[6] 202.124.76.66 ppp[1098]: tun0: IPCP: deflink: State change Ack-Sent --> Opened ppp[1098]: tun0: IPCP: deflink: LayerUp.
This is the IPCP part of a unsuccessful handshake
ppp[1661]: tun0: Phase: bundle: Network ppp[1661]: tun0: IPCP: FSM: Using "deflink" as a transport ppp[1661]: tun0: IPCP: deflink: State change Initial --> Closed ppp[1661]: tun0: IPCP: deflink: LayerStart. ppp[1661]: tun0: IPCP: deflink: SendConfigReq(1) state = Closed ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: State change Closed --> Req-Sent ppp[1661]: tun0: LCP: deflink: RecvProtocolRej(2) state = Opened ppp[1661]: tun0: LCP: deflink: -- Protocol 0x80fd (Compression Control Protocol) was rejected! ppp[1661]: tun0: CCP: deflink: State change Req-Sent --> Stopped ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(1) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(2) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(2) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(3) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(3) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(4) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(4) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(5) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(5) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(6) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(6) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(7) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(7) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(8) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(8) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(9) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(9) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(10) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(10) state = Req-Sent ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: PRINBNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: MS NBNS req 130 - NAK?? ppp[1661]: tun0: IPCP: SECNBNS[6] 10.11.12.14 ppp[1661]: tun0: IPCP: MS NBNS req 132 - NAK?? ppp[1661]: tun0: IPCP: Primary nameserver set to 10.11.12.13 ppp[1661]: tun0: IPCP: Secondary nameserver set to 10.11.12.14 ppp[1661]: tun0: IPCP: deflink: SendConfigReq(11) state = Req-Sent ppp[1661]: tun0: IPCP: IPADDR[6] 0.0.0.0 ppp[1661]: tun0: IPCP: COMPPROTO[6] 16 VJ slots with slot compression ppp[1661]: tun0: IPCP: PRIDNS[6] 10.11.12.13 ppp[1661]: tun0: IPCP: SECDNS[6] 10.11.12.14 ppp[1661]: tun0: Command: /dev/ttyp1: quit ppp[1661]: tun0: IPCP: deflink: SendTerminateReq(11) state = Req-Sent ppp[1661]: tun0: IPCP: deflink: State change Req-Sent --> Closing ppp[1661]: tun0: IPCP: deflink: RecvTerminateAck(11) state = Closing ppp[1661]: tun0: IPCP: deflink: LayerFinish. ppp[1661]: tun0: IPCP: Connect time: 11 secs: 0 octets in, 0 octets out ppp[1661]: tun0: IPCP: 0 packets in, 0 packets out ppp[1661]: tun0: IPCP: total 0 bytes/sec, peak 0 bytes/sec on Thu Apr 24 13:39:49 2008 ppp[1661]: tun0: IPCP: deflink: State change Closing --> Closed ppp[1661]: tun0: IPCP: deflink: RecvConfigNak(11), dropped (expected 12) ppp[1661]: tun0: Phase: bundle: Terminate
Posted on 2008-04-27 22:00:09, modified on 2008-04-27 22:00:00
Tags: Networking, Free Internet
I had the luxury of a short break at the Magenta Shores near Tuggerah Lake and being for four days away without internet access is a real challenge. But nothing to worry about, the hotel has internet access for an outrageous price. So, how does their network work?
IP address allocation is easy: Use DHCP and you get an IP address out of the 10.0.0.0/21 range. Your default gateway is a FreeBSD box which blocks all traffic except ICMP and DNS and listens on port 3128 (with a Squid proxy), port 80 (Apache + mod_php), port 53 (DNS), port 22 (SSH), port 25 (SMTP) and port 21 (a non-anonymous FTP server).
The first /24 of the /21 has a couple of other IP addresses in it which respond to pings, but it all were switches. With passwords.
So, what fun can we have?
First, the SMTP server is forwarding all messages to the internet: You can send email but you can't receive it.
The proxy server is properly locked, I couldn't find a way around it. All traffic towards it was redirected to the webserver on the default gateway asking for your password. In the root of the webserver was a menu for hotel management and system management, but it was all password protected.
The DNS service on the default gateway is working too, and... port 53/UDP is unblocked! I bought half an hour of internet time from them, used the SSH over the HTTP/CONNECT trick and seven minutes later I had an OpenVPN link up and running through that hole. I had been thinking about using the DNS tunneling application, but I would never manage to get that up and running in the half hour I paid for.
Just running tcpdump itself shows that there is a lot of multicast traffic being broadcasted by the default gateway. Unfortunately mplayer couldn't make sense of it. The traffic was most likely for the boxes connected to the TV (and the network of course): Resetting one gave a DHCP request and an IGMP join packet. The boxes itself were without brand, name, type or anything else useful to identify them: I didn't open them because they were attached with tie-straps which I couldn't replace.
And the last thing I did was running traceroute in firewall evasion mode. After the default gateway came an 192.168.1.1 address and then an iiNet broadband connection. Funny that they didn't manage to change the 192.168. address to an 10. address.
Posted on 2008-04-13 15:00:09, modified on 2008-04-13 15:00:00
Tags: Bacula, Networking
Today I finished my "Bacula client-side user exclusion flag"patch. Let me explain what it is:
How it works: In your FileSet add the option IgnoreDir:
FileSet {
Name = "Remote Specified"
Include {
Options {
signature = MD5
}
File = "\\</etc/bacula-include"
IgnoreDir = .nobackup
}
Exclude {
File = "\\</etc/bacula-exclude"
}
}I can drop files called .nobackup in directories I don't want to backup:/usr/local/etc /etc /home
[~] edwin@>find . -name .nobackup cvs/ports/.nobackup temp/.nobackup tmp/.nobackup www/cache/.nobackup
Posted on 2008-04-12 09:00:09, modified on 2008-04-12 09:00:00
Tags: IPv6, Networking
In a whimp last month I decided to apply for a chunk of IPv6 IP space at APNIC. Why? No idea, but it was influenced by the advertisements of APNIC about IPv6 training in Sydney and a request of PIPE Networks about searching for people who want to do IPv6 on their regional internet exchanges in Australia. It took some time before I my hands on it, mostly due to incorrect configured webservers whose emails get blocked because their SMTP envelope from addresses are not verifiable. But that's being taken care of by APNIC :-)
Last thursday I got an email that I have been allocated a chunk of IPv6 IP space:
Woohoo! This /48 is mine.[~] edwin@k7>whois -A barnetwork-ap-20080410 % [whois.apnic.net node-2] % Whois data copyright terms http://www.apnic.net/db/dbcopyright.html inet6num: 2001:DF0:9::/48 netname: barnetwork-ap-20080410 descr: BarNetwork Pty Limited, Internet Service Provider, Sydney, Austral ia country: AU admin-c: EG46-AP tech-c: EG46-AP mnt-by: APNIC-HM status: ASSIGNED PORTABLE remarks: -+-+-+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+ remarks: This object can only be updated by APNIC hostmasters. remarks: To update this object, please contact APNIC remarks: hostmasters and include your organisation's account remarks: name in the subject line. remarks: -+-+-+-+-+-+-+-+-+-+-+-++-+-+-+-+-+-+-+-+-+-+-+-+-+-+ changed: [email protected] 20080410 source: APNIC
A /48 is big. If you take into consideration that an IPv6 address is 128 bits, then it's very big. But luckely that IPv6 subnets are the upper /64 of an address, so we only have (in the old IPv4 terms) an IPv4 /16. An IPv4 /16 is big too, it's 65536 subnets we can allocate now. 65535 subnets of the size of an /64 is bigger, It's something like 1.2 * 1024.
Anyway, how good are we at this IPv6 stuff? Our FreeBSD and Linux servers will have no problem with it. Windows boxes also will be fine, the ones that run Windows 2003 that is. Our Extreme Networks backbone network equipment is fine with it. Our Juniper IPSec routers do support it. Our Cisco Call Manager based telephone system does not support it. Oh well, enough to play with.
The first thing I need to do is to get APNIC to create DNS NS records for 9.0.0.0.0.f.d.0.1.0.0.2.ip6.arpa to our nameservers. I keep you posted on the progress!
Posted on 2008-02-18 16:00:00
Tags: VegaStream, Networking, SNMP
[~/snmp] root@lizard>snmpwalk -v 1 -c public test-vega RFC1213-MIB::sysDescr.0 = STRING: "Vegastream IP Telephony Gateway (VEGA-6x4)" RFC1213-MIB::sysObjectID.0 = OID: RFC1155-SMI::enterprises.4686.11 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (305772277) 35 days, 9:22:02.77 RFC1213-MIB::ifNumber.0 = INTEGER: 2 RFC1213-MIB::ifIndex.1 = INTEGER: 1 RFC1213-MIB::ifIndex.2 = INTEGER: 2 RFC1213-MIB::ifDescr.1 = STRING: "VS LAN Port 1" RFC1213-MIB::ifDescr.2 = STRING: "VS LAN Port 2" RFC1213-MIB::ifType.1 = INTEGER: ethernet-csmacd(6) RFC1213-MIB::ifType.2 = INTEGER: ethernet-csmacd(6) RFC1213-MIB::ifMtu.1 = INTEGER: 1514 RFC1213-MIB::ifMtu.2 = INTEGER: 1514 RFC1213-MIB::ifSpeed.1 = Gauge32: 104857600 RFC1213-MIB::ifSpeed.2 = Gauge32: 10485760
104857600 is 100 * 1024 * 1024, or their idea of an 100 Mbps ethernet.
Posted on 2008-02-10 20:00:00
Tags: Networking, DNS, dnstracer
The first, alphabetically sorted, DNS root servers has been assigned an IPv6 address. It is not the first one, relatively speaking: f.root-servers.net, h.root-servers.net, j.root-servers.net, k.root-servers.net and m.root-servers.net had one before a.root-servers.net. So what?
Since it's the first letter in the alphabet, programs will use the a.root-servers.net as their first source to get information from the DNS system. So does dnstracer, one of my tools to gather information about possible issues with the DNS system.
To start dnstracer, you can give it an initial DNS server where it should start its queries with regarding to a certain domain. To prevent you from having to enter a.root-servers.net, you can just give the string ., which will be replaced internally with a.root-servers.net. That part works fine.
Dnstracer also has an option to disable IPv6 queries during the diagnostics phase. That part also works fine.
What didn't work fine was the part which did do the initial DNS server, the a.root-servers.net and the option to disable IPv6 queries: It didn't disable the IPv6 query for the initial DNS server. The result? The initial request for a.root-servers.net always returned the IPv6 address, even if you disabled the IPv6 queries. And since 95+% of the popuplation of this planet still doesn't have access to an IPv6 network, the tool didn't work anymore.
Well, it worked if you used b.root-servers.net instead of ., but it needed to fixed properly too.
The fixed version can be found at http://www.mavetju.org/download/dnstracer-1.9.tar.gz, the FreeBSD port is updated.
Posted on 2008-01-31 20:00:00
Tags: Networking, tftp, FreeBSD
It all started when we got some new routers, which told me the following when trying to upload configuration or download images from it: The TFTP server doesn't support the blocksize option.
My curiousity was triggered, it took me some reading of RFCs and other documentation to find out what was possible and what could be done. Was plain TFTP very simple in its handshake, TFTP with options was kind of messy because of its backwards capability: The first packet returned could either be an acknowledgement of options, or the first data packet.
Going through the source code of src/libexec/tftpd and going through the code of src/usr.bin/tftp showed that there was a lot of duplicate code, and the addition of options would only increase the amount of duplicate code. After all, both the client and the server can act as a sender and receiver.
At the end, it ended up with a nearly complete rewrite of the tftp client and server. It has been tested against the following TFTP clients and servers:
It supports the following RFCs:
It supports the following unofficial TFTP Options as described at http://www.compuphase.com/tftp.htm:
From the tftp program point of view the following things are changed:
If you try this tftp/tftpd implementation, please let me know if it works (or doesn't work) and against which implementaion so I can get a list of confirmed working systems.
For now, the new implementation can be found at as a port in net/freebsd-tftp until it has been imported back into the FreeBSD base system.
Posted on 2007-12-14 21:00:00, modified on 2007-12-17 07:30:00
Tags: Huawei E220, FreeBSD, Networking, 3G
I'm the lucky owner of a Huawei E220 3G USB modem. For certain degrees of lucky that is: Getting it to work has been an interesting challenge.
The E220 is by default (FreeBSD 6.3-BETA) recognized as an umass(4) (USB Mass Storage Device) device. That's true, there is a memory disk in it. It is recognized as /dev/cd0 and you can mount it with mount_cd9660 /dev/cd0 /mnt, giving you 10 megabytes of space.
But that is not what you bought the E220 for. You want to use it
as a modem. You need to change a couple things in your kernel to
get it all working.
First, add a line to /sys/dev/usb/usbdevs:
See the
FreeBSD PR usb/118686 for a full patch.
And to /sys/dev/usb/usba.c:
/* HP products */
product HP2 C500 0x6002 PhotoSmart C500
/* HUAWEI products */
product HUAWEI MOBILE 0x1001 Huawei Mobile
+product HUAWEI E220 0x1003 Huawei E220 HSDPA USB Modem
/* IBM Corporation */
product IBM USBCDROMDRIVE 0x4427 USB CD-ROM Drive
{ USB_VENDOR_OPTION, USB_PRODUCT_OPTION_GT3G },
/* Option GlobeTrotter 3G QUAD */
{ USB_VENDOR_OPTION, USB_PRODUCT_OPTION_GT3GQUAD },
/* Huawei Mobile */
{ USB_VENDOR_HUAWEI, USB_PRODUCT_HUAWEI_MOBILE },
+{ USB_VENDOR_HUAWEI, USB_PRODUCT_HUAWEI_E220 },
{ 0, 0 }
};
Second, you have to disable the umass(4) device from your kernel configuration and rebuild a new kernel. Will this break things? Most likely, specially when you have USB disks and sticks. But you will load it as a module, which will undo the damage you just did to it. In your /boot/loader.conf, add the following lines:
ubsa_load="YES"
umass_load="YES"
ubsa(4) is the USB support for Belkin serial adapter. That sounds more like a modem. Next, reboot the machine and see if all modules are loaded correctly:
# kldstat
Id Refs Address Size Name
1 20 0xc0400000 72e17c kernel
2 1 0xc0b2f000 e6a4 if_iwi.ko
3 3 0xc0b3e000 2f9c firmware.ko
4 1 0xc0b41000 6cdc ugen.ko
5 1 0xc0b48000 75b4 umass.ko
6 1 0xc0b50000 7958 ng_ubt.ko
7 2 0xc0b58000 cb78 netgraph.ko
8 1 0xc0b65000 2ef0 ubsa.ko
9 2 0xc0b68000 4374 ucom.ko
10 1 0xc0b6d000 5c304 acpi.ko
11 1 0xc4868000 30000 iwi_bss.ko
Next, plugin your E220 and see what happens:
ucom0: HUAWEI Technologies HUAWEI Mobile, rev 1.10/0.00, addr 2
ucom0: Could not find interrupt in
device_attach: ucom0 attach returned 6
Yes! No! Something is wrong. But a little bit of finger skill will
resolve it:
At the USB modem side of the USB cable (not the computer side!),
carefully pull the cable from the modem and when you see this on
your screen, plug it back in:
ucom0: at uhub1 port 1 (addr 2) disconnected
ucom0: HUAWEI Technologies HUAWEI Mobile, rev 1.10/0.00, addr 2
It's loaded! If it doesn't work at the first go, try it a couple
of times more. Once you are without the dreaded "Couldn't find
interrupt in" message you are a happy person.
Check /dev/cuaaU0, it exists! If you have comms/minicom installed, you can do this:
OK
ATI
Manufacturer: huawei
Model: E220
Revision: 11.117.06.00.100
IMEI: 358191018517800
+GCAP: +CGSM,+DS,+ES
OK
And if you dial out, you will end up with a whooping 7.2Mbps connection;
OK
ATDT *90#
CONNECT 7200000
Now it is time to train ppp(1). Add this to your /etc/ppp/ppp.conf:
three:
set device /dev/cuaU0
set speed 57600
set phone *99\#
set authname
set authkey
set ifaddr 10.0.0.1/0 10.0.0.2/0 0.0.0.0 0.0.0.0
set vj slotcomp off
add default HISADDR
And dial:
$ ppp three
ppp> dial
Ppp>
PPp>
That's not good, there should be three capital P's... Going through the PPP log, you will see that the IPCP layer doesn't get initialized properly. Why? No idea. How to resolve it? Put your USB modem in a windows machine, run it once there and put it back into your FreeBSD machine. Why? No idea. It works. The USB modem didn't work with under MacOS/X neither until I did this trick. Why? No idea. It works.
Dial again:
$ ppp three
ppp> dial
Ppp>
PPp>
PPP>
$ ifconfig tun0
tun0: flags=8051 mtu 1500
inet 119.11.32.164 --> 10.0.0.2 netmask 0xffffffff
Opened by PID 22997
$ ping www.freebsd.org
PING www.freebsd.org (69.147.83.33): 56 data bytes
64 bytes from 69.147.83.33: icmp_seq=0 ttl=48 time=477.662 ms
64 bytes from 69.147.83.33: icmp_seq=1 ttl=51 time=416.606 ms
^C
--- www.freebsd.org ping statistics ---
3 packets transmitted, 2 packets received, 33% packet loss
round-trip min/avg/max/stddev = 416.606/447.134/477.662/30.528 ms
Your console and /var/log/messages will be spammed with "kernel: ucom0: usba_request: STALLED" messages for some reason, but it works.
Posted on 2007-11-28 23:00:00
Tags: Networking, System Monitoring, SNMP
Yesterday I had a chat with a friend about computer networks, hardware upgrades and system monitoring and I found out that I had created in the last couple of years a very robust and detailed network monitoring and systems monitoring system, and that it has made my life a lot easier than what I could have gotten.
For example, in Nagios we monitor nearly all aspects of our FreeBSD based servers: Not only the standard memory, CPU and diskspace, but also the answer from the DNS server on it, the presence of the crond, snmpd, inetd, sshd and syslogd. Not only do we monitor if all required processes are running, but also if their PID files are there and if the processes in these PID files do exist. And we monitor the status of the RAID cards, the status of the ethernet cards and were the default gateway points to. And the uptime of the server and the offset of the NTP synced time of the server.
With regarding to network devices (routers, switches) we monitor the uptime of the device (these things reboot faster than Nagios can detect), we monitor the status of all ports (duplex, speed, operational status), temperature and status of the power supplies. And the status of the OSPF neighbours and BGP neighbours, plus a list of expected networks in the routing table.
Network link devices (antennas, fibre convertors, laser heads) which support some form of remote management are checked the same: ethernet link status, radio link status, uptime. Anything which will display possible problems with it.
For our PABX's we monitor the status of the PRIs, the status of the IAX and SIP destinations.
Call it overdone, call it wasted too much time on monitoring... But when I replace a server or a device on the network, I would like to know without too much hassle if everything is back in order once I turn it on without having to go through too much hassle: When my monitoring program says everything is fine, I know everything went fine.
Posted on 2007-09-19 13:00:00
Tags: Cisco, Networking, SNMP
When you try connecting strange little switches and hubs to a Cisco 3560 PoE switch, your port might end up in "err-disabled" state:
That port is unusable until the end of time, or until somebody manually shut and no-shut it.Gi0/41 **IP Phones & PCs* err-disabled 8 a-half a-10 10/100/1000BaseTX cisco#show interfaces gi0/41 GigabitEthernet0/41 is down, line protocol is down (err-disabled)
Having experienced this once or two, it's a pain in the bottom and since I'm obsessed about monitoring, I tried to find out how to determine this remotely:
ifAdminStatus is up, but the "show interface" says it's down.RFC1213-MIB::ifDescr.10141 = STRING: "GigabitEthernet0/41" RFC1213-MIB::ifType.10141 = INTEGER: ethernet-csmacd(6) RFC1213-MIB::ifMtu.10141 = INTEGER: 1500 RFC1213-MIB::ifSpeed.10141 = Gauge32: 10000000 RFC1213-MIB::ifPhysAddress.10141 = Hex-STRING: 00 16 46 B6 DC A9 RFC1213-MIB::ifAdminStatus.10141 = INTEGER: up(1) RFC1213-MIB::ifOperStatus.10141 = INTEGER: down(2) RFC1213-MIB::ifLastChange.10141 = Timeticks: (3564555759) 412 days, 13:32:37.59
So nothing! There is no way to find an interface in the err-disabled state via SNMP!
Posted on 2007-09-04 14:00:00
Tags: Networking, SNMP
We have a new remote accessible KVM switch. Of course we monitor from it as much as possible, including the sysUptime (an excellent way to catch devices which reboot within a one minute period). Only...
Only, the sysUptime gives an interesting answer:
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (270416251) 31 days, 7:09:22.51 [...reboot...] DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (4291368169) 496 days, 16:28:01.69 [...reboot...] DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (957) 0:00:09.57 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (1163) 0:00:11.63 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (1334) 0:00:13.34 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (4291368783) 496 days, 16:28:07.83 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (4291368895) 496 days, 16:28:08.95 DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (4291369006) 496 days, 16:28:10.06 [...]
It looks good for a couple of seconds, but then... Disabling NTP resolves these symptons.
Posted on 2007-06-21 09:00:00, modified on 2007-06-21 14:00:00
Tags: Networking, Rant, TCP-IP stack, Windows
Recently I had to redo the design of the machine with our public websites, and after an earlier successful implementation of virtualisation with FreeBSD jails, I decided to put them all in their own private jail, with their own public IP address, too.
Since I'm a firm believer in "eat your own stuff" and my website was on the list of sites to be moved, I decided to do that one first. The IP range we have for it was 202.83.176.0/24, and since the first half of it was already in use by other services, I started to go down from 255.
To make life easier for us, we use a lot of dynamic routing in our network. Also with jails: They're defined on the loopback interfaces and the subnet masks are all /32's. The combination of these two should make it easy to move them around if necessary without having to worry about physical machines and subnets and DNS.
So, we have this new webserver (my webserver, so somehow important to me) on 202.83.176.255 and it seems to work fine. I can access it from inside the network, I can access it from outside the network, I see webbrowsers and spiders connecting to it. Life is good!
Except... I get reports from people saying that they can't get to my website, that there is some kind of DNS error: Cannot find server or DNS error is what Internet Explorer tells them. I ask them: "Can you ping the machine? "No that's not workin." "Can you telnet to it?" "No, it says Connect failed.". I don't see anything in the logs, I don't see anything on the network. No idea what goes wrong here...
Finally I get the same message from friends who have elite skillz in the ancient arts of ping, traceroute, telnet and tcpdump (Hi dvl, koitsu!). And we start trying: Yes, we can ping 202.83.176.255, so there is nothing wrong on the end-to-end network layer. No, we can't ping 202.83.176.255, but I saw their ICMP packets on the webserver. From inside the jail, I can connect to their hosts, so there is nothing wrong with TCP sessions. We advertise a /21 to the world, so it won't be a network boundary problem. One of them can connect to the webserver (He's running FreeBSD), and one of them cannot (He's running Windows), I see the packets of the first, but not the packets of the second (whose ICMP packets I saw). Then the one with FreeBSD tries it with his Windows machine and he can't suddenly anymore. I think we narrowed the problem down to one thing: Microsoft Windows (Ouch, it did it again).
We do more tests: On the Windows machine, we cannot ping 202.83.176.255 (but I see the ICMP packets. We cannot setup a TCP session to it (and I don't seen any TCP packets). We can ping 202.83.176.254, and we can setup a TCP session to it. Now put one and one together....
Historically, 202.83.176.255 is in a class C subnet, going from 202.83.176.0 to 202.83.176.255. These days, with Classless Inter-Domain Routing, that subnet can be split in many little subnets, or be part of a supernet. Somehow, Windows still thinks in classfull subnets (You can see it with the default subnetmask it suggests when you configure an IP address on a network interface). And it prohibits TCP traffic halfway in the IP stack traffic to that IP address. To test this, we tried the following on the Window machines:
But still:
Anyway, the webserver now runs on 202.83.176.248 and Windows machines are happy again.
See also the thread at DSL Reports.com.
Update: The problem is confirmed in Windows2000, Windows2003 and Windows XP. Vista handles the ICMP and TCP packets as expected.
Posted on 2007-06-13 22:00:00, modified on 2007-07-05 09:00:00
Tags: Networking
Due to a recent change in network infrastructure, we needed to move the place where the IP NATting is done. Logically speaking it's now done in the middle of the network, so that the traffic from the users (on the left) is passing through it, but the traffic from the servers (on the right) is not going through it and that sometimes gives problems.
In the right-hand side of the network we have an Extreme Networks BlackDiamond 8806 (to the servers) and an Extreme Networks X450 (to the BD8806 and the Internet), and the idea was that all traffic with RFC IP addresses as source going through it would be redirected through a NAT gateway. Great design, should work without problems!
entry redirect_to_nat {
if match all {
source-address 10.0.0.0/8 ;
} then {
redirect 10.252.13.38 ;
}
}
The manual says "You can use the statement configure access-list <aclname> < ingress | egress>". The BD8806 says "I only know ingress filtering". Yes, that is right. So instead of egress filtering on one port, we need to do ingress filtering on 47 ports. Not my idea of a good time. To do egress filtering you need a BD10808 or an 12804, not an BD8806....
No problem, then we do ingress filtering on the X450 connected to it! That one supports ingress filtering, but... It doesn't support the "redirect" command: You need an X450a or an X450e for that. Curse, swear, stamp-with-feet-on-the-ground-while-screaming.
To make a short project long... We have ordered another X450a...
Update: The new X450a came without a core licence, which means that it won't support BGP. We had two leftover core licence vouchers of earlier X450's which were never used. And today we found out that, despite that they are bought for the same price and offer the same functionality, that they can't be used on an X450a.
More updates: Because we couldn't install a real license, we installed a trial license. They give you the Core license functionality, are valid for 30 days and give you the opportunity to install your hardware without having to wait two days before the real license arrives. You can upgrade without rebooting from no license to trial license, and from no license to real license. To upgrade from trial license to real license you need to use the command "clear license-info", which tells you to reboot, but after the reboot the trial license is still there. It wasn't until we got into debug mode of the switch (which you can only do with the help of Extreme Networks TAC), entered the command "debug epm clear trial-license", and rebooted the switch.
Posted on 2007-04-10 08:22:12, modified on 2007-04-22 21:45:00
Tags: Networking, SNMP
Recently we obtained a couple of new network hardware devices, to make our network saver and our life easier. But what do they run?
The first one is a Citrix VPN/SSL device. That one is easy to identify, specially if you have SNMP enabled on it:
RFC1213-MIB::sysDescr.0 = STRING: "Linux net6gateway 2.4.24 #29 SMP Wed Nov 17 16:55:58 PST 2004 i686" RFC1213-MIB::ifDescr.2 = STRING: "eth0"
That's Linux for you!
The second one is a Tippingpoint SMS server:
RFC1213-MIB::sysDescr.0 = STRING: "Linux ips-manager 2.6.15-1.2054_FC5smp #1 SMP Tue Mar 14 16:05:46 EST 2006 i686" RFC1213-MIB::sysObjectID.0 = OID: NET-SNMP-TC::linux
Linux too.
Later on, during troubleshooting a software on it, I found out that
it is running Fedora Core 6.
But... the Tippingpoint IPS devices itself:
RFC1213-MIB::sysDescr.0 = STRING: "TippingPoint IPS" RFC1213-MIB::ifDescr.1 = STRING: "lo0" RFC1213-MIB::ifDescr.2 = STRING: "fxp0"
That's some kind of *BSD.
Later on, while talking to tech support, I found out that it is
running VxWorks.
Posted on 2007-03-17 17:15:07, modified on 2007-03-17 18:34:03
Tags: Networking, DHCP
This APC power rails is a managed powerboard, which you can access via (ssh|telnet|http|etc).
By default it takes its address via BOOTP, but it can be done via DHCP too. But then, only when it finds a proper cookie in the DHCP answer. After some trying out, this is what you have to configure in the ISC-DHCP server to get it all working:
option space APC;
option APC.cookie code 1 = string;
class "apc-rpdu" {
match if substring (option vendor-class-identifier,0,3) = "APC";
vendor-option-space APC;
option APC.cookie "1APC";
}Posted on 2007-03-13 16:13:57, modified on 2007-03-13 19:10:20
Tags: Networking, Adventures
For our network, we are looking at getting a Dark Fibre backbone between our main locations and the data centre. High speed networking, that can only mean one thing: biiiiiiiig trouble.
It started with my absolute zero knowledge about using fibre as a network medium. Yes, I knew that it has to do with sending light through a small plastic tube and that it goes fast, but that is about the amount. Right now I know all pits you can end up in: Been there, done that, got the t-shirt.
First thing, there is single mode and multi mode fibre. Single mode is, as far as I get it, good for long distances, multi mode is, as far as I get it, good for short distances. For weeks and weeks I was told we were getting multi mode, and I was panicking because there so many things to chose from with regarding to multi mode, depending on the distance you cover. At the end, we ended up with single mode and all was good and fine.
Second thing are the connectors on the fibre cable. There are SC connectors, which are big ones, and there are LC connectors, which are small ones. Yes, S for large and L for small. Who could have thought that?
And then, you plug the connectors in a router/switch itself via a GBIC or plug it in a media convertor which converts it to ethernet. Of course, the GBIC needs an LC connector most likely and a media convertor uses an SC connector most likely.
So we have (2+(n-1))*2*2 ways to arrange things. Increase that with my absence of knowledge (which is rapidly decreasing now) plus the idea of using different media convertors halfway the project, that will give you about 3,142 ways to order things.
And yes, we did do them all wrong...
I was told we had ordered multi mode and we have been promised single mode fibre. We ordered LC connectors on the fibre, and then we moved from GBIC to the media convertors, so we had to change put fibre patch leads in between them which were of course male-male instead of female-male so we had to put another bunch of convertors in (it is a good thing that the stuff is passive :-)
To gain bandwidth and redundancy, we have bought special media convertors which can do sending and receiving over one cable. So you can have double the everything! Only problem, the media convertors we got were not the ones we ordered: You need one with frequency A for the local side and one with frequency B for the remote side. And we got two with frequency A. The good news is that we already had ordered another set of media convertors, and we were able to swap them both for frequency B: that way we can shuffle them around and end up with a working set!
But at the end, we got our Dark Fibre backbone up and running! And indeed, it's faaaaaaaast! (Well, 1Gbps between long distances is always fast :-)
Posted on 2006-11-14 14:01:40, modified on 2006-11-14 14:20:18
Tags: Networking, DNS
I was looking for a program to see if an IP address was tagged in one of the spam black lists on the internet. I saw dns/rbllookup, which did the basic stuff.
But boy, it was a little bit outdated. Last update was 2003. It contained a lot of blacklists which were shut down ages ago, and it didn't have a proper configuration file, and it didn't print the TXT records.
Anyway, four hours later and a lot of internal redesign, it now supports
It's faster. The 700 RBLs in the Moensted list are done, with standard options, in 110 seconds, and with 500 requests at once it's handled in 35 seconds.
It is available as dns/rbllookup-ng.
Posted on 2006-06-16 11:01:52, modified on 2006-06-16 11:12:42
Tags: Networking, Rant, DNS
One of our users complained that the LawLink website (http://www.lawlink.nsw.gov.au) was very slow. I checked our traffic report webpage, and it looked fine. But why didn't it work for him? The problem lies in DNS:
[~] edwin@k7>dig lawlink.nsw.gov.au ns ;; ANSWER SECTION: lawlink.nsw.gov.au. 80018 IN NS ns.magna.com.au. lawlink.nsw.gov.au. 80018 IN NS kettle.magna.com.au. ;; ADDITIONAL SECTION: ns.magna.com.au. 79883 IN A 203.111.0.10 kettle.magna.com.au. 79887 IN A 203.111.0.13
Looks fine... FIrst nameserver
[~] edwin@k7>dig @ns1.lawlink.nsw.gov.au www.lawlink.nsw.gov.au a ;; ANSWER SECTION: www.lawlink.nsw.gov.au. 0 IN A 203.3.176.80
Besides a TTL of 0 which is very strange, this one works fine. Next one!
[~] edwin@k7>dig @ns2.lawlink.nsw.gov.au www.lawlink.nsw.gov.au a ;; connection timed out; no servers could be reached
Unreachable! Now it starts to make sense.
Due to the TTL of 0, which means that the answer never gets cached, and half of the advertised DNS servers unreachable, it will take some time to get an answer for the hostname www.lawlink.nsw.gov.au.
Typical case of having your domains hosted by somebody who has zero clue about how DNS works. Way to go Magna Data!
Posted on 2006-05-17 15:45:37, modified on 2006-05-17 15:54:45
Tags: Networking, Voice over IP, Cisco
The Cisco 7970 phones have a nifty feature: IP Phone Services. With it, you can access services on the internet (for example the stock value of CSCO). I have been asked to make some nifty features, but the phone has some funky HTTP client code.
This is how our services are configured in the Cisco Call Manager
Service URL: http://xml.barnet.com.au/echo.xml
And this it the HTTP request the phone sends
GET /echo.xml?demo=text HTTP/1.1. Host: 202.83.176.80:80. Connection: close. User-Agent: Allegro-Software-WebClient/4.20. Accept: x-CiscoIPPhone/*;version=3.0, text/*,image/png,*/*. Accept-Language: en. Accept-Charset: iso-8859-1. Cookie: ASPSESSIONIDQQSCDATD=IIHNKHFBLCNEAGLMFDIEEIGN.
So despite that the service has the full hostname, the Host line in the HTTP request contains an IP address. It's HTTP/1.1, so the Host line is required. RFC2616 says this about it:
14.23 Host
The Host request-header field specifies the Internet host and port
number of the resource being requested, as obtained from the original
URI given by the user or referring resource (generally an HTTP URL,
as described in section 3.2.2). The Host field value MUST represent
the naming authority of the origin server or gateway given by the
original URL. This allows the origin server or gateway to
differentiate between internally-ambiguous URLs, such as the root "/"
URL of a server for multiple host names on a single IP address.
Host = "Host" ":" host [ ":" port ] ; Section 3.2.2
A "host" without any trailing port information implies the default
port for the service requested (e.g., "80" for an HTTP URL). For
example, a request on the origin server for
<http://www.w3.org/pub/WWW/> would properly include:
GET /pub/WWW/ HTTP/1.1
Host: www.w3.org
A client MUST include a Host header field in all HTTP/1.1 request
messages . If the requested URI does not include an Internet host
name for the service being requested, then the Host header field MUST
be given with an empty value. An HTTP/1.1 proxy MUST ensure that any
request message it forwards does contain an appropriate Host header
field that identifies the service being requested by the proxy. All
Internet-based HTTP/1.1 servers MUST respond with a 400 (Bad Request)
status code to any HTTP/1.1 request message which lacks a Host header
field.
Reading this, it looks like the IP address isn't even allowed there. But it should have been xml.barnet.com.au.
Posted on 2006-04-20 16:06:44, modified on 2006-04-20 16:15:25
Tags: Networking, Postmaster, SMTP, Email
Todays idiot mail server of the week award goes to webcontrolcenter.com!
All seems to go fine, until you try to send email as postmaster:
[~] edwin@k7>telnet mail22.webcontrolcenter.com 25 Trying 216.119.106.23... Connected to mail22.webcontrolcenter.com. Escape character is '^]'. 220 mail22.webcontrolcenter.com ehlo edwin.adsl.barnet.com.au 250-MAIL22 Hello [203.222.131.252] 250-SIZE 31457280 250-AUTH LOGIN CRAM-MD5 250 OK mail from:<> 250 OK <> Sender ok rset 250 OK mail from:<[email protected]> 250 OK <[email protected]> Sender ok rset 250 OK mail from:<[email protected]> 550 Sender is not allowed. Connection closed by foreign host.
And they rudely close the TCP session for you too!
Posted on 2006-03-29 13:10:58, modified on 2006-03-29 13:13:05
Tags: Networking, SMTP, Email
I've just enabled sender verification on our mailservers.
What does that mean you might wonder, and how is that different from grey-listing.
With sender verification, postfix checks if the address in the MAIL FROM command gets accepted by the MX servers of that address. It tries to do it realtime, but if it takes too long (>6 seconds) it will temporary fail the SMTP session and waits until the sending MTA retries.
Grey-listing on the other just temporary fails the first delivery attempt and waits until the sending MTA retries.
Keep in mind that two different things are checked here:
One day I'll be brave enough to do also SPF checking (Allowed delivery of email for that domain by that MTA) and everything is as open as it will be, or as closed as it will be.
Posted on 2006-03-25 19:07:17, modified on 2006-03-25 19:46:11
Tags: Networking, VLAN
Todays router adventure consisted of getting VLANs working with the X450s between the SJH building and 400 Harris street (where I'm not welcome anymore if I don't wear closed-toe shoes).
The idea is very simple: Take an ethernet link, define one or more VLANs on top of it, give certain VLANs a better priority than the others.
Buuuuuut.... No matter what we did, it didn't work.
After three hours reading up on documentation ("it can't be that difficult, can it?"), I called Uecomm and asked them the question "Is this possible at all?". Some seconds later the guy on the other side said: "No".
That was the short answer. The long answer is that it is supported if the CPE supports 801.1qinq, a protocol to support tagged VLANs in tagged VLANs.
So much for a great plan to make the whole network a little bit more workable.
Additional note: Also PIPE networks doesn't support this by default. Scary. This is not what I had in mind when we bought an ethernet service.
Posted on 2006-02-15 13:22:52, modified on 2006-02-15 13:55:21
Tags: Networking, DNS, DDOS
Dear Script kiddies, Blackmailers and other thugs on the internet,
Please stop abusing my computer as a reflector for your 'greater plans' on the Internet.
Edwin
13:05
I get a phone call via my VoIP phone. Halfway the call, the call, it just drops dead and I see the phone rebooting. Funny, not something I see often since I moved from wireless ADSL to just-use-an-ethernet-ADSL.
13:06My VPN connection is... getting... very... sluggish. Yes, sluggish is the word. Trafshow to the rescue!
13:06Wonder why there is so much DNS traffic going on:
From Address To Address Prot Bytes CPS 63.214.168.62..15796 192.168.1.1..53 udp 48193 11632 192.168.1.1..53 63.214.168.62..15796 udp 488276 65655
A general WTF comes up in my mind. Anyway, now that I know it's DNS traffic, let's see what it is.
13:07# tcpdump -s 1500 -ni sk0 port 53 13:07:17.035118 IP 63.214.168.62.20435 > 192.168.1.1.53: 15043+ [1au] ANY ANY? x.p.ctrc.cc. (40) 13:07:17.035258 IP 192.168.1.1.53 > 63.214.168.62.20435: 15043- 1/1/2 TXT[|domain] 13:07:17.176355 IP 63.214.168.62.15879 > 192.168.1.1.53: 13909+ [1au] ANY ANY? x.p.ctrc.cc. (40) 13:07:17.176515 IP 192.168.1.1.53 > 63.214.168.62.20435: 13909- 1/1/2 TXT[|domain] 13:07:17.225230 IP 208.222.0.82.9761 > 192.168.1.1.53: 24263+ [1au] ANY ANY? x.p.ctrc.cc. (40) 13:07:17.225398 IP 192.168.1.1.53 > 208.222.0.82.9761: 24263- 1/1/2 TXT[|domain]
Somebody is asking my nameserver for x.p.ctrc.cc. Why me? And why do I give answers (and why is 1500 bytes not enough to hold the answer?
First things first:13:09# ipfw -a l ipfw add 50 deny udp from 63.214.168.62 to me dst-port 53 ipfw add 51 deny udp from 208.222.0.82 to me dst-port 53
Why does my nameserver actually answer this request? I mean, I'm not authoritative and I have disabled recursion and I have... oh wait... This new machine still has a virgin named running.
acl nobody {
none;
};
acl everybody {
any;
};
acl we {
192.168.0.0/16;
127.0.0.0/8;
};
options {
allow-recursion {
we;
};
};I forgot, the payload. Why wasn't 1500 bytes enough to hold the answer?
$ dig x.p.ctrc.cc any ;; Truncated, retrying in TCP mode. ; <<>> DiG 9.3.1 <<>> x.p.ctrc.cc any ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 55082 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1 ;; QUESTION SECTION: ;x.p.ctrc.cc. IN ANY ;; ANSWER SECTION: x.p.ctrc.cc. 81842 IN TXT "............................... ................................................................................ ................................................................................ ................................................................" "............. ................................................................................ ................................................................................ ................................................................................ .." "........................................................................... ................................................................................ [...] ............................................................................" ". ................................................................................ ........" ;; AUTHORITY SECTION: p.ctrc.cc. 81842 IN NS 321blowjob.com. ;; ADDITIONAL SECTION: 321blowjob.com. 27224 IN A 66.98.217.195 ;; Query time: 12 msec ;; SERVER: 127.0.0.1#53(127.0.0.1) ;; WHEN: Wed Feb 15 13:30:05 2006 ;; MSG SIZE rcvd: 4015
That's 4015 bytes of data to be sent out for every request. No wonder my VoIP phone dropped out.
Moral of the storyKeep your configurations secure. Do not allow SMTP relaying, do not allow DNS recursion. There are people out there who don't play nice.
13:50Weblog finished :-)
Posted on 2005-09-13 14:08:27, modified on 2006-01-09 16:29:23
Tags: Happiness, Networking, Voice over IP, DHCP
64 bytes from 10.192.14.243: icmp_seq=25 ttl=127 time=0.532 ms 64 bytes from 10.192.14.243: icmp_seq=26 ttl=127 time=0.546 ms 64 bytes from 10.192.14.243: icmp_seq=27 ttl=127 time=0.564 ms 64 bytes from 10.192.14.243: icmp_seq=28 ttl=127 time=0.564 ms 64 bytes from 10.192.14.243: icmp_seq=29 ttl=127 time=0.570 ms 64 bytes from 0.0.0.0: icmp_seq=34 ttl=249 time=0.420 ms 64 bytes from 0.0.0.0: icmp_seq=35 ttl=249 time=1.807 ms 64 bytes from 0.0.0.0: icmp_seq=36 ttl=249 time=1.283 ms 64 bytes from 0.0.0.0: icmp_seq=37 ttl=249 time=1.036 ms 64 bytes from 0.0.0.0: icmp_seq=38 ttl=249 time=2.385 ms 64 bytes from 10.192.14.243: icmp_seq=39 ttl=249 time=0.764 ms 64 bytes from 10.192.14.243: icmp_seq=40 ttl=249 time=2.500 ms 64 bytes from 10.192.14.243: icmp_seq=41 ttl=249 time=0.507 ms 64 bytes from 10.192.14.243: icmp_seq=42 ttl=249 time=2.383 ms 64 bytes from 10.192.14.243: icmp_seq=43 ttl=249 time=0.612 ms 64 bytes from 10.192.14.243: icmp_seq=44 ttl=249 time=2.373 ms
Posted on 2005-07-06 10:14:34, modified on 2006-01-09 16:29:23
Tags: Networking
With the move to my new place, and the unavailability of ADSL, I have fallen back on the Unwired service. It's a wireless network over Sydney, very easy if you're roaming or need a backup plan. Only the traffic paths are euhm... relatively strange...
From the Unwired network to the BarNet service network:
traceroute to 202.83.176.1 (202.83.176.1), 64 hops max, 40 byte packets 1 r220-101-28-1.gwy.unwired.net.au (220.101.28.1) 2 v1676-cr1.equ.syd.bbn.unwired.net.au (220.101.190.105) 3 ge-2-2.br2.equ.syd.bbn.unwired.net.au (220.101.190.206) 4 202.167.228.16 (202.167.228.16) 5 syd010.pacific.net.au (61.8.0.20) 6 FastEthernet0-0-0.syd005.pacific.net.au (61.8.2.253) 7 fe0-22.ross-glb.comindico.com.au (210.23.140.214) 8 ge4-0.wsr03-kent-syd.comindico.com.au (203.194.29.244) 9 ns1.barnet.com.au (202.83.176.1)
From the Unwired network to the BarNet VoIP network:
traceroute to 202.83.178.1 (202.83.178.1) 1 r220-101-28-1.gwy.unwired.net.au (220.101.28.1) 2 v1676-cr1.equ.syd.bbn.unwired.net.au (220.101.190.105) 3 ge-2-2.br2.equ.syd.bbn.unwired.net.au (220.101.190.206) 4 203.94.172.37 (203.94.172.37) 5 AS24228.sydney.pipenetworks.com (218.100.2.31) 6 voip-equipment-1.barnet.com.au (202.83.178.1)
From the BarNet network to the Unwired network:
traceroute to 220.101.29.114 (220.101.29.114) 1 AS9837.sydney.pipenetworks.com (218.100.2.41) 2 aggr-rtrsw01-vlan2.powertel.net.au (202.92.64.129) 3 210.87.9.230 (210.87.9.230) 4 ge-2-2.cr1.equ.syd.bbn.unwired.net.au (220.101.190.205) 5 v1676-ar1.hur.syd.bbn.unwired.net.au (220.101.190.107) 6 r220-101-29-114.cpe.unwired.net.au (220.101.29.114)
As you see, a lot of different paths...
Posted on 2005-05-25 10:28:14, modified on 2006-01-09 16:29:23
Tags: Networking, Spam, SMTP, Email
If you think this is bad: (mavetju.org isn't served by 200.121.183.223)
Received: from mavetju.org ([200.121.183.223])
by imta02sl.mx.bigpond.com with ESMTP
id <[email protected]>;
Tue, 24 May 2005 23:20:49 +0000
message-id: <[email protected]>
Wait until you see this:
Return-Path: [email protected] Received: from APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr (APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr [82.121.170.216]) by mx1.midcoast.com.au (8.13.1/8.13.1) with SMTP id j4N6sWvS003077 for <[email protected]>; Mon, 23 May 2005 16:54:47 +1000 Received: from mail3.barnet.com.au by APlessis-Bouchard-152-1-59-216.w82-121.abo.wanadoo.fr (8.9.3/8.9.3) with ESMTP id PCEIP7onXFNw for <[email protected]>; Mon, 23 May 2005 14:41:04 -0700 Received: from (root@localhost) by mail3.barnet.com.au (8.12.8/8.12.8/Submit) id 1GaCy2wErDj5Ks for <[email protected]>; Mon, 23 May 2005 14:41:04 -0700 Date: Mon, 23 May 2005 14:41:04 -0700 From: Edwin Groothuis <[email protected]> Reply-To: Edwin Groothuis <[email protected]> Message-ID: <[email protected]>
What do the headers says?
Why is this worsening? It is because the email actually looks, for the untrained eye and a lot of automatic header-parser programs, like it was coming from mail3.barnet.com.au:
In the first example, everybody who knows a little bit about SMTP headers first checks if 200.121.183.223 is somewhat related to 16wardell.com.au.
In the second example, you have two more lines to parse. I admit that the syntax of the second-last line isn't proper (it is missing the hostname/ip address between brackets in the from field), but for the rest looks pretty good.
What is still wrong with it?
Could this have been prevented if mx1.midcoast.com.au would have done SPF checks? Yes. The SPF tests would have failed on every received line with a hostname.
Posted on 2005-05-05 08:21:52, modified on 2006-01-09 16:29:23
Tags: Networking
We got complaints from people outside our network that the performance of our webservers was very bad. And it was, I didn't get more than 5 to 10 kbps, while I was used to 180 kbps.
The slowness was only when the traffic was going via our primary uplink, not when it was going via our backup uplink, so it wasn't a webserver issue.
I could do all ping-tests I wanted, not a single packet was lost. But bulk transfers just sucked big time.
Tcpdump showed there was about 10 seconds of traffic and 5 seconds of silence, in which the TCP layer was trying to get acknowledgements through.
When reporting it to our internet provider, former Comindico but these days SPTel (or SPT as they announce themselves over the phone), they closed the ticket saying "ping tests show there is no packet loss". Three seconds after my phone call to them explaining that they should read what I described as the problem, with a proper mention of the URL they should try to download, they had reopened the ticket.
After four hours I got called back with the request for more information. I asked them if they had tried to download the file at the URL I had supplied. The person said "No". It took them three times to get the right URL out of my email. And then he downloaded it at about 1.5Mbps. I was stunned.
Why did it work inside their network and not outside their network? What was wrong with me? (don't answer please).
Later that evening I checked of the webserver again, and saw the reason it was so fast: Somebody else within Comindico who had looked at the ticket had already downloaded the whole file once and their proxy server had cached it. And the logfile said: '"GET /~edwin/kiki.asf HTTP/1.0" 304'. 304 means: File not modified. And the proxy then uses the local copy. I'm glad they tested the speed between their proxy and their desktop, but it didn't help me.
(Mental remark: Support personal at ISPs should have the option of not going through a proxy if they want to troubleshoot problems. They often need raw TCP access to figure out what is happening and forcing them to go through proxies will blurr their view. Just like it did here).
So the problem was still there and it got me annoyed. I decided to call Uecomm, our provider of the link towards Comindico. They provide a Layer 2 Ethernet Service and maybe they could help me out with more information. Their NOC answered and accepted the problem. Half an hour later the link to Comindico went down again, and came up. This can't be a coincidence.
I tried again and got a clean 50kbps over it. I tried a file transfer from Freefall.freebsd.org and got a clean 180kbps. I was happy again. And the phone rang, Uecomm again. They had found the problem, one of the ethernet switches of them had the link configured as half-duplex. After they had set it to full-duplex again (and the link bounced) it was all back again. Life is beautiful again!
Posted on 2005-04-19 22:34:41, modified on 2006-01-09 16:29:23
Tags: Networking, Rant, Spam
I was doing some network traces yesterday, and found these in my logs. Destination host is a Cisco 2821.
After spam via email, spam via instant messaging and spam via voice-over-ip, the next big thing is.... spam via the MS-RPC protocol! Check the following network traces:
U 61.235.154.101:57710 -> 202.83.178.14:1027
..(.......................{Z........O...,....."'..m...-.....................................SECURITY....................ALERT.......................Microsoft Windows has encounted an Internal Error
Your windows registry is corrupted.
Microsoft recommends an immediate system scan.
visit
http://e-regfix.com
to repair.
.
#
U 61.152.158.123:32780 -> 202.83.178.14:1026
..(.......................{Z........O.....P.|../.E..n..,..................i.................SECURITY....................ALERT...........%.......%...SECURITY ALERT : Windows has detected 10 Spyware programs installed on your computer!
Spyware causes pop up messages , tracks your online activities and displays advertisements.
Your Anti-Virus and Firewall will not remove Spyware.
Visit: www.antieye.com for free removal information!
.
Bunch of sad-sad-sad persons....
Posted on 2005-03-24 10:41:11, modified on 2006-01-09 16:29:23
Tags: Networking, Printers, DNS
We have the Kyocera KM-4035 network printer/scanner. Beautiful machine, it can copy, print and scan. It accepts print jobs from the network, and it can send scanned pictures as PDF to your mailbox.
Well, most of the time. Sometimes it refuses to send emails. Why?
To scan, you need to press the scan button. And sometimes, it just says "SMTP server could not be found". Very annoying. And what was more annoying was that the problem was not easily reproducable, it was actually very hard to figure it out.
To make a long story short, the problem lies in the DNS request of the scanner:
12:54:30.879447 10.200.5.11.1024 > 10.200.5.1.53: 19311 A? smtp.banco.net.au. (47) 0x0000 4500 004b 0a59 0000 ff11 91ad 0ac8 050b E..K.Y.......... 0x0010 0ac8 0501 0400 0035 0037 28ad 4b6f 0000 .......5.7(.Ko.. 0x0020 0001 0000 0000 0000 0473 6d74 7005 6261 .........smtp.ba 0x0030 6e63 6f03 6e65 7402 6175 0000 0100 0100 nco.net.au...... 0x0040 0000 0000 0000 0000 0000 00 ...........
At offset 0x001c the DNS header starts: 0x4b6f (=19311) for the identification, 0x0000 for the flags, 0x0001/0x0000/0x0000/0x000 for the number of requests/answers/authority/additional resource records and the question: who knows the A record for smtp.banco.net.au.
The DNS server for that LAN, at 10.200.5.1, is a caching-only forwarding name server. It does know where to ask for others, but itself isn't authoritative for any domains. It will give answer to questions of which the answers are cached, or to questions which have the RD (Recursion Desired) flag set. The RD flag is normally set for DNS request from simple clients (PCs, network equipment etc). If the RD flag is not set, it indicates that the device (most likely a DNS server) asking the question is smart enough to know how to handle answers with referrals.
So the scanner sends a question without the RD flag.
12:54:30.879929 10.200.5.1.53 > 10.200.5.11.1024: 19311 3/2/2 CNAME smtp.barnet.com.au., CNAME mail2.barnet.com.au., A 202.83.176.13 (169)
12:51:51.747207 10.200.5.1.53 > 10.200.5.11.1024: 27028 0/13/13 (454)
How can it be resolved?
The model of the printer/scanner is: KM-4035 Network Scanner
The scanner firmware is: KM-4035 Ver2.62.8
The network firmware is: NS-30 Ver1.3.00
Kyocera has been informed.
Posted on 2005-01-21 10:59:36, modified on 2006-01-09 16:29:22
Tags: Networking
We have a ExtremeWare Summit/400 gigabit ethernet switch/router and a RiverStone RS/3000 fast ethernet switch/router. When using a single ethernet cable between the two, we often peaked at 80-90% utilisation and therefor added more ethernet cables and trunked them into one big virtual ethernet cable.
The configuration statements
Configuration on the RiverStone RS/3000:
smarttrunk add ports et.1.3 to st.1 smarttrunk add ports et.1.4 to st.1 smarttrunk add ports et.1.11 to st.1 smarttrunk add ports et.1.12 to st.1 smarttrunk create st.1 protocol no-protocol ! vlan add ports st.1 to rss-summit vlan create rss-summit port-based id 17 !
And on the ExtremeWare Summit/400
configure sharing address-based ip-source-dest create vlan "summit-rss" configure vlan "summit-rss" ipaddress 10.192.98.2 255.255.255.0 configure vlan "summit-rss" add port 5 untagged enable sharing 5 grouping 5,6,7,8
The commands to check if the status of the trunks
For the RS/3000:
barnet-core-sjh# port show port-status st.1
Flags: M - Mirroring enabled B - MLP Bundle S - SmartTRUNK port P - Configured as 802.1p
Negot- IFG Link Admin
Port Port Type Duplex Speed iation Value State State Flags
---- --------- ------ ----- ------ ----- ----- ----- -----
st.1 SmartTRUNK n/a n/a n/a Up
et.1.3 10/100-Mbit Ethernet Full 100 Mbits Auto Up Up
et.1.4 10/100-Mbit Ethernet Full 100 Mbits Auto Up Up
et.1.11 10/100-Mbit Ethernet Full 100 Mbits Auto Up Up
et.1.12 10/100-Mbit Ethernet Full 100 Mbits Auto Up Up
And for the Summit400
Summit400-48t:27 # show ports configuration
Port Configuration Monitor Fri Jan 21 00:14:31 2005
Port Port Link Auto Speed Duplex Flow Ld Share Media
State Status Neg Cfg Actual Cfg Actual Ctrl Master Pri Red
================================================================================
trunk-rss-1 ENABLED A ON AUTO 100 AUTO FULL NONE 5 UTP
trunk-rss-2 ENABLED A ON AUTO 100 AUTO FULL NONE 5 UTP
trunk-rss-3 ENABLED A ON AUTO 100 AUTO FULL NONE 5 UTP
trunk-rss-4 ENABLED A ON AUTO 100 AUTO FULL NONE 5 UTP
To see the utilisation
On the RS/3000
barnet-core-sjh# smarttrunk show distribution st.1 SmartTRUNK Member % Link Utilization Link Status Grp Status ---------- ----------- ----------------- ----------- ---------- st.1 et.1.3 30.40 Forwarding Up st.1 et.1.4 33.03 Forwarding Up st.1 et.1.11 29.55 Forwarding Up st.1 et.1.12 25.30 Forwarding Up
On the Summit/400
Summit400-48t:29 # show ports stats
Port Statistics Fri Jan 21 00:21:46 2005
Port Link Tx Pkt Tx Byte Rx Pkt Rx Byte Rx Rx
Status Count Count Count Count Bcast Mcast
================================================================================
trunk-rss-1 A 3863921 319130686 260162 32668772 0 0
trunk-rss-2 A 1240651 120939628 323512 41293811 0 0
trunk-rss-3 A 902577 102275906 413173 49910620 0 0
trunk-rss-4 A 1323322 156944892 549445 65218016 1 45830
Differences in physical cable selections
Both the RS/3000 and Summit/400 have different algorithms for choosing the physical cable the ethernet packet goes over:
The RS/3000 selects on stream: That is the combination of IP address of source and destination and the TCP/UDP port (if it are TCP or UDP packets). If the data is coming from the RS/3000, having multiple streams of data flowing between the computers would fully utilise the trunked ports.
The Summit/400 selects on source and destination IP address. If the data is coming from the Summit/400, having multiple streams of data flowing between the computers would utilise one cable between the Summit/400 and the RS/3000 .
Posted on 2004-11-09 09:45:04, modified on 2006-01-09 16:29:22
Tags: Networking
At BarNet, we are experimenting with different backup solutions for our fast radio links. Right now we are using the Unwired solution.... Read on!
Unwired is in theory the ideal backup ISP: Their network is via wireless access points, so where-ever you are, just connect your Unwired modem to your computer and you're online. No need for cables, telephone lines etc. Just use DHCP.
There are some issues though...
Posted on 2004-10-25 23:06:15, modified on 2006-01-09 16:29:22
Tags: Networking, SMTP, Email
Some mailers are dumb (see previous article), but some people who configure them are dumb too! After a mass mailing, I got the following errors:
If it is a temporarily error, give me a 4xx error, not a 5xx error!<[email protected]>: host mailhub.totel.ru[213.242.36.74] said: 550 sorry, user is temporarily unavailable (#5.1.1 - chkusr) (in reply to RCPT TO command).
Yes. But I'm in Australia. Wrong hemisphere mate!<[email protected]>: host chaos.fxp.org[209.251.159.150] said: 554 <mailout2.barnet.com.au[218.185.88.16]>: Client host rejected: No China Spam (in reply to RCPT TO command)
Can you please check my SPF records? Thanks!<[email protected]> host mailmaster.MysticWALL.COM[210.249.95.190] said: 550 <mailout2.barnet.com.au[218.185.88.16]>: Client host rejected: We don't accept mail from spammers (in reply to RCPT TO command)
At least they checked the content, but it would be nice if they could get me the score from the spam-scanner so I could see if I was maxed out or that it was just with my toes over the line.<[email protected]>: host mx.argosnet.com[213.30.158.180] said: 554 5.7.1 The message was categorized as SPAM. (in reply to end of DATA command)
Posted on 2004-10-25 16:39:16, modified on 2006-01-09 16:29:22
Tags: Computers, Networking, Email, Rant, SMTP
Recently we implemented so called greylisting on our mail servers. This means that all incoming SMTP sessions with the following new combinations of sending mail server IP address, sender email address and recipient email address gets temporary rejected (SMTP error code 450, meaning: try again later).
From RFC2821: 4yz: Transient Negative Completion reply
The command was not accepted, and the requested action did not occur. However, the error condition is temporary and the action may be requested again. The sender should return to the beginning of the command sequence (if any). It is difficult to assign a meaning to "transient" when two different sites (receiver- and sender-SMTP agents) must agree on the interpretation. Each reply in this category might have a different time value, but the SMTP client is encouraged to try again. A rule of thumb to determine whether a reply fits into the 4yz or the 5yz category (see below) is that replies are 4yz if they can be successful if repeated without any change in command form or in properties of the sender or receiver (that is, the command is repeated identically and the receiver does not put up a new implementation.)
This saves us about 99% of the incoming spam and viruses and is a relief for our mailboxes and the email virusscanners.
Now the bad news, there are some very brain-dead SMTP servers on the internet...
And guess what? They all run on MS Windows. Who had expected that?
Here is the list of them:
MailMax from SmartMax Software Inc.. When receiving an 450, they bounce the mail back to the sender. And this is the error message they are getting:
The 'To' address [email protected] was rejected by the remote server.Which mailserver was it talking to? What was the full error message? What was the error code? And why do you say it's a permanent error while it was a transient error? BRAIN DEAD!This is permanent error, and the message will not be retried any further.
And they claim on their website:
IMPORTANT: Codes that start with 4 and 5 are the ones that tell you that your message won't be sent until you find and fix the problem.You! Yes you! You should fix the problem, and not the other side, or the MailMax mail server.
Update: With their latest version *version 5.5), at least the error messages are better:
Sorry, your message from [email protected]> to <[email protected]> could not be delivered. The specific error is: 450 <[email protected]>: Recipient address rejected: BarNet Engineered Transit Delay -- 39 seconds This is permanent error, and the message will not be retried any further.
Still it's a 'permanent' error, but at least it's visible for the person the email was returned to that they interpreted it wrongly.
Another Update
Sorry, your message from <[email protected]> to <[email protected]> could not be delivered. The specific error is: 450 <[email protected]>: Recipient address rejected: BarNet Engineered Transit Delay -- 45 seconds 2 attempts will be made to re-send your e-mail. Each attempt will be 3 hours apart.
That's much better! Everybody upgrade to the latest version!
CapeSoft Mailer by CapeSoft. It also immediately bounces the email without retrying. BRAINDEAD!
Bigpond.com by Telstra
This is all the attempt of Telstra (Australian ISP) to handle SMTP sessions with a 450 status:
Oct 29 16:17:56 mag postfix/smtpd[10870]: NOQUEUE: reject: RCPT from gizmo06ps.bigpond.com[144.140.71.41]: 450 <[email protected]>: Recipient address rejected: BarNet Engineered Transit Delay -- 45 seconds; from=<[email protected]>, to=<[email protected]> proto=SMTP helo=<gizmo06ps.bigpond.com>
That's all: one attempt. And the sender doesn't get an "Your email has failed" message. BRAINDEAD!
InterMail from Openwave Systems Inc..
It doesn't retry at all. (experienced with ozemail.com.au)
Posted on 2004-07-02 10:54:01, modified on 2006-01-09 16:29:22
Tags: Networking, FreeBSD
At BarNet, our backup link to the internet goes via a different ISP which is charged per megabyte. Being the backup link, this normally doesn't give much problems. Except when we suddenly got a nasty bill for it. Time for some investigation... (read on)
At this moment, we have say four class C networks (192.168.0.0/24 - 192.168.3.0/24). The first two are allocated, the last two are in use for later. Our IP space is part of the IP space of the backup ISP. Our default gateway is pointing to our normal ISP, and we route the IP space of the backup ISP directly to our backup ISP. Or in a picture:
0.0.0.0/0 -> normal ISP 192.168.0.0/16 -> backup ISP 192.168.0.0/24 -> LAN A 192.168.1.0/24 -> LAN B
What happens when a packet comes in for 192.168.0.1? It gets forwarded to the LAN A. What happens when a packet for 192.168.1.1 comes in? It gets forwarded to the LAN B. And what happens when a packet for 192.168.2.1 comes in? The network 192.168.2.0/24 is not in the routing table, so it takes the next smallest one: 192.168.0.0/16. And the packet goes happily on its way to the backup ISP. But then, it gets send back to us, because the backup ISP knows that 192.168.2.0/24 is on our network. It keeps bouncing between the backup ISP and us until its TTL has expired. Each other bounce is charged to us...
The solution is to add an entry for all our networks in the routing table of one of the machines which gets null-routed: routed into a "network"-device which just discards all packets. Don't go playing with firewall rules, that will only cause problems later when the spare IP space gets used and things suddenly don't work anymore and nobody knows why.
On FreeBSD, the discard device is called disc(4). Just kldload if_disc and you can ifconfig ds0. An IP address is not necessary (trying to add one on FreeBSD 4.7 causes the machine to panic, but with 4.10 it worked). In your /boot/loader.conf, add a line with if_disc_load="YES". In your /etc/rc.conf, add a line with cloned_interfaces="ds0". And as the last one, add a line to your /etc/rc.conf with route_discard="192.168.0.0/22 -interface ds0".
From that moment, your routing table will look like:
0.0.0.0/0 -> normal ISP 192.168.0.0/16 -> backup ISP 192.168.0.0/22 -> ds0 192.168.0.0/24 -> LAN A 192.168.1.0/24 -> LAN B
and there will be no more billing for unwanted traffic from the backup ISP!
Posted on 2004-05-02 00:03:20, modified on 2006-01-09 16:29:22
Tags: Networking, SSL
We at BarNet purchased a wildcard certificate from FreeSSL to secure our web transactions, to authenticate our IMAP and POP servers and to support TLS for the SMTP sessions.
TLS (Transport Layer Security) for SMTP consists of two parts
The above example works because the certificate my MTA presents is an internal client certificate. But the official certificate which we purchased is only valid for server usage, not for client usage.
Issuer: C=US, O=FreeSSL, CN=ChainedSSL CA
Subject: C=AU, O=*.barnet.com.au,
OU=https://services.choicepoint.net/get.jsp?430367485,
OU=See www.freessl.com/cps (c)04,
OU=Domain Control Validated,
CN=*.barnet.com.au
X509v3 extensions:
Netscape Cert Type:
SSL ServerNote there that it only says "SSL Server" and not "SSL Client".
With the result that we see this in our logfiles:
May 1 00:01:03 mag postfix-dbmail/smtpd[54595]: verify error:num=26:unsupported certificate purpose May 1 00:01:03 mag postfix-dbmail/smtpd[54595]: Unverified: subject_CN=*.barnet.com.au, issuer=ChainedSSL CA
Oh well, this just means that we for outgoing sessions are using our self-signed certficates again. Moral of the story: make sure you know what you get.
Posted on 2004-02-27 18:47:32, modified on 2006-01-09 16:29:21
Tags: Networking
Before you read this, I want to make it clear that I am totally sane. Well, mostly. Regarding networking related issues you can trust my sanity. That's why this strange experience is bothering me.
My computers are connected to the internet via an ADSL modem, which is connected to the Comindico network. The machines I do most of my work on are also connected to the Comindico network. So most of the time the traffic is on the place it should be in a couple of hops:
Hostname %Loss Rcv Snt Last Best Avg Worst 1. ??? 2. rns02-kent-syd.comindico.com.au 0% 15 15 22 19 30 94 3. 203.194.20.193 0% 15 15 21 19 24 32 4. ge6-2.1000.cor01-kent-syd.comindico 0% 15 15 20 18 22 28 5. fe0-0.wsr03-kent-syd.comindico.com. 0% 15 15 18 18 40 204 6. barnetworks-link.syd.comindico.com. 0% 14 14 22 19 22 29 7. tim.barnet.com.au 0% 14 14 22 19 25 47
Sometimes when I'm browsing the internet my HTTP session is captured and answered by an MS IIS server which tells me that the page doesn't exist. This is browser independent, it has happened with Mozilla, in a text based browser and with wget. Destination independent too, it happens to sites attached to the Comindico network, other Australian ones and overseas ones.
This is strange, because as far as I could tell these sites should be running some version of the Apache webserver...
It wasn't until today that I got a name with it: The construction site company. I was trying to update a ticket in our tracking system and suddenly got this ugly orange screen.
Please study the image: It's an IIS 404 screen. It refers to rt2.barnet.com.au, the site I was trying to access. It identifies itself as www.theconstructionsite.com.au. It has a banner on it refering to infomail.
The machine with hostname www.theconstructionsite.com.au (203.39.34.202) is located on the Telstra network. The image with 'infomail' came from the site www.infomail.com.au (203.39.34.203), also located on the Telstra network and owned by the people of thecontructionsite.com.au.
1. ??? 2. rns02-kent-syd.comindico.com.au 0% 29 29 20 19 35 255 3. 203.194.20.193 0% 29 29 34 18 27 101 4. ge6-2.1000.cor01-kent-syd.comindico 0% 29 29 21 18 24 48 5. ge5-0-0.bdr01-kent-syd.comindico.co 0% 29 29 20 19 37 132 6. POS2-1-0.un1.optus.net.au 0% 29 29 22 20 27 115 7. 203.202.36.9 0% 29 29 23 20 27 99 8. gigabitethernet4-3.ken12.Sydney.tel 0% 29 29 22 21 41 241 9. FastEthernet1-0.chw13.Sydney.telstr 0% 28 29 23 21 33 107 10. constr19.lnk.telstra.net 0% 28 28 28 27 45 130 11. 203.39.34.202 0% 28 28 34 27 39 164
I checked the DNS cache, it had still more than 800 seconds before the record for rt2.barnet.com.au would expire. I checked the DNS querylog, no strange requests were made before the request for www.infomail.com.au.
I don't run a proxy server, I don't have strange firewall rules. And when I reloaded the page the one I expected shows up again.
This happens about once per week that I'm aware of (I have a lot of HTTP traffic by automatic scripts which try to recover from errors so they would just retry when they get this message).
I know it is nitpicking of me, and that I just should ignore it, but I can't stand it when a network designed to be logical does things which are illogical like this. Specially not when they can be a sign of something else going wrong.
More pictures with information: Mozilla - Media Info Mozilla - Links Info Mozilla - General Page Info
I've contacted my ADSL provider about it (that's not Comindico) and they will be looking into it, but I'm afraid it will be as much mystery for them as it is for me.
Posted on 2004-02-01 09:58:42, modified on 2006-01-09 16:29:21
Tags: Computers, Networking, Email, Viruses
Remember the old rule of the thumb regarding email and viruses? "As long as you don't start the attachment, you are safe."
The computer world has evolved since then... Read on!
Up to the release of MS-Word 6, the line between safe and unsafe attachments in your email was simple: if the file was executable (i.e. did it end with .exe, .com or .bat), then it was not safe. Was it is a text file, a document, a spreadsheet, then it was safe to open.
MS-Word 6 introduced a new feature: A scripting language in the word-processor, and it was installed with scripting enabled by default. There were a couple of issues with it:
The thin border between safe and unsafe attachments was breached: documents could suddenly have executable payload.
The first Word virus was the Concept Word Macro virus. It didn't do much besides displaying a dialogbox with a "1" on the screen when an infected document was loaded and infecting other documents with itself. It didn't do any damage further, it was just annoying.
The next feature was the capability of the Word scripting language to interact with the MS Outlook mail reader. Where a Word viruses up to that point was only able to propagate slowly via shared Word documents, it suddenly had access to a faster path: It could send itself via Outlook to everybody in the address book on the infected computer, and with a little bit of luck the recipients would open the Word document and the infection would spread.
Beside the technical capabilities to get this virus spreading, the virus needed to convince the receiver that it was safe to open the attachment. Welcome to the social engineering department. The first step is to make sure the receiver trusts the source. Since the virus gets the receivers' address from the address book of the user whose Word document is infected, that means there is some kind of trust relationship between the sender and receiver, and thus most likely also between the receiver and the sender. The second step is to use a catchy text in the body while referring to the attachment, for example by referring to the document as a shared secret between the sender and receiver.
The first virus which successfully combined these two issues was the Melissa virus. When a user opens an infected document, the virus is activated and it infects the NORMAL.DOT file so all newly created documents will contain the virus; sends itself in an email with the following text to the first 50 people in the adddress book of the infected user:
Subject: Important Message From username Here is that document you asked for ... don't show anyone else ;-)'
It is hard to resist emails with this text, even if you are made suspicious by the fact that you got five of them already, all from different sources...
When displaying emails, most of the modern browsers only show the text and/or the HTML parts of the email. The last line of defense with regard to email based viruses was the fact that users needed to open the attachment before the virus became active.
If you could execute code in the HTML part of a document, you would be able to infect the computer without having to open an attachment.
Fortunately this is not so easy anymore. JavaScript, one of the programming languages which can be embedded in HTML, isn't capable of accessing files on the local disk. Java, a programming language which is often embedded in web based applications, doesn't allow remote applications to access local disks. ActiveX, Microsofts competitor of Java, shouldn't allow remote applications to access local disks, but due to some implementation issues this was sometimes still possible.
So if you were able to make an HTML message with ActiveX components which bypassed the ActiveX security, you would be able to create a file on disk from just viewing the email message.
The BubbleBoy virus was the first virus which contained ActiveX code which bypassed the security and wrote a file to disk to the startup directory of Windows. That was all the email did. But when the computer was restarted, the file was started and the the virus started to propagate to everybody in the address book of the infected user.
After these viruses, the users knew they had to keep an eye open for attachments which were executables, and Word and other MS-Office related documents. Luckily, images and audio files (GIFs and MP3s for example) are still safe. So a quick visual inspection of the name of the attachment should tell if it is safe or not.
MS-Outlook, for example, doesn't by default display the full filename. So an attachment with the name test.doc would be displayed as a test with a Word document icon above it. And an attachment with the name test.gif.exe would be displayed as test.gif with an executable icon above it. If the user only checks the filename displayed, he would see the image filename and assume it was safe to open. Of course he would know immediately know he had ran a program instead of having opened an image, but the damage is done.
The payload of the Badtrans virus appeared under the filenames of README.TXT.exe and s3msong.MP3.pif. When the extension wasn't shown, the filenames looked innocent. When opening the attachment the executable was run. To soothe the user into thinking that the application hadn't ran, it showed a dialog box saying "File data corrupt: probably due to bad data transmission or bad disk access".
By sending infected emails through the mail application on the infected computer, MS-Outlook creates some side effects:
To overcome these problems, viruses started to be designed with their own SMTP engines. This way the ISP would be circumvented and would it be harder for the sender to find out where the infected emails were coming from.
Because the sender address could be faked now, more social engineering tricks can be performed. For example, email can be faked to come from the MAILER-DAEMON, which is normally computer generated email coming from SMTP gateways informing you that the email sent couldn't be delivered, and often attaching the full original email to the bounced message. Opening the attachment is one of the ways to find out which email wasn't able to be delivered.
To counter these kind of viruses, ISPs started to block all outgoing SMTP sessions except the ones coming from and to their own SMTP gateways.
The Frethem virus was the first virus with its own SMTP engine on board.
The MyDoom virus was one of the first viruses with its own SMTP engine on board and which also sends emails to common names (bill, john, mike etc) of target domains. The faked sender addresses will get the undeliverable messages.
The virus scanner on both the SMTP gateway of the ISP and the computer which retrieves the email should be able to open the attachments to check for viruses. But what if the attachment is an encrypted ZIP file and the key to open it is given in the email? Unfortunately, there is no clear method to prevent problems with these kind of email viruses.
The Mimail-M virus had an encrypted ZIP archive attached:
For unzip archiver download WinZip: http://download.winzip.com/winzip81.exe Password for archive is "kiss". Attached file: wendy.zip
Posted on 2003-11-25 14:18:03, modified on 2006-01-09 16:29:21
Tags: Networking, Rant
There is an old joke: The great thing about standards is there are so many to choose from.. This log is not about that but more about the point that if you stick to a standard you should implement it properly.
Comindico is one of the australian providers for dialin services. If you are an ISP the workflow goes like this: An user dials in to a Comindico terminal server, that terminal server asks the Comindico radius server for authentication, that radius server asks your radius server for authentication and the yes or no goes back the whole way to the terminal server which either lets you in or disconnects you. Works fine in theory, and mostly in real life too.
Your radius server can give more information to the Comindico radius server, for example an IP address and subnet mask. An maximum session time limit and your DNS servers. It all works fine, as long as you keep in mind that you take the right attributes and dictionary.
Comindico says "Please use Ascend-Client-Primary-DNS and Ascend-Client-Secondary-DNS for this". They are defined in the Ascend dictionary (number 529) as attributes number 135 and 136.
Except in the radius server from Comindico, there they are in the default dictionary.
With the result that their broken radius doesn't understand my perfectly legal answer with all the information in it. And I have to put these attributes in my default dictionary, where they will be overwritten the moment I update my software and the whole system will come apart if the IANA ever approves attributes 135 and 136 in the default dictionary.
Moral of the story: If you use an open standard, use it the way it was intended to be and don't invite your own wrapper around it.
This whole story wouldn't have been here if I wasn't reminded about this whole drama by the move to a new ADSL provider which is nothing more or less than a reseller of the Comindico ADSL services. Once we finally had the authentication of our users working, we couldn't get the DNS servers configured correctly because they haven't figured out the story above yet. If ever.
Standard compliant radius packet:
13:30:56.513559 172.16.1.10.1812 > 192.168.1.14.4738: rad-access-accept 62 [id 68]
Attr[ Framed_ipaddr{203.111.122.2} Framed_ipnet{255.255.255.255}
Vendor_specific{........X.} Vendor_specific{........X.}
Session_timeout{168:00:00 hours} ]
0x0000 4500 005a bba3 0000 3f11 b8ae dab9 580a E..Z....?.....X.
0x0010 cb6f 090e 0714 1282 0046 0eb6 0244 003e .o.......F...D.>
0x0020 6224 b0bb d92e 341e 14dd e2c2 b0ce abde b$....4.........
0x0030 0806 cb6f 7a02 0906 ffff ffff 1a0c 0000 ...oz...........
0x0040 0211 8806 dab9 5801 1a0c 0000 0211 8706 ......X.........
0x0050 dab9 580e 1b06 0009 3a80 ..X.....:.Comindico compliant radius packet:
13:28:51.958102 172.16.1.10.1812 > 192.168.1.14.4738: rad-access-accept 50 [id 67]
Attr[ Framed_ipaddr{203.111.122.2} Framed_ipnet{255.255.255.255}#136#135
Session_timeout{168:00:00 hours} ]
0x0000 4500 004e f27a 0000 3f11 81e3 dab9 580a E..N.z..?.....X.
0x0010 cb6f 090e 0714 1282 003a a842 0243 0032 .o.......:.B.C.2
0x0020 c1a0 ac29 4931 4fbf 3440 7714 9d52 c3ea ...)[email protected]..
0x0030 0806 cb6f 7a02 0906 ffff ffff 8806 dab9 ...oz...........
0x0040 5801 8706 dab9 580e 1b06 0009 3a80 X.....X.....:.Spot the difference. And be afraid.
Posted on 2003-11-21 23:47:26, modified on 2006-01-09 16:29:20
Tags: Coding, Networking, DHCP, DHCPDUMP
DHCPDUMP version 1.6 is released.
Fixed are:
- display of pad options
Added are:
- display of option 83 (and others)
- flushing of stdout after printing one packet.
Available via http://www.mavetju.org/unix/general.php.
Posted on 2003-11-18 00:14:19, modified on 2006-01-09 16:29:21
Tags: Networking
What started as two people on level 14 of the St James Hall building who couldn't work anymore at the end was nothing less than a whole floor who didn't have internet access.
Monday late in the afternoon I was experimenting with a guy from C. (the telephone company used by BarNet) to get an Voice over IP card working. Didn't work, at the end he pulled out all cables from the PBAX towards the router and went home.
At that same moment SJH level 14 disappeared from the network and nobody informed us that the internet didn't work again (this was 16:22)
The next morning, because nobody had informed us, it still was gone from the network. Interestingly enough, two people complained at our standby phone saying that their internet didn't work. A quick scan showed their IP addresses as 169.254.x.x, meaning: no answer from the DHCP server. Big mistery, I hadn't changed anything in the DHCP configuration for that subnet last night.
Michael had to go to court, so I jumped on the train to the SJH building and started to look what was actually happening. No answer from the DHCP server, no packets over the line, nothing. Two more people came to me saying that their internet didn't work. This was strange, suddenly the whole floor seemed to be out!
Level 13 and level 14 of the SJH building are on the same IP subnet, so why could one work and one not? The lights at the switch were on. Except for one: the one to the switch/router. Back on level 13 my biggest fears got confirmed: The interface towards level 14 didn't have a network cable in it. And the cable which came from level 14 was plugged in into a port on the switch on level 13.
Quickly plugging them back solved the problem. According to C., the company who had done work the day before in that room, they hadn't touched that cable. Gnomes. Network gnomes. Or people who don't want to say they screwed it up.
Posted on 2003-04-01 21:10:55, modified on 2006-01-09 16:29:20
Tags: Networking, DNS, dnstracer
This article regarding dnstracer was published in the SAGE-AU Advice Volume 9 Number 1 (March 2003) and The Journal of AUUG Inc. Volume 24 number 1 (March 2003)
The Domain Name Server system is a globally replicated and distributed database which primary translate hostnames (www.sage-au.org.au) into IP addresses (66.216.68.159), route mail (@sage-au.org.au) to mailhubs (sagemx.sage-au.org.au) and converts IP addresses (66.216.68.159) into hostnames (platypus.instaweb.com.au). Without it, we would have to use remember the IP addresses of the servers we want to connect to (telnet 131.155.132.36 4000) and it would be very hard to send emails as easy as it goes today (mcvax!moskvax!kremvax!chernenko).
Normally you don't have to worry about DNS, you just get the settings for the nameserver you have to use via PPP when dialing into an ISP or via DHCP when connecting to a LAN at a company. They make sure that their nameservers know where to get the rest of their data, which are initially the root-nameservers.
The root-nameservers are the 13 (13 logical, but physical more) most important nameservers on the internet. They know where the rest of the DNS servers can be found.
Furthermore you have master and slave servers for a domain: the data for a domain is only manually changed at the master, the slaves transfer the data via the internal DNS mechanics.
If you're requesting the IP address of www.sage-au.org.au your nameserver will ask one of the root-servers for it. It will reply that it doesn't know it, but that the answer can be found at the DNS servers for .au and supplies a list with them and their IP addresses (The list is known as Authority Data, the IP addresses are known as Additional Data). Your server will ask the question again at one of the servers responsible for .au and get a similair answer: it doesn't know it, but it hands you a list of servers for .org.au and their IP addresses. This goes on until you're at the servers which are responsible for sage-au.org.au, in which case you get the IP address of www.sage-au.org.au (Answer Data).
If you're requesting the IP address of www.sage-au.org.au your Your server now caches the data for .au, .org.au, .sage-au.org.au and www.sage-au.org.au for a short time (the Time To Live) so that following requests for that data doesn't need to explore so much, it just can do a quick lookup of in it's own cache and returns the answer.
The DNS system is not really a SPOF, it is designed as a globally replicated and distributed database which means that if you can't reach one of the servers, you can try it at a different one. As there are 13 root-servers which know where to find the rest, there are 6 servers for the .au domain (6 logical with a total of at least 8 IP addresses), there are 9 servers for the .org.au domain and two servers for the sage-au.org.au domain. The location of the servers on the internet and replication is used to overcome connectivity problems. Regarding the network, there isn't much which can go wrong. Regarding the administrative side of it, that's where things go wrong.
When you register a new domain, you are asked what the nameservers are and if necessary also the IP addresses. Furthermore, these nameservers have to be configured to answer requests for that new domain and to exchange information between them. And actually data has to be served on that domain. Five places for things to go wrong!
At the time of writing, one of the domains of a nameserver for .org.au has expired (for people interested: optus.net has expired at December 16th 2002 and after half a month it still hasn't been re-registrered). That means that the IP address of the nameserver audns01.syd.optus.net can't be found and that this server will never be queried (after all, if you don't know an IP address you can't connect to it)
Changing the IP address of a nameserver is a pain and often it will be forgotten on one or two machines (Remember that switch in the cupboard which got installed a long time ago? Yes, that one too has the IP address of the DNS server hardcoded). Or that the registrar makes it impossible to change the IP address of the nameserver via their website because of all kind of internal checks.
Lame servers are servers which are mentioned in the NS records for a domain but are not authoritative for that domain. This can happen because of a typo in the IP address or a change which has never been fully finished (new server added while it wasn't ready or old server data removed but never from the NS records).
Stealth servers are servers which are not mentioned in the NS records but are authoritative for that domains. For example servers which have been removed from the NS records but the configuration of the server never updated.
When data is changed on the master server, the slaves will have to transfer it from there. But sometimes they can't because the master has disabled it for some reason. In that case the data on the slaves will get more and more obsolete.
DNS server software has strange habbits and one of them is often that if you end a name without a dot, it will add the current domainname to it. So if you see a zonefile with www.sage-au.org.au.sage-au.org.au, you know that they forgot to end it with a dot at the end.
Remember the traceroute(8) utility? It shows the path an IP packet takes when you send it to its destination IP address. Remember ntptrace(8)? It shows the path of NTP servers which your NTP client is syncing on. Dnstracer is something similair, it shows you where a DNS server will go for its information. So if you want to know the path to www.sage-au.org.au:
[~] edwin@k7>dnstracer -s . -o www.sage-au.org.au Tracing to www.sage-au.org.au via A.ROOT-SERVERS.NET, timeout 15 seconds A.ROOT-SERVERS.NET [.] (198.41.0.4) |\___ SEC3.APNIC.NET [au] (202.12.28.140) | |\___ ns3.melbourneit.com [org.au] (203.27.227.10) | | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) Got authoritative answer | | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) Got authoritative answer | |\___ ns3.ausregistry.net [org.au] (203.18.56.43) | | |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | | \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | |\___ ns2.ausregistry.net [org.au] (203.18.56.42) | | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | |\___ ns1.ausregistry.net [org.au] (203.18.56.41) | | |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | | \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | |\___ audns01.syd.optus.net [org.au] (No IP address) | |\___ au2ld.csiro.au [org.au] (130.116.2.21) | | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | |\___ dns1.telstra.net [org.au] (203.50.5.200) | | |\___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | | \___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | |\___ box2.aunic.net [org.au] (203.202.150.20) | | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) | \___ ns4.ausregistry.net [org.au] (210.8.15.253) | |\___ ns2.sage-au.org.au [sage-au.org.au] (130.102.171.100) (cached) | \___ ns1.sage-au.org.au [sage-au.org.au] (203.27.221.52) (cached) |\___ SEC1.APNIC.NET [au] (202.12.29.59) | |\___ au2ld.csiro.au [org.au] (130.116.2.21) (cached) | |\___ dns1.telstra.net [org.au] (203.50.5.200) (cached) | |\___ box2.aunic.net [org.au] (203.202.150.20) (cached) [...] ns1.sage-au.org.au (203.27.221.52) www.sage-au.org.au -> 66.216.68.159 ns2.sage-au.org.au (130.102.171.100) www.sage-au.org.au -> 66.216.68.159
Just like expected: the server goes to a root-server, the servers for the .au domain, the servers for the .org.au domain and the servers of the .sage-au.org.au domains. The answers received are printed at the end and they agree on it.
Sometimes it will go wrong, for example when a lame server is detected:
[~] edwin@k7>dnstracer -o -s RELAY-1.FTEL.CO.UK fataldimensions.nl.eu.org Tracing to fataldimensions.nl.eu.org via RELAY-1.FTEL.CO.UK, timeout 15 seconds RELAY-1.FTEL.CO.UK (192.65.220.24) |\___ ns.cistron.nl [nl.eu.org] (62.216.31.55) Got answer |\___ ns.lf.net [nl.eu.org] (212.9.160.1) Got answer |\___ ns.eu.org [nl.eu.org] (137.194.2.218) Lame server |\___ ns2.ispi.net [nl.eu.org] (206.131.193.15) Got authoritative answer |\___ ns.patriots.net [nl.eu.org] (206.131.200.40) Got authoritative answer \___ auth1.dns.elm.net [nl.eu.org] (81.17.34.251) Got authoritative answer [...]
The difference between "Got answer" and "Got authoritative answer" is that the first one can be a cached answer, while the second one is one from a server which admits that its responsible for that domain.
See http://www.mavetju.org/unix/dnstracer.php for more information about the dnstracer utility and how to obtain it. For FreeBSD and OpenBSD, it is in the ports-collection. For Linux, there is an RPM for it. Otherwise, just grab the tarball and compile it.