MavEtJu's Distorted View of the World - Riverbed
Book: Troubleshooting Riverbed Steelhead WAN OptimizersDifferent kind of networking people
A proper WAN Optimization analogy
Buying Riverbed things on eBay
AusNOG04 talk slides
Riverbed Blog Authorism
Riverbed Certified Solutions Professional
Training in San Francisco (part 4)
Training in San Francisco (part 3)
Training in San Francisco (part 1)
Back to index
Book: Troubleshooting Riverbed Steelhead WAN Optimizers
Posted on 2014-05-20 08:00:00
Tags: Riverbed, Books
My first book, named Troubleshooting Riverbed Steelhead WAN Optimizers, has been published officially. Well, made available to Riverbed customers with access to the Riverbed Support website. It has been a multi-year effort and as usual the most difficult part was to get approval for external publishing.
The things that went smooth were the editing and the publishing: That approach was taken more or less from how the FreeBSD Project makes their documentation: The source is based on DocBook XML (Was DocBook SGML when I started) which compiles it into various formats like HTML, PDF and ePub. This makes sure you have the style and the content separated, so that even if you make certain style changes afterwards you just do it in the style sheets and it automatically propagates through the whole output without the fear of missing a part somewhere in the text.
But after two chapters of that SGML/XML, I had enough of the excess of markup tags like <para> and <sectx> which presence made my automatic reformatting habbit spending more time on the smoothness of the tags than on the content. Time for a different approach!
In the past I have worked with HTML, various *roff formatters and Perl's perldoc and ended up with a mixture of things: Use the *roff .-prefix to initiate a command, use the HTML h1-h5 and i/tt/b kind of tags for the commands and use the automatic approach of perldoc to know where the paragraphs end. And then you end up with something like:
.h2 Steelhead xx20 series models The xx20 series models consist of a set of desktop models, 1RU rack models and 3RU rack models. To upgrade between models, except [...] .h3 No route to host A .i No route to host error is shown when the remote host does not exist and the router on that subnet sends back an ICMP Host Unreachable message or when
After that a convertor changes it into a real DocBook XML document:
which then gets converted into HTML or PDF.<sect2 ><title>Steelhead xx20 series models</title> <para>The xx20 series models consist of a set of desktop models, 1RU rack models and 3RU rack models. To upgrade between models, except [...] <sect2 ><title>Telnet failure scenarios</title> <sect3 ><title>No route to host</title> <para>A <emphasis role="italics">No route to host</emphasis> error is shown when the remote host does not exist and the router on that subnet sends back an ICMP Host Unreachable message or when
Easy editing, easy markup, easy publishing. That worked fine. The book was used internally at Riverbed for training the new TAC engineers and everybody was very excited about it.
What didn't go smooth was to get the approval to publish it to customers. Everybody saw the value in the book, realized that education at this part could help releasing pressure on the TAC engineers and improve the customers Steelhead experience.
But nobody knew how to take up the book and help to get approval to make it available for the outside world. People who promised to help me moved outside Support or outside Riverbed, I was pushed around by various departments to the same various departments which already told me earlier that I should talk to the people who just send them back to me. It shouldn't be so difficult, the TAC does have access to the full contents of their own website and the TAC does publish KB articles of various quality levels without going through proper review before they get published.
So after three years of editing and publishing internally and trying to get it to the outside world, it has finally been done: My first book named Troubleshooting Riverbed(r) Steelhead(r) WAN Optimizers is available for the Riverbed customers. You can download it at http://rvbd.ly/1p5MMgu or http://supportkb.riverbed.com/support/index?page=content&id=S24051.
No comments | Share on Facebook | Share on Twitter
Different kind of networking people
Posted on 2013-08-09 18:00:00
Tags: Rant, Networking, Riverbed
In the first thirteen years of my working life I have encountered a lot of different people in the field of networking. And for some reason they were all skilled, experienced and willing to learn. They understood their stuff, in case of issues a pointer to the right direction was enough to help them out.
In my experience at the Riverbed TAC I have encountered several new kind of networking people, although I wouldn't call them all "networking" people.
- You have the ones described above: Call in with an issue and just need a clue on where to start looking. They know their network and they know how the WAN optimization works. When looking at packets together and explaining what can be seen, they already know where to look before I can ask them "Is there anything in the network which would strip these TCP options?".
- Outsourced network manager and network architects: Oh how do I pity you. Your company got this multi-million dollar contract for the network, but WAN optimization is more than the network. It is the servers and the clients too, but because you only got the network you are not allowed to talk to the company which is managing the servers and the company which is managing the end-user support. The company which is managing the servers will not tell you that they upgraded to a newer version of whatever application and blame the fact that half of the company can't retrieve their email anymore on you! You will never be able to optimize encrypted MAPI traffic because the team of the other company which manages Active Directory doesn't want your Steelhead appliance to be able to perform delegation, let alone be configured as a read-only Domain Controller...
- You have people who have inherited a network and just found out that besides routers, switches and firewalls you have another kind of network equipment and they have no idea what... These are people who can be saved, if introduced properly. Explain what WAN optimization is, give them documentation, explain what the issue is and follow up a week later. If it works out, they will end up as the first group. If not, they will end up as the next group...
- Outsourced NOCs. They seem to look at a screen and when there is a different colour than green they will identify the brand of the device and call the TAC. They refuse to do basic troubleshooting themselves nor learn from previous cases opened and need to be asked for the system dump every time they open a case. You will get emails from them every eight hours with the request for an update because that is when they handover their cases but don't bother to forward the next steps to the original customers because they are not online yet... Getting information from the device in question goes most of the time without a problem. However getting more information about the issue will have to go through the NOC, who rephrases it and forwards it to the user, who answers the wrong question and sends it to the NOC and who forwards it back to you...
Did I miss a category? Most likely because I have repressed them, very deep...
No comments | Share on Facebook | Share on Twitter
A proper WAN Optimization analogy
Posted on 2011-03-29 18:00:00
Tags: Networking, Riverbed
Even after working for Riverbed Technology for two and a half years now, I still have to come up with a bullet-proof analogy of how WAN optimization works.
Consider a network from one side of this planet to the other side: A round trip time of 300 milliseconds for 20 million meters. Of that 300 milliseconds, you have two factors: The speed of light to get from A to B, and serialization delay which happens at every hop and is related to the bandwidth and the size of the packet. One is constant, the other one is variable.
To move data in a stream, the throughput is limited to the smallest bandwidth in the path. Although you can move data faster on other parts, it all has to go through this one.
As an example, say you have a stream of 300 Mb and a smallest bandwidth of 2 Mbps. Without any protocol overhead, this will take 20 minutes to go through there. With a fourty byte protocol overhead and an MTU size of 1500, this will take 20 minutes and 33 seconds.
Now with WAN optimization. It consists of three parts: Optimization on TCP level, which has been ignored for now. Latency optimization on application specific protocols, which has been ignored for now. And data optimization, where the data is either compressed or only referred to. If the same or similar data gets transfered via two WAN optimizers twice, the first time you would get a relative small reduction factor, depending on the compressability of the data, while the second time you would get a large reduction factor because the data patterns is alreayd known on both devices.
If that 300 Mb is split into segments of 1024 bytes, making it 300 000 segments, and each segment has a 64 byte label, you end up with only about 19 Mb worth of labels.
Transfering that 19 Mb through a 2 Mbps link will be take 80 seconds, about 15 times faster.
Now back to the topic: A good analogy for WAN optimization.
Is it faster than light? It feels like it, but the Round Trip Time of the WAN is still the same. And the speed the packets go via is still the same.
Is it a "wormhole"? Wormhole-based paths which are shorter than a non-wormhole-based paths. The path travelled travelled for optimized traffic still has the same distance.
Is it comparable with ships, where goods are stored in containers (labels) and then transported in large bulk carriers? If the speed limit of other ships was limited to the speed limit of the bulk carriers, then it would be a good start.
Is it a train analogy, where passengers are cramped into carriages and efficiently transported across the rail network? It could be, except that on the railroad network everything is put into train carriages and transported efficiently on it. Comparing it with the French TGV and the Japanese bullet trains does not work neither, because the speed of the packets is still the same while these trains are way fast.
So, the analogy needs to use the same speed limits on the transport mechanism, and needs to give the impression that the delivery gets faster without changing the distance.
The best thing I come up with is transport of goods via large trucks instead of via small delivery vans: Goods are shipped via small delivery vans to a distribution point, stored into a single large truck which then uses the same transport infrastructure as small fast vans would have used if they would have transported their payload. Instead of a long convoy of small vans, you get one truck towards the distribution point which there gets reloaded into numerous small vans. The only thing which does not make sense yet is that small delivery vans are often 2 x 4 x 1.5 meters and big trucks are 3 x rather long x rather high, which gives the impression that size still matters while this isn't the case on WAN optimized traffic...
That is the problem if you work with magic :-)
No comments | Share on Facebook | Share on Twitter
Buying Riverbed things on eBay
Posted on 2011-01-10 23:00:00
Tags: Humor, Riverbed
Just for the fun of it, I checked out some Riverbed appliances and hardware on eBay.
- DELL RIVERBED SERVER 3GHZ 2GB RAM - Nice piece of hardware for 188 dollars. You can't use it as a Steelhead appliance anymore: "Operating System Not Installed". Just put FreeBSD or Linux or it and you have a great server.
- Riverbed Steelhead Central Management Console 8000 - For 500 dollars you have a Dell PowerEdge 850 only, because: "unit is unlicensed" which means that the CMC software won't function. And before you can get the license keys from Riverbed, you need to get a support contract for it.
- Riverbed Dual Copper GIG-E Bypass PCI-X CMP-00028 - For 25 dollars you get a nice bypass card which only works on the xx10 and xx20 series hardware, not on the xx50 hardware. Make sure you check the Bypass Card Installation Guide before you buy these things!
- Riverbed Steelhead-2000 P4SCE 2U Server w/P4 2.2GHZ/2GB - For 500 dollars you have a chassis without any capabilities, because "hard disk drives are not included.". That is not a Steelhead appliance, that is a normal computer.
- Used Riverbed Steelhead 2010 WAN Acceleration Appliance - Nothing spectular here, except for the picture on the side. With a picture like this (despite the bad photoshopping), who can resist such a deal?
- RIVERBED SHA-06020 STEELHEAD MODEL 6020 - Imagine paying 4000 dollars for a 120 thousand dollars box. Oh, but it doesn't come with any harddisks. Which means you have to buy new harddisks from Riverbed. But a support contract first. Plus you still have no way to get the operating system on the machine. So you buy a computer, not a Steelhead appliance. Again.
So far no luck, still haven't been able to find a full working Steelhead appliance on eBay. Time to go to bed!
No comments | Share on Facebook | Share on Twitter
AusNOG04 talk slides
Posted on 2010-09-16 13:00:00
Tags: AusNOG, Riverbed
The slides of my talk at the AusNOG04 conference about "WAN Optimization - Changes to more than just a box".
No comments | Share on Facebook | Share on Twitter
Riverbed Blog Authorism
Posted on 2009-10-12 22:00:00
Tags: Riverbed
Guess what! After a "Social Media Introduction Course" last week, today I've been approved as an author for the Riverbed Blog Website.
I still don't know what the first article will be about, but I will give it my best try (Did you expect anything else here?)
No comments | Share on Facebook | Share on Twitter
Riverbed Certified Solutions Professional
Posted on 2009-02-25 09:30:00
Tags: Riverbed, Rant
It seems a long time ago since I've started at Riverbed TAC, but it's only four months (three months if you consider that the original training didn't finish until I was there for four weeks). So far so good, I completed the probation period of three months and this morning I sat for my RCSP exam.
I don't really like the style these exams are being taken in: You get 70 questions, some with one correct answer and some with multiple answers you need to pick. So after 30-45 minutes, you have enough of it and want only to know one thing: Do I have to come back next week or can I not worry about it for the next two years.
But the final result doesn't matter, you are given a list of topics and the percentage of how much of each topic you had correct. So you don't know which ones you had wrong. Or if the ones on which you doubted yourself on were correct or not. It is just a snapshot of how much you know now and if you get a lucky batch of questions you pass. These exams are not for learning from your mistakes, not meant to improve yourself.
Now the good news, I don't have to do it again for another two years!
No comments | Share on Facebook | Share on Twitter
Training in San Francisco (part 4)
Posted on 2008-11-20 19:00:00
Tags: Travelling, Sunnyvale, Riverbed, Trains, Project VP
Part two of the training (hands on, dirty details, procedures) was done in the Sunnyvale offices of Riverbed. Were the offices in San Francisco in the middle of the city (well, mostly), these ones were in the middle of Sunnyvale. Going there would be simple, grab the BART train, grab the Caltrain train and grab a taxi for the last two kilometers. I mentioned the issues with BART earlier, now its CALTrains turn.
Caltrain was GREAT! It started with the buying of the ticket where I didn't get 14 dollars back in quarters, I got 14 dollars back in dollar coins. I knew about the existence of silver dollar coins (collector items), but that was the first time I have seen normal dollar coins. Later in the week I found out that it costs the US government about 500 million dollars per year to replace all the worn-out dollar bills while it would only cost a fraction of that if they moved to dollar coins which would last about 30-40 years before they need to get replaced.
The second great thing was the bell on the front of the train, which gets sounded when it arrived at a station or a railway crossing. The third great thing were the railway crossings, something which I haven't seen for a long time but remember from an earlier life: One in eastern Eindhoven and one in Geldrop. Bing-bing-bing-bing they go when the train drives by. The fourth great thing was the presence of a train conductor which was there for questions and support. I am not sure if they are on the CountryLink trains in Australia, but they surely are not in the Sydney CityRail trains.
The design of the interior of the train is euhm... interesting. (Fifth great thing!). To get in the train you have to climb from the platform (which is at a certain height) up three steps into the train (which is thus higher). That is the ground level of chairs, in two rows with each two chairs on your both and left hand side. Then the upper level of the train, which is accessible via a step of stairs leading you to about one-and-a-half meter above the ground level. That distance is not high enough to make it possible for the people on the ground floor to stand up, so they have upstairs only one chair per row, both on the left and right hand side of the train, and a huge gaping hole in the middle of the upper level. Like I said, interesting design but not worlds smartest I think.
Two zones later, from Millbrae to Sunnyvale and I was at the place I would call home for the next week: A Best Western hotel. The week before I was in a hotel in inner San Francisco, now a Best Western hotel. Let me describe the differences:
| San Francisco | Sunnyvale |
|---|---|
| Queen-size bed | King-size bed |
| Shower and bath | Shower and spa |
| Internet (wired and wireless) | Internet (wired and wireless) |
| 42 channels on TV | At least 99 channels on TV |
| Movie on Demand ($$$$) | Video library (Free!) |
| Breakfast ($$$$) | Breakfast (Free!) |
| Dinner / Bar | Pizza / Starbucks near the supermarket |
| Soda vending machine | Soda + Snack vending machine |
| 10 minutes walk to work | 10 minutes walk to work |
| San Francisco | Sunnyvale |
| US$ 279 per night | US$ 119 per night |
On average, Sunnyvale was better for the money, but San Francisco had the location. And less crap on TV :-)
Because the availability of the video library, I took the opportunity to watch some old movies: Timeline by Michael Crighton (for medieval castle lovers), Snatch (very funny movie) and Rat Race (always good for a laugh). There was a fourth one but I can't remember anything about it.
Just as I met up David Thiel and Anton Holleman in San Francisco, I met up with people here too: Jos Backus, former colleague of Origin and FreeBSD enthusiast, Xen Li and Marcel Moolenaar, both FreeBSD developers. Thanks for the hospitality guys!
The time in Sunnyvale flew by and before I knew it was time to go back to Australia. This time I was lucky on the plane: The two people next to me didn't show up and I moved from an aisle seat to a window seat (I did offer it first to a mother with her child but she refused for some silly reason) and I slept for about ten hours on the plane. Business class comfort for an economy class price!
At home, Project Vegetable Patch needed moral support and some pruning, but it has survived my absence. Little Dirk had his hair cut, Hanorah has learned to walk and Naomi was very glad to have me back and take care of the kids for the rest of the weekend :-)
No comments | Share on Facebook | Share on Twitter
Training in San Francisco (part 3)
Posted on 2008-11-08 19:00:00
Tags: Travelling, San Francisco, Riverbed, Steelhead
This course about the Riverbed Steelhead appliance has given me some very interesting details of the design of the Steelhead appliance and the capabilities. The speed improvements on network traffic, either due to initial compression, or later when it detects patterns it has sent out earlier or because it optimises protocols like CIFS, NFS or HTTP(S), their are very impressive. As they say, it makes your data on the servers on the data-center feel like it is sitting under your desk. Plus the integration in your network makes it close to zero-administration, all you need to start is two Steelhead appliances, two ethernet cables, two IP addresses and their default gateways and it's up and running. Talk to me if you want a demo :-)
What happened the rest of this week? The USA got a new president, I met up with David Thiel for dinner on Sunday, I met up with Anton Holleman for dinner on Wednesday and Saturday, I met a lot of collaegues at the San Francisco Riverbed offices, I walked in San Francisco through Chinatown, the Financial District, to the Tram museum, around the piers at the east coast because I couldn't walk over the Bay Bridge (Whoever decided to make a bridge without a pedestrian lane?).
Television is here euhm... less than interesting. Except for PBS there aren't much channels I did recognize or would recommend. Discovery Channel has "in-taxi" gameshows, there are seventy channels with people competing for something and then giving or getting feedback on how they were doing and half of the TV series are an insult to my brains. The presenters of the Weather channel seem to be the most enthousiastic ones in their profession. At least in Australia there is 40% of the channels worth watching, it's about 2.5% here. Not to mention the quality of the surfaces with the "same" colour on NTSC... (enough for now, it just makes me open the window and shout "I am sick of this and want everybody to know")
The best pub I've been in was Eddie Rickerbacker's on 133 2nd street. It has old motorbikes hanging on the ceilings, miniature trains in the vitrines and a miniature railroad around the wall. And a fat cat walking around... A very nice place to be.
So what is left for the rest of the week? Tomorrow I will go to Sunnyvale, first with the BART train service and then with the CalTrain service (Yes, I love trains!). Over the week I will have my share of cases at the Riverbed TAC in Sunnyvale.
No comments | Share on Facebook | Share on Twitter
Training in San Francisco (part 1)
Posted on 2008-10-31 07:00:00
Tags: Travelling, San Francisco, Riverbed, Steelhead
In the next two weeks I will be on training for my new job at Riverbed in San Francisco and Sunnyvale. It will be two courses, Steelhead Appliance Deployment and Management and Steelhead Mobile Client Installation and Configuration.
A Steelhead Appliance is a device in your network, one centrally at your servers and one or more remotely on the user LANs, which provides WAN optimisation, WAN acceleration and Wide Area File Services. Or in plain English: It makes the data over your networks be transferred faster.
The best comparison for it is your webbrowser: When you initially a page with images, it will download them all from the webserver and store them in a local cache. If the next page you go to refers to these images, it will just use them from the cache instead of downloading them again. That is Wide Area File Services.
The next one, WAN optimisation, is the HTTP protocol. The old version setup a new TCP session for every HTTP request, the new version can continue on an earlier used TCP session.
And the last one, WAN acceleration, is compression of the static HTML pages by the webserver: Compressed data is often smaller, so it will be there faster.
The Steelhead appliances don't only work on the HTTP protocol, but also (including but not limited to) with CIFS (Windows File Sharing), MAPI (Exchange server), NFS. And for protocols it doesn't know about, it will still do compression.
That is the hardware version, which works on your WAN/LAN infrastructure. There is also the version called Steelhead Mobile for on notebooks computers, it will give you, at home over a VPN for example, the same performance boost as you have on your LAN at work.
Like I said, the first week is training, the next week is for getting my troubleshooting experience. From what I've read up so far it is pretty technical stuff I will get trained with, so it can't be all bad :-)
If you are in San Francisco between 2 November and 9 November or in Sunnyvale between 9 November and 14 November and want to meet up, drop me an email!
No comments | Share on Facebook | Share on Twitter