Improve Network Stability By Preventing Loops

tiger-and-turtle-52691_640Network stability is very important to your users.  If your network is stable, your users will forget that there even is a network.  This is the goal really.  For the network to work so well that it is taken for granted by the users.  If your network, however, is not configured correctly it can become unstable, start crashing frequently, or just run slowly.  A major culprit when it comes to network crashes, instability, and slowness is the network loop.

What is a Network Loop?

A Network Loop is the result of network switches or hubs being plugged into themselves or into each other more then once.  When switches connect this way, network packets can be bounced between two switches infinitely.  In certain circumstances they can even begin to be duplicated over and over again.  Doubling is number each time.  This problem happens because of a quirk with how networks function.

If a switch doesn’t know which port to send a packet out, it will actually send the packet out all its ports except for the port it received the packet on.  It will do this to ensure the packet will get to its destination.  Once the switch learns which port a computer is connected to it will stop sending packets to that computer out all its ports and only send them out the port the destination computer is on.  This behavior works really well and can automatically optimize the network.  But, this structure breaks down when there is a loop.

Simple Two Switch Example

Lets do a quick rundown of what happens on two switches when a packet is sent out and a loop is present.  When switch “A” sends out a packet destine for a computer it hasn’t seen before it will send the packet out all its ports except for the port the packet came in on.  In this case, switch “B” will receive TWO copies of the packet, one copy for each cable connecting the two switches.  Remember, because there is a loop, there are two or more cable connecting these switches.  Now, switch “B” has two packets it has to send out, and if it, like switch “A”, hasn’t seen the destination computer on the network before it will send each of the two packets it has out all of its interfaces.  In the process it sends copies of the packets back to switch “A”.  Both switches will repeat this process as quickly as they can and begin to flood the network with copies of these packets causing slowness or even complete outages for users on the network.

Two Technologies To Help

Switch manufactures have come up with 2 ways to address this issue.  Spanning Tree and Loop Protection.  Each method addresses a slightly different situation so you should implement both techniques on your network.

Spanning Tree

Spanning Tree works between managed switches.  You configure it on your switches and it will allow you to actually connect multiple links between switches.  When you do this, spanning tree will disable any redundant links automatically to prevent loops.  What this feature allows you to do is plug in multiple cables between key switches and create fail over paths.  So, should one of the cables break, spanning tree will turn back on the other cable and traffic can continue to flow even though you have a broken cable.  It should be noted that spanning tree runs on a per-VLAN basis rather then a per-port basis.  So to really have protection against loops, you will want to enable spanning tree for all your VLAN’s.

Below I cover how to enable this feature on the most common types of both Cisco and HP switches.  Though the specific commands are little different, the concept of how the feature is enabled is really the same for both manufactures.

Cisco

To enable spanning tree on a Cisco switch do the following.

To disable spanning tree on a Cisco switch do the following.

HP

Enable spanning tree on an entire HP Procurve switch by doing the following.

Enable spanning tree on a specific VLAN on a HP Procurve switch by doing the following.

Disable spanning tree on HP Procurve switches by doing the following.

Loop Protection

Loop protection is a feature that allows you to limit the damage a loop may cause when you have unmanaged switches on your network.  An unmanaged switch is a switch that has no configuration options.  Think, switches you get at Best Buy for your home.  They do the trick in really small setups, they are technically switches because the learn and switch packets based off computer MAC addresses, but beyond that unmanaged switches have no additional features.

Loop protection allows managed switches to cut off access to the network for any segment that it detects a loop on.  So, if someone connects an unmanaged switch to your network and then connects that switch to itself, creating a loop on the unmanaged switch, a switch running loop protection will disable the network port the unmanaged switch is connected to.

This does cause an issue for computers connected to the unmanaged switch that has the loop; however, it prevents the problems created by the loop from spreading throughout the network and bringing down the entire system.

Here is how you configure loop protection on Cisco and HP switches.

Cisco

Step one: Your done.  Cisco calls their loop protection feature loopback detection and it is enabled by default.  You can turn the feature off if you want (not recommended) by doing the following.

If you turn it loopback detection off and want to turn it back on, or simply want to make sure it is enabled issue the following command to turn it back on.

HP

HP calls their loop protection feature loop protect.  On HP Procurve switches you will do the following to enable loop protect.  These commands are issued in the “global config” mode.

  1. Configure the switch to use loop protect on an interface or VLAN basis
  2. Set a disable timer so the switch knows how long to keep the port offline once it sees a loop
  3. Enable loop protect on specific ports or VLANS.

Loop Protect on Ports

I prefer configuring loop protect to work on a port basis.  It is more granular in my opinion and feels more intuitive to me.  Here is code chunk on how to configure loop protect on a port basis.

Loop Protect on VLANs

I recommend configuring loop protect on a per-port basis, but if you want to configure it on a per-VLAN basis here is code chunk for you.  When configured on a per-VLAN basis loop protect will be enabled on any port that is running the specified VLAN.

Conclusion

Spanning tree and loop protection can go a long way toward improving the stability of your network.  I recommend using these features because turning them on is simple enough and doing so can really improve the experience your users have on your network as well as make life a little easier on you.

Do you plan on or have you already enabled spanning tree and loop protection on your network?

5 Documents Network Administrators Should Maintain

Avoid Confusion and Unnecessary Documentation

Network documentation can be a pain to maintain.  People often fall into a trap by thinking they are done installing a device or fixing an issue once things are online.  Really, they aren’t done until they’ve documented the work.  Trust me, no, you won’t remember what IP address was assigned to some random device in a year.  No, you aren’t going to remember where that one-off wireless access point was placed in 18 months when it is giving you problems.  So, help yourself out and acknowledge that you need to do some documentation.  But which documentation should be done?

Documentation Summary

Some documentation is just a waste of time while some is invaluable and referenced all the time.  This article is here to save you some time and show you which documentation to focus on.  Over the years, I’ve found the following documentation to be the most useful.  Focus on these 5 documents and you will get the most benefit for the least amount of effort.

  1. Inventory – Inventory your switches, routers, firewalls, servers, printers, desktop computers, etc. so you know what you have.
  2. Layer 1 Diagram – This diagram will show what devices are physically plugged into other devices.
  3. Layer 3 Diagram – This diagram will show the logical structure of your network.  It will identify device IP addresses, subnets and their uses, security measures and where they are in the “flow” of traffic, etc.
  4. IP address charts that show what IP’s are statically assigned to systems like servers and printers, what pools of addresses are being used for client computers, and more.
  5. Floor plans with equipment location – Just little marks on whatever map of the building you have will save you loads of time.

Documentation Details

Inventory

An inventory of equipment should go without saying.  You can maintain an inventory in a spreadsheet, in help desk software, or other management software.  Just pick a tool and be consistent with it.  In your network equipment inventory you should include at least the following information.

    • Manufacturer
    • Model
    • Purchase Date
    • Primary MAC address
    • Assigned IP address
    • Location
    • Access Method (HTTP interface, Remote Desktop, SSH, etc.)
    • Notes

Layer 1 Diagram

A “layer 1″ diagram is a diagram that maps out the physical links between all your network devices.  Layer 1, in this context, refers to the physical layer of the OSI reference model.  In this diagram you should have icons for each of your devices and lines connecting them all together.  These lines should be labeled with the interface information from each device so you know what port connects to what port on the next network device.  Using a feature like CDP or LLDP is very handy when crating this diagram.  CDP and LLDP both give you information about any device that supports CDP or LLDP which is directly connected to the device you are currently logged into.

Layer 3 Diagram

A “Layer 3″ diagram is a diagram that shows the IP addresses of all your routers, firewalls, and other layer 3 devices as well as how they interconnect and how the subnets are laid out on your network.  Layer 3, in this context, refers to the network layer of the OSI reference model.  This diagram should be similar to your layer 1 diagram in look but will have different information on it.  Here, you only have to document devices that separate subnets like firewalls, routers, vpn concentrators, etc..  Mare devices with a square or circle and label them. The interconnect them with lines if they share a subnet.  Label the subnet and the IP for each of the devices on the line.  If there is a subnet off of a router that has no other routers, this is called a “stub”, then just draw a line off your router icon and label the subnet.

IP address charts

This is a document that has all the network subnets that you maintain and all the reserved IP addresses that each of these subnets has.  So, when you create a server subnet, wireless users subnet, etc. you would use this document to design and maintain the address space.  This document is also where you should keep what IP’s you are assigning statically to any device.  When you give that new server an IP address, note it here (and in the inventory).

The tool that you use for this doesn’t matter much for small shops, but if you are growing at any rate I would recommend a purpose built tool.  An IP address management tool (aka IPAM) can be used but they can be expensive.  I have found a free tool called racktables that has done the job well enough, so I would recommend starting with racktables and see if you like it.  It is a software package you install onto a server and access the info through a web interface.  NOTE: If you use an online tool like racktables make sure you keep the occasional backup locally on your desktop or laptop.  I print to pdf every so often just to have something in an offline formate.  This backup process doesn’t have to be to formal, just do something from time to time.  If you are having a network issue and need access to this information you don’t want your only copy to be accessible only through a broken network!

Also, it may be tempting to automate this document and have realtime DHCP information populate the tables and update them automatically.  I recommend against that for this specific document.  The reason is that this document is not supposed to tell you the current state of things is but rather what the current state of things should be.

Floor plans

This is really helpful in finding that random wireless access point you installed 18 months ago.  Trust me, you are not going to remember where you put it.  So, just document it real quick on a floor plan of some sort and move on to more important things.  In addition to being useful to you as the person responsible for the network, this type of document is a huge help to management, external contractors, as well as system planning and design.

All that you have to do is get a floor plan from someone that would have a copy, maintenance/owner/city hall…, and come up with your own basic key (AP’s are triangles, switches are squares, etc.), create a key so others can read the chicken scratch and start marking up the map.  These documents don’t have to have exactly which AP is in what location, though that is helpful.  They can just simply indicate that there is an AP there-there-and-there.  You don’t need to over do this document.

Closing Notes

Key to keeping these documents up to date without feeling like they are eating up all your time is to include them into your normal troubleshooting, equipment replacement, etc. processes.  Just tag “document” onto the end of whatever process you already follow.  Spend a couple weeks deliberately documenting after each task is completed before you move on to the next.  It will be a habit before long and you will be so thankful that you have the documentation when you need it.  It is well worth the time to maintain these 5 simple documents.

Tell us your story!  What documents do you prefer to maintain?  Have you had issues recently where one of these documents could have helped?

5 Most Common Causes of Network Slowness

The Problem

I shiver whenever I hear: “The network is slow”.  I would rather have things either be broken or not broken.  Troubleshooting a grey area like “slow”, especially when it is subjective, can be a test in troubleshooting skill and patience.  So, I’ve made this post to help you out when you inevitably run into this complaint from clients on your network.

The Summary

Over the years, I have noticed various themes when it comes to network slowness.  The following 5 things are what I have found to be the most likely causes of slowness on your network.

  1. DNS is not working correctly
  2. Specific service and not the network is running slowly
  3. Bad Cable, Cable Connection, or Wireless signal strength
  4. Bandwidth bottleneck on the network
  5. Consumer grade equipment between the client and the enterprise network

The Details

DNS is not working correctly

I put this first because it I have run into it countless times and it is easy to troubleshoot.  What is happening in this scenario is that a clients computer is asking a DNS server for the IP address that is linked to a specific DNS name (like www.computernetworkbasics.com).  If that DNS server is offline the client computer waits a timeout period and then asks the next DNS server in its list. The second DNS server responds with an IP address and the client computer moves along and talks to the server it wanted to.  To a user on a client computer having this problem the network appears to be running slowly.

The trained eye will notice this issue right away because when you try do something on the network, like visit a website, the computer will wait, wait, wait, then load the webpage quickly.  So, first a delay, then fast loading.  To check to see if this is the issue I like to do two different things.

First, I try visiting a webpage using its IP address rather then its DNS name, then visit it using the DNS name and see if there is a difference in load times.  This doesn’t always work though, because some pages load ads or redirect you to the DNS name, re-creating the problem even if you navigated directly to the sites IP.  It is still a worthwhile test to try.  Especially if you are trying it to a web server that you administer inside your network.

Second, you can use a DNS lookup tool like nslookup to see if a specific server is responding and how quickly it is responding.  As of this writing nslookup works on all major platforms including Linux, OSX, and Windows.  Here are a couple examples on how to use nslookup.

Just putting a domain name in after nslookup will tell you if your primary DNS server is responding to you.

[bash]nslookup www.computernetworkbasics.com[/bash]

You can also specify a DNS server to try if you want to test the secondary DNS server or a third party DNS server.  Here we ask googles DNS server (8.8.8.8) to look up the domain name.

[bash]nslookup www.computernetworkbasics.com 8.8.8.8[/bash]

Try there commands out now so you can get an idea of what a normal response looks like.  Then, when a slowness call comes in do a quick check of DNS from your computer as well as the computer that is having the slowness issue.

Specific service and not the network is running slowly

This is a typical issue too.  I have gotten many slowness complaints about the network when it was actually a slowness issue with the service on the other side of the network that they were trying to use.  The way to troubleshoot and see if this is the issue is to use an alternate instance of the same type of service.  If browsing a website is slow, browse a different website.  If streaming video from Youtube is slow, stream video from Vimeo to see if it is slow as well.  If email slow on your email servers, try gmail or another email service and see if it is slow as well.

If all instances are slow you need to keep looking at the network.  If one is good and the other is not then is it probably not the network and you should look at software on the local computer, check the status pages for the service that is running slow (maybe they are doing maintenance), or look at any filters that you may have in place like a web-filter or packet shaping device.

Bad Cable, Cable Connection, or Wireless signal strength

Yes, bad cabling or poor wireless signal can cause slowness for computers on your network.

To test wireless signal strength you will have to go on site and run a signal strength tool like netstumbler and istumbler or run a RF tool like Wi-Spy.  Strength tools will tell you signal strength in the area you are checking out and a RF tool will tell you if there any forms of interference in the area like microwave ovens, blue-tooth devices, or other wifi access points.

For wired issues you should be able to check the port statistics from your switch for the port that the problematic client is on and see if there are any errors, or if the interface is constantly turning on then off (called link-flap).  You can also use a tool like a link-runner to test the cabling.  Fortunately, for cabling problems, most of the issues happen with the patch cable or at the connectors on either end of the structured cabling (i.e. the cables in the walls).  So, first, just try swapping out the patch cable.  However, if you are using a tool like the link-runner noted above, it will tell you exactly where the cable problem is if there is one.  They can correctly report where problems with cabling are by reporting how far away the issue with the cable is from the link-runner (i.e. there is a break in the cable 30 feet down the cable).  Pretty cool stuff!

Bandwidth bottleneck on the network

It may very well be that you aren’t giving your users enough bandwidth or a specific user is hogging it all at the moment, causing issues for all the other users on the network.  To figure this out you will have to look at the usage statics on your switches and routers.  Particularly paying attention to the bandwidth usage on the links between all the switches and routers.  In this scenario, it can be a huge pain to troubleshoot and determine the cause, location, etc. of the problem.  However, if you have planned ahead a little bit and have been monitoring usage statics on your devices troubleshooting this type of issue is really simple.  It is so simple, I suggest that if you are currently having an issue you think is bandwidth related, I would spend time setting up a monitoring system, then use it to see if there is a bandwidth problem.  Then, you will have it in place for in the future as well.

There are a number of bandwidth monitoring tools available (free and retail).  They all serve the same basic purpose and everyone end up having their favorite.  I started early in the game and have settled on Cacti as my preferred utilization monitoring system so I recommend it.  It is not that hard to set up, will run on multiple OS’s, it can monitor anything with an SNMP OID, it supports SNMP MIB’s, it is open source, and it is free for anyone to use.  I suggest setting up bandwidth and error monitoring on ALL ports on your network.  I’ve done this with cacti installs on retired desktop computers and monitors over 10,000 network ports with no issues.

To identify a bandwidth issue with a tool like cacti you simple look at the little charts that it produces and see if utilization is consistently near 85% or more of a links total bandwidth.  Often, the chart will have a plateau on it if there is a bandwidth issue.  It would normally have either a bell curve or lots of little up and down spikes to it if there is enough bandwidth available.

To address a bandwidth issue you can either throw more bandwidth at it, or find the heavy users and restrict their usage.  If you are monitoring all network ports finding the heavy users is as simple as looking at all the charts and seeing which clients are using lots of bandwidth.  If it isn’t just a couple clients causing the bandwidth problems, then it is probably time to add more bandwidth to your bottleneck.  You can do this by upgrading the equipment on both ends to handle faster connections, or you can install additional cable runs between the two devices and use a bonding technique like ether-channel.  This basically creates a single virtual link between two devices by combining the multiple physical ports on your switches.  THIS REQUIRES CONFIGURATION of the switches before the multiple physical links are connected.  If you don’t configure your switches for this feature and you plug multiple connections in, you will run the risk of creating a “loop” and taking down large portions on your network.  Be careful here!

Consumer grade equipment between the client and the enterprise network

Another very common issue I run into are consumer grade network devices (like your Best Buy linksys or netgear blue light specials) in random offices.  They are unmanaged, don’t run at “wire speed”, and fail frequently.  Inevitably, when I first go into a company, 75% of the problems they are having are a direct cause of the use of this type of equipment.  I suggest proactively eliminating these devices when ever possible.  Everyone involved (you, your clients, your boss) will be happier.

It can be a hard sell though to management.  They see the price tag compared to the managed switches/routers and argue that they “work” so why can’t we use them?  Don’t fall into this trap.  You are asking for trouble, and will get it, if you use these types of devices in any network of size.

Conclusions

There are, of course, many more causes of network slowness that you may run into in the trenches.  The 5 issues described above though, have been at the heart of the vast majority of network slowness problems that I have come across.  So, start troubleshooting slowness issues with these 5 points in mind and you will save yourself lots of time and headache.  Better yet, consider proactively looking for these issues when there aren’t any client asking for help and you will be a rockstar in the eyes of your clients.

Do you feel there should be something else in this list of network slowness causes?  What have you run into?  Share and help the world out!

Automatically Turn off Wireless in OSX Including Mountain Lion (and Mavericks!) (and Yosemite)

NOTICE:

This post has been migrated to my Mac Admin Blog www.grivet-tools.com.

I’ve been focusing on that site and though: 1) the content made more sense over there and 2) I would maintain the content better on that site then I have here.

As an added bonus, I’ve built a pkg installer that you can get by signing up for the trivet-tools email list. Don’t worry, we don’t spam. I can’t stand the junk either! So to get the pkg just head over to the article and follow the email list subscription link at the bottom of the post.

And now, back to the original article …

systempreferences-networkThe Problem:

On the Network that I’m in charge of there are hundreds of OSX desktop computers running a range of OS’s including Leopard, Snow Leopard, Lion, and Mountain Lion.  Often I find the wireless card on these computers turned on even though they were connected to the wired network.  People just thought the wireless card had to be on for the computer to work.  But, turning the cards on often caused odd connectivity issues for people as well as kept using up a bunch of the wireless IP addresses by requesting a lease from our DHCP servers and then turning around an not using it.  Sometimes this would use up all the wireless IP addresses allocated at a site and would prevent people from being able to connect to the network!

The Solution:

To address this I wrote a script that turns off the wireless card on one of our computers if it is connected to our wired network.  If it is disconnected from our wired network, it automatically turns on the wireless card, providing a network connection failover for these computers.  As the article title suggests, I’ve updated the script to work on Mountain Lion as well as Leopard, Snow Leopard, and Lion.

The Summary:

Basically, putting two files onto your computer is all that needs to be done.  Slap the two files onto your computer, tweak each one for your environment, and make one of them executable.  The first file is a script that, when run, will check to see if your computer is connected to the wired network, then turn on or off your wireless network card appropriately.  The second file is a “plist”, or configuration file, that is named a special way and put in a special location.  Once in place, any time any network interface changes its state (i.e. is turned on or off) this plist tells the computer to run the first file we put in place.  Here is a check list of the steps you will have to take:

  1. Configure the script file
  2. Put the script file into an appropriate location on your computer
  3. Make the script file executable
  4. Configure the “plist” file to point to your script file
  5. Put the “plist” file into the proper location on your computer
  6. Restart your computer

The Details:

Well, lets get to it.  Here are each of the steps in greater detail.  Note that you will have to open your terminal application to issue some of the commands that are found in these steps.

Configure the script file

I put the following script together.

Basically, this script pulls your systems OSX version and wired card IP info.  If the IP address is present then it issues the correct command for your OSX version to turn off your wireless card.  If the IP is not in the range you specified, or is nonexistent, then it turns on your wireless card.

To get the above script to work in your environment all you need to do is copy the above code into a text file on your computer, then make the file executable. We go over these steps below.

Put the script file into an appropriate location on your computer

I put the script file into a sub folder in a community scripts folder on my Mac’s.  This structure works well, because as you add more scripts and automate more tasks having a “normal” location that houses all your scripts is really useful.  The folder I’ve ended up using is /Library/Scripts/NetBasics/.  You can make sure this folder exists by issuing the following command: (note: sudo make you run as root and will prompt you for a password before it will run.  This is the password for your current account as long as it is allowed to administer the computer)

If you saved the script code above to a text file on your desktop named “wireless.sh” and you make the directory using the command just mentioned, then you can move the file into the folder by issuing the command:

Make the script file executable

Change file names as needed, but if you are using the filename wireless.sh in the “NetBasics” folder then you can make the script executable by issuing the following command:

Configure the “plist” file to point to your script file

Copy and paste the following code into an empty text file on your computer.  Name that file com.computernetworkbasics.wifionoff.plist.  Then edit the file if needed.  The “path” given in the configuration file below is consistent with the rest of this article.  If you changed the path to the script to something else then edit the path to that script file in the plist as well.

This plist basically links a trigger (the up/down state change for any of your network cards) to a task (run the other script that put in place).  So, any time your wired or wireless network card turns on/off or connects/disconnects the script that we put in place will run.

Put the “plist” file into the proper location on your computer

Make sure the folder that we are putting the file into exists:

Now put the plist into the following special location. The command assumes you worked on the file on your Desktop.

By putting the plist in this folder you are telling the OS to load the plist at boot and link the network card state change trigger to the script that we wrote.

Also, the plist needs to be owned by the user “root” and in the group “wheel” before it will work. (Thanks to Edward in the comments below for pointing that out!)  To change ownership of the plist issue the following command.

Restart your computer

Done!  Now if the script is configured correctly for your environment and everything is in the right place then if you plug your wired network card in your wireless card will automatically turn off.  If you try turning it back on it will just repeatedly turn itself back off.  Also, if you pull the wire out of the wired network card your wireless card will automatically turn itself back on.

This has been a very helpful script in the Mac environments that I work in!  How do you plan on using the script?

Awesome Clipboard Manager Clipmenu

I’ve been using a clipboard manager called clipmenu for some time now and love it.  I found myself annoyed on systems that don’t have it.  Basically, a clipboard manager just keeps a history of the last 30 or so things you have copied.

Give clipmenu a shot.  It’s free!  http://www.clipmenu.com

So if you use “command-c” or use right-click or the menu to copy something in OSX then an entry will go into the clipmenu as well.  You can then use the keyboard shortcut “command-shift-v” to pull up the clipmenu history and paste anything that is in the history.  I’ve started to just copy things, one after the other, (copy, copy, copy).  Then I can paste the things into the command line or another document, etc. in the order that I want to.

Want to take notes from an article you are reading?  Just copy it and keep reading.  Copy another quote later if you want, your previous copy will not be overwritten.  Now, when you are all done with the article you can paste, paste, paste, and have all your notes.  Maybe into Evernote so you can find them easily later on.

Repetitive task become a lot less repetitive!  Especially if you are doing 2 tasks over and over again and have to alternate the tasks.  Basic copy/paste breaks down in that situation.  Clipmenu has made life much easier for me now that I can copy the two things once, and alternate between pasting them.

Useful OSX Command Line Tools

terminal-ifconfigAfter working in computer networking for so many years I have grown to prefer the command line.  Ideally, a Unix/Linux/OSX command line.  I find myself ending up in the terminal to solve just about any hurdle I come across now.  Here are a few commands I commonly use in my daily life.  Currently I use OSX as my primary OS, so these are all OSX commands, but more then likely will run on any Linux/Unix system.

grep

If there is one tool you should learn for the command line it is grep.  Grep will search through files and look for a string that you give it.  You can also “pipe” output of one script or command into grep and have it filter the results for you.  Some examples of grep may be:

“Look for the word ‘bob’ in all the documents in my Documents folder.”

“Look for a filename that includes ‘pdf’ on my Desktop.” Here I use “ls” to list the files and folders on the desktop and “pipe” it through grep to filter the results.

ifconfig

If you need to get IP information quickly ifconfig will be your friend on OSX.  ifconfig will give you interface information for all of your interfaces.  Nice for finding out IP address, MAC address, etc.

ipconfig

ipconfig will give you some of the same information as ifconfig, but it will fill in some of the gaps as well.  It is really useful in finding out info that your computer was given by DHCP.  Like DNS server, IP gateway, IP address, subnet mask, etc.  For ipconfig you do have to specify an option and usually an interface.  The following command I use most often.  “en0″ in OSX refers to the wired network card.  Replace it with “en1″ if you want the same info for the wireless card.

scp

Scp stands for “secure copy”.  It can pretty much be used in all the same ways as “cp” (the copy command) but as the added ability to copy file between systems if the systems support ssh.  I use scp all the time to copy files to and from my laptop when I’m working on a server or another system.  Best of all, ssh only needs to be enabled on my laptop if I issue the command on the other system.  To copy a local file to a remote computer you would do the command below.  This would copy the file name “file.doc” from your Documents folder locally and put it in the “username” users Documents folder on the computer located at IP 10.10.2.20.

tail

tail will dump the last few lines of a text file to your screen.  So if you want to see the end of a file for some reason tail is great.  I use tail on log files all the time.  It has an option “-f” that follows the end of a file.  So you run tail with the -f option and, for a log file as an example, as log entries are put into the file they will also show up on your screen.  Very helpful for troubleshooting.

Additional info

The commands I listed above are just a couple commands that I use all the time.  They are more advanced then basic “moving around” commands like ls, cd, rm, and pwd; but are less advanced then some other commands that I use all the time like sed, awk, vim, and nc.  I’ll cover some of those other commands in future posts.

The Difference Between TCP and UDP

TCP and UDP are protocols that run on top of the IP protocol.  An application like a browser, VoIP phone, or VPN client will use one of these protocols to communicate with a peer computer or server.  Lets take a quick look at each of these protocols, their benefits, and their flaws.

TCP

TCP behaves like a phone call.  It follows a “handshake” process where a conversation is set up.  TCP mimics a phone call in this way.  When you call someone, someone will answer the phone usually with a greeting.  You the caller then give a greeting back then have a conversation.  When you are done, you both say bye and then hang up.  TCP behaves the same way.  The benefit to behaving this way is that the calling party know that the receiving party got the message.  The problem with TCP is all that talking back and forth takes up time (Bandwidth in the network world).

TCP is kind of like a phone call.  There is a greeting, conversation, goodbye, and the caller knows the message was received.

UDP

UDP behaves like texting rather then a phone call.  You send a message out to a friend not knowing if or when they will get it.  You basically fire off the message and hope that it gets their.  No greeting or goodbye, just the message, going in one direction.  UDP doesn’t make any attempt at ensuring that a message makes it to its destination.  The benefit to UDP is that sending the message is less complicated and quick.  The problem with UDP it that you have no way of knowing if the message made it or not.

UDP works like a Text message.  It is sent off and just assumed that it got where it was going.

Couple Quick Examples of TCP and UDP

TCP Example: Web Traffic – HTTP

Web traffic uses a protocol called HTTP.  HTTP uses TCP because when a computers web-browser asks for a webpage from a web server the server wants to know that the browser actually received all the packets from the server.  The web server wants to ensure that all the content from a web page has been received by the browser.  If some of the packets from the web server were dropped by the network the web server would need to send the packets again or the web page would not be readable by the web browser.  Webpages, especially ones with lots of images, can be relatively large in size.  So having a little extra overhead during the webpage loading “conversation” doesn’t really add much time to the overall conversation but does add a lot of value.

UDP Example: Voice Phone Call – VoIP

Many VoIP (Voice over IP) systems use UDP for communication.  They use UDP because voice packets are notoriously sensitive to slowdowns on a network.  VoIP packets also don’t need to ensure that all packets get to their destination.  Each VoIP packet is just a little bit of audio from one phone to another, so if a tiny bit of audio goes missing during a conversation it doesn’t really matter.  We also wouldn’t want to try to have a phone re-send that audio.  This would either delay a conversation for no good reason or play a little bit of audio out of order which would be more confusing then just dropping the bit of audio and moving on with the conversation.

Questions?

I hope this brief and VERY simplistic view of TCP and UDP was helpful.  Do you have any questions about networking?  Let me know!

Getting to Know your Neighbor

A Puzzle Without a Picture

Rubiks CubeEver try to put a puzzle together without a picture to go by?  Ever try to find your way around a new city without referencing a map?  Having a simple guide or reference helps tremendously when trying to get around.  It is critical when trying to solve a problem, especially when the pressure is on to get it solved.  When troubleshooting, if you have no reference point, you have to create one before you can move forward.  So, in the networking world, where can you get a map?  What do you use as a reference picture when you set about solving a networking puzzle?

When troubleshooting, if you have no reference point, you have to create one before you can move forward.

Going Door to Door to Create a Map

If you don’t have documentation or your documentation is out of date, troubleshooting a network problem can be a challenge.  We all know this, but we get busy and documentation can easily fall by the wayside, or we inherit an undocumented network and don’t know where to begin.  So, what now?  How can I easily create a physical map of my network?  You can use a feature called LLDP or CDP to figure out the physical layout of your network.  If you are using managed switches and routers they probably have a feature on them called CDP or LLDP.  CDP (aka, Cisco Discovery Protocol) and LLDP (Link Layer Discovery Protocol) are a simple and very useful feature.  They difference between the is that CDP is proprietary to Cisco while LLDP is an open standard.  Both are basically a simple way to figure out what network devices are plugged into each other.

You can use a feature called LLDP or CDP to figure out the physical layout of your network.

How LLDP and CDP Work

Managed network devices will send LLDP or CDP packets out roughly once a minute on all network ports.  These packets include the following key information.

    1. Name – The name of the device sending the packet
    2. IP – The IP of the device sending the packet
    3. Port – The port info of the sending device that the packet was sent on
    4. VLAN – The native VLAN for the port that the packets was sent on

The other network devices that are directly connected to the device that sent the LLDP/CDP packets will note what port it received the packet on and the information in the packet.  These packets are not forwarded on to any other device.  The result is that every device that supports LLDP/CDP creates a list of all the network devices it is directly connected to and on what port it is connected on.

Every device that supports LLDP/CDP creates a list of all the network devices it is directly connected to.

The Meaning of it All

When I first learned about this feature I thought, “That’s nice, but I doubt I’ll use it that much.”.  I was totally wrong.  This is a feature that you can use all the time.  It helps identify loops in your network (you see the same neighbor switch twice, or you see the switch you are on as a neighbor to itself), it helps locate where users connect into the network (using a device that understands LLDP/CDP like a Fluke LinkRunner), it helps you see if a computer/printer/etc. is in the right VLAN for the IP that it is using, and it helps you trace cabling all over the building from the comfort of your desk.

LLDP and CDP enable you to trace network cables from the comfort of your desk.

So have you ever used LLDP or CDP in your environment?  In what ways has this simple feature helped you?

What is the difference between a router and a switch

The Short Answer:

switch-router

To Diagram, use Circles for Routers, Squares for Switches

So, what is the difference between a router and a switch? A router deals with IP addresses.  Using IP addresses to decide what to do with the traffic it is seeing.

A switch deals with MAC (Media Access Control, aka hardware/physical) addresses.  Using MAC addresses to decide what interface to send traffic out.

The Extended version:

Lets talk about IP and MAC addresses for a minute to better understand “the short answer” above.

IP Addresses:

IP addresses are displayed as 4 numbers between 0 and 255 separated by periods.  An example could be “8.8.8.8” or “10.32.24.1”.  These numbers have a specific structure to them that is used by routers.  Every system on the internet has a “unique” IP.  There is a network mask that goes along with the IP that tells routers how to break the number down into pieces.  IP addresses are normally assigned to you by your ISP.  There are, however, some IP network ranges that anyone can use.  The problem with these numbers is that they can only be used within your own network.  These networks will not be routed by internet service provides so if you use these IP’s on your network, you will need to use something called NAT to enable the computers on your network to access the internet.

Please note that these addresses are part of the IPv4 specification.  IPv4 has been around for a while and is what most of us are using right now.  IPv6 is a new kid on the block.  It isn’t as widely used (yet) so we will forgo discussing it for now.  You can check out my IPv6 blog posts to learn more about IPv6.

IP addresses are like your street address. They relate to where you are and change when you move.

MAC Addresses:

The MAC address is a number that network card manufactures assign to every card they produce (wired, wireless, etc.).  Switches listen for this address when a computer talks on the network.  It will take note of which port that MAC address was seen on and any traffic going to that MAC address will be sent out just that one port.  This keeping track of where every computer is sitting on the network helps the network send traffic only to the ports that need the traffic, making the network more efficient.

MAC addresses are like your name. Once you have been given one it doesn’t change, whether you live in London or Sydney.

So why both IP and MAC?

The network uses both because it has a need to manage traffic quickly, minimize how much information it has to know to work, and a need to talk to any given computer on the planet on a whim.  The MAC address is really only used locally on a network.  You computer knows and/or can figure out the MAC addresses of nearby computers and network equipment (like routers).  It doesn’t need to know the MAC of all the computers on the internet, just the local ones.

To talk to computers that are on the internet your computer uses an IP address.  Normally, it will use DNS to convert a name like www.computernetworkbasics.com into an IP address.  Then it uses that IP address as the destination for packets it is sending to the website.

MAC addresses are nearsighted and IP addresses are Farsighted.

The traffic flows like this:

Relay Race

Relay Race: People are MAC addresses, the Baton is the IP.

Your computer figures out that the IP address is on another network so it sends a packet containing the destination IP address but (here is the tricky part) with the local routers MAC address.  This is “switched” over to the router, who looks to see if it knows what to do with that packet based on its IP address.  Once it makes a decision it will send the packet to the next router using its own MAC and the next routers MAC but keeping the original source and destination IP addresses.  This process creates magic and, wha-la, you are able to talk to this website and pull this webpage down to view it.

MAC addresses travel like a relay race while IP addresses travel like a marathon runner.

What other aspects of computer networking do you have questions about?  Drop me a line or make a comment below.

The Network Admin That Can Read Minds

Like all magic tricks, this one is simple, it is all about preparation, it is impressive to the uninitiated, and surpassingly boring to the ones in the know.  Want to know how to read your clients minds?  All you have to do is read your network and server log files!

This will impress people.  They will come walking into your office and say, “I’ve been having a problem opening this file on the server lately…” and before they even finish you will say “I noticed that a little bit ago and I think I’ve got it fixed for you…”.  Talk about an impressed client!

Seriously though, when was the last time that you checked you log files?  I know, I hate doing it too, but it is really a necessary task.  It can make you look like a mind reader and make your life a lot easier.  Something I never like hearing from clients is “It hasn’t been working for a month now but I really need it to work right away!”.  First off, if it wasn’t working a month ago maybe we could have reported the issue a month ago and it would be working by now. Second, now we are in firefighter mode running around in a panic to solve a “sudden” issue.  Logs are a immunization against this situation.

Most people think of log files as a post-mortum kind of thing.  That they are good for going back and figuring out what happen when something goes horribly wrong.  They do help with in that situation, but they are far better when used in a proactive way.

When checked regularly, log files:

  1. Cease to be so confusing.  You get familiar with the layout, where they are, what logs to what file/location, etc.
  2. You get a feel for when something is up.  There are more logs, there are lots of different logs, there are log entries you have never seen before, etc.
  3. They will reveal problems long before clients will notice them and/or report an issue.  It is so much nicer when a log files tells you about an issue rather then your company CEO!

Okay, so you now know you need to be checking log files proactively.  So you now need to start spending hours every day looking at them right!?  Of course not, even if you did it once a week you will start finding issues before clients do and both of you will be happier.

Here is what I recommend to get you started with the habit of checking log files.

  1. Pick a convenient time, where you will be able to look at the log files for 30 minutes. (Monday mornings, Friday Afternoons, whatever).  You can start with once a week.
  2. Schedule it in your calendar and have your computer/smart phone/whatever you use remind you to check the logs.
  3. Pick a system you haven’t looked at recently and start checking its log files.
  4. If you find a log entry:
    • Indicating a problem you can solve in 3 minutes or less then solve it right then.
    • Indicating a problem you know how to solve but it will take more then 3 minutes, then make a note and address the problem after your 30 minute checking window.
    • You have never seen before, start googling it.  It is okay to use the allotted time to help you research and learn the meaning of log files you are looking at.

That’s it!  This simple weekly check will start moving you away from running around putting out fires and will enable you to prevent fires in the first place.

In addition to checking log files weekly, I’ve implemented some more fun and interesting ways to monitor log files all the time.  Making the log files more visual and representing them in real time enables you to get a feel for the baseline for your log files, then it is really easy to know when something is up, even if you don’t know what exactly just yet.  One of my favorite log visualization tools is gltail.  It is a bit of a bear to set up though and is more useful if you have a syslog server in place.  These systems will all be topics of future posts, but for now, just start checking the log files!