Typical Nerd Network?

2016 November 20
by Daniel Lakeland

So, I'm not sure how typical this is for people who actually use computers for work and such, but this is more or less how my home network looks. With my main workstation taking files off the home server via NFS, it seemed like a good idea to split the server and printer off from the Buffalo WiFi "router" (no longer routing, just a fancy AP) and the ATA so that all the calls will be going over a separate cable from any filesystem operations.

Once gigabit fiber WAN hit, I needed to move the routing load from the little Buffalo router to the home server, which is when I added a "Smart Managed" switch. Doing that also lets me prioritize voice packets via DSCP on the local LAN so if I'm doing something like talking on the phone while browsing PDF scans of documents, the traffic to my computer reading images off the server doesn't interfere with my calls.

The home server runs on an Asrock Rack J1900D2Y which is a mini-itx motherboard with a dedicated IPMI port (shown in red). The switch also lets me prioritize IPMI traffic (slightly lower than the voice traffic but higher than default traffic) so that if something goes wrong and I need to reboot the server I can connect without lag. The server is headless and has no keyboard.

I discovered that the IPMI port on these (and also on Supermicro mobos) can failover to be shared with the first ethernet port on the motherboard. This causes no end of trouble when the IPMI traffic starts appearing on one of the links in the bonded LAG group. It seemed like the solution would be to force the IPMI to only use the dedicated port, but I haven't yet figured out how to make that actually happen (altering it in the IPMI network settings didn't seem to work, I think I need to get into the BIOS but that requires a reboot of the server while using IPMI and then I have been losing my IPMI connection and can't see the BIOS screen because it tries to failover during boot). A reasonable solution was to statically allocate the IPMI MAC address to the port where the dedicated IPMI link goes into the managed switch. At least then the switch doesn't start getting confused about the LAG group, and the IPMI port eventually figures out that it can't talk to anyone on its failover port so it sticks to Dedicated.

Yellow links show battery-backed UPS power, so I can still have internet access and phone calls for several tens of minutes to an hour after power goes out. That duration was much longer when the whole thing was routed by the Buffalo device, but it's still a reasonable amount of time since the little Mini-ITX server, switch, and RAID enclosure all use about 8% of the max UPS output, or something like 75 watts.

I'm pretty sure this system could handle a full office for a medium business of around 100 people with just the addition of a couple of POE switches and some desk phones (and a whole lot more floor space!). For a system like that I'd probably move the file server function to a cluster of two servers running glusterfs to handle hardware failures and planned downtime more smoothly.

As it is right now the whole thing works pretty smoothly and gives me a place to store large files and archive lots of photos without having my important files connected directly to my desktop machine where I run more bleeding edge kernels and occasionally run large computations that have a chance of accidentally filling all the RAM and bringing the machine to its knees (there was some particularly nasty issue trying to plot some of my recent graphs. I think ggplot was making a full copy of a large data frame once for each plot in a spaghetti plot, leading to 65GB RAM usage... moving to data.tables helped a lot, and also re-thinking how the plot was constructed so that it was using just one copy of the table).

How much effort goes into your home network? Considering how many WiFi networks I see where no-one even bothered to set the ESSID to something other than ATTWiFi-992 or whatever, I'm guessing it varies a lot. One thing that seems clear though is that lots of people are frustrated with their home networks as they load on more and more devices. Even a non-tech-savvy family of 2 adults and 2 teens probably has a minimum of 10-15 WiFi devices these days given smartphones, tablets, laptops, a security camera or baby monitor, a game console, Roku/FireTV/Chromecast etc.

The fact that this technology works as well as it does is testament to how inefficiently our radio spectrum is being used. Think of the whole FM Radio, TV, and spectrum in use for business, police, fire, and soforth. Something like 50MHz to 1GHz is in my opinion utterly wasted compared to what could be done with modern techniques like Frequency Hopping Spread Spectrum or dynamic frequency allocations via negotiation protocols (compare to the way DHCP works for IP addresses).


No comments yet

Leave a Reply

Note: You can use basic XHTML in your comments. Your email address will never be published.

Subscribe to this comment feed via RSS