Category: sysadmin

I knew about Gluster File system, but it’s the first time I heard of Bees with Machine Guns! This article provides an insider’s view on how an online magazine company scale up their back-end to prepare for Kim Kardashian’s backend ;-). If you are a sysadmin or web engineer I bet some parts of the article will make you smile.

How PAPER Magazine’s web engineers scaled Kim Kardashian’s back-end (SFW) — The Message — Medium.

As known as “what to do when a poisoned China DNS is pointing at you”.

furbo.org · Fear China.

Was doing a routine scan when I spotted an unfamiliar address on the network: 192.168.200.1. Strangely arp doesn’t reveal its MAC address, which seems odd given that this is a private IP address used internally.

Traceroute reveals the truth:

> tracert 192.168.200.1

Tracing route to 192.168.200.1 over a maximum of 30 hops

Trace complete.

It seems someone has a misconfigured or misbehaving router that’s exposing private IP addresses. Let’s hope it is not storing something incredibly important.

Great that DigitalOcean now has IPv6 support. Another reason for moving from Amazon. Not to mention that it’s cheaper in most usage.

Announcing IPv6 Support in Singapore | DigitalOcean.

It happened. Or should I say, almost happened.

As we all know, the hard disk (mechanical ones, that is) is the component that has the highest chance of failure in any computer system. One day I was doing a routine backup of my notebook. My backup solution is rather simple, consisting of no more than rsync. I had left it running a full backup in the background before I went out, expecting it to complete before I return, since only differences are copied. To my surprise when I returned, it was still running and my notebook felt very hot. Much hotter than usual, and that says something, as my notebook reaches uncomfortably heaty temperature after long usage. I blame it on the GPU/hard disk. The copying appears to be stuck at 76% on a particular large file. After terminating it and manually copying the file to my backup hard disk, it remained stuck at 76%. First sign that something is wrong. To be sure that it wasn’t my backup hard disk that’s having problem, I made a copy of the file on the same drive. Yup same thing happened. I immediately stopped any attempts to access the file to avoid aggravating the problem. Conventional wisdom in hard disk recovery says that when a hard disk is showing signs of failure, do not access the “bad” parts ‘cos it could cause the problem to get worse.

A hard disk replacement is imminent, which is not a big deal. Except that it could mean reinstalling everything from scratch. Or not. I’m really not looking forward to spending days fighting with a new OS. So cloning the existing hard disk is my plan.

Step 1: research

Before doing anything that could lead to further data loss, it is always good to read up. My concerns were 1) data integrity 2) preservation of Windows license. As the Windows license that came with the notebook is an OEM license, I wasn’t sure if it could survive the cloning process – with a retail Windows license you could activate on up to X times I think. The recommended way to backup a Windows machine is to use Windows System Image Backup. Unfortunately it can’t be used in my case. My second idea was to use dd. However I’m aware that dd could run into trouble with reading bad parts. Finally I decided on ddrescue, as it appears to be addressing what I need from dd, but with more features targetted towards hard disk recovery.

Step 2: execute

I got a larger hard disk as recommended by most articles. I also needed a way to attach the new hard disk to my notebook. Here’s where my trusty SATA to USB adapter comes in handy. For the benefit of others who may want to do the same, the steps are:

• download Knoppix Linux ISO
• burn to CD, or if you’re lazy like me, create a bootable USB thumbdrive with it using Rufus
• boot up to Knoppix
• select shell
• lsusb to see what USB devices are attached
• insert SATA to USB adapter
• lsusb to see what’s added
• dmesg to see the newly added device. note the new device name
• (assuming old hard disk is /dev/sda and new hard disk is /dev/sdc) take a deep breath and type:
  ddrescue -f -n /dev/sda /dev/sdc /root/rescue.log
• if there are no errors, hurray! you can stop here. Otherwise, type:
  ddrescue -d -f -r3 /dev/sda /dev/sdc /root/rescue.log

In my case there was 1 x 8192 bytes of error after the first command. After running the second command, it was reduced to 1024 bytes. Ok, it wasn’t as bad as I thought 🙂

Step 3: verify

• Unscrew the hard disk compartment and replace the old hard disk with the new one. Replace cover.
• Boot up.

At this point, if it works it should be pretty obvious. I’m glad to report that everything works as planned. wmic diskdrive shows the new hard disk details. Oh, and Windows didn’t complain. An unexpected good news is after the upgrade, things are speedier and my notebook doesn’t feel as hot as before. Hurray! 😀

Ever wonder what information you can derive from a MAC address (eg. 50-e5-49-12-34-56)? Well, the first 6 letters uniquely identifies the manufacturer of the device. Eg. MAC addresses starting with 50E549 are from Gigabyte. You can download the whole list of manufacturer OUI (ORGANIZATIONALLY UNIQUE IDENTIFIER) from here. Or you can use a lookup service like this.

Here’s a small utility that I’ve written to make it easier to visualize package changes in a Debian-based system.

Sample output:

2013-03-06 (Wed)

   purged> 13:30  ecryptfs-utils ()
           12:37  ecryptfs-utils (68-1+lenny1)
           12:37  keyutils (1.2-9)
           12:37  libecryptfs0 (68-1+lenny1)
           12:37  libgpgme11 (1.1.6-2)
           12:37  libpth20 (2.0.7-12)
   purged> 12:36  mimms ()
uninstall> 12:36  libmms0 (0.4-2)

2013-02-28 (Thu)

           23:36  mimms (3.2.1-1)
           23:36  libmms0 (0.4-2)

2013-01-30 (Wed)

  upgrade> 17:16  samba-common (2:3.2.5-4lenny15)
           17:16  smbfs (2:3.2.5-4lenny15)
  upgrade> 17:16  samba (2:3.2.5-4lenny15)
  upgrade> 17:16  smbclient (2:3.2.5-4lenny15)
...snipped...

Read more here: https://github.com/boviner/pkghist/

While searching for a solution to stop @eaDir from being generated on the Synology NAS, I came across this blog, which has quite a number of good articles that sysadmins will find useful.

Will certainly be keeping this in my bookmarks for reference.