3D programming

Interactive Fluffy Ball

This is a fun little interactive app that showcases the power of shaders in modern browsers. This looks like one of those Three.js demos, but it is actually written in a cross-platform language called Haxe that is able to compile to different platforms including JavaScript, C++, C#, Java, JVM, Python etc.

Source: Marimo

cloud programming sysadmin

How I reduced a WordPress database size by 85% and memory consumption by 20x

I was helping a friend to troubleshoot their e-commerce site. It was running on WordPress using WooCommerce as the e-commerce backend. Like most WordPress sites, it was installed with a ton of plugins. My friend complained that the site performance has been getting slower and slower, to the point where a page load can take anywhere from 2-3 seconds to a failing to load at all. Getting to wp-admin also took forever.

At first, there are a lot of pieces to unravel, since the cause might be anything. The backend was running on AWS. The WordPress site is running as a docker container on the EC2, while the database is running on a RDS instance. It uses Cloudflare tunnel to connect the public hostname to the docker container. Seems like a decent setup.

While I do use WordPress (this site runs on WordPress), I am not a WordPress developer so I was not familiar with where things might go wrong. My first intuition was to check the plugins, since not all WordPress plugins are well written and some are notorious for taking up a lot of resources. Unfortunately isolating plugin resource usage by instrumentation was not possible as far as I know, due to the way WordPress/PHP works. After comparing the set of plugins with another site which did not exhibit the same behaviour, I decided to try other approaches.

I tried the usual tricks, like enabling proxying in Cloudflare, using a caching plugin, upping the EC2 instance size and RDS instance size. I even added a robot.txt to prevent bots from crawling the site for the time being. Those tricks helped a little, but did not resolve the problem.

Using docker stats, I noticed that CPU and memory usage is extremely high for the container, compared to others. CPU consumption is often >100% with every page load and memory usage spiked to 14GB after a while. Another unusual sign is the size of the database. For a site with around 500 products, the database size is >600MB.

That is when I chanced upon this article when searching for the symptoms.

The problem WordPress sites can run into is when there is a large amount of autoloaded data in the wp_options table.

If you return anything below 1 MB you shouldn’t be worried. However, if the result was much larger, continue on with this tutorial.

I ran the query in the article and it returned the following.

Wait. The autoload_size is ~570MB (!). I wrote a SQL command to find all the options which are larger than 1MB.

The results range from 1MB all the way to 13MB.

For the uninitiated, wp_options is akin to Windows registry, and it has become a dumping ground for plugins to store values that they might need. Most of the values in this option should be configuration values (like siteurl) which should take up just a few bytes. wp_option also has a field “autoload” which states whether the option should be loaded on every page. Storing 13MB in an option value and setting it to autoload is just insane. The total size of autoload options in the table turns out to be >500MB. Every page load is querying >500MB of data from the database and processing those data. No wonder the site is crawling!

Inspecting those options shows them most of them have the prefix _transient, which means they can be safely deleted. After making a backup of the database, I deleted all transient options. wp_options went from 556MB to 46MB, a reduction of >90%. The total database size went from 645MB to 84MB, a reduction of >85%. Memory consumption also dropped by 20x (from ~14GB to ~700MB). More importantly, the site is now super fast which is extremely important for an e-commerce site.

The results are very telling from the RDS dashboard.

Average CPU utilization has dropped to <3% and average database connections is now near zero.

Aside from noticeable performance boost for the site – average page loads within 1s – another bonus from these optimizations is that we can now use smaller EC2 and RDS instance types for better cost savings. Hopefully this article is useful as a reference for others in similar situations.

cloud programming

Web Push for Web Apps on iOS and iPadOS | WebKit

This is good news as it further expands the capabilities of web apps. This addresses a longstanding request for web apps to deliver notifications. Note that web push only works if the web app is added to Home Screen. It is to limit web apps that aggressively ask for too many permissions.

With iOS and iPadOS 16.4 beta 1 comes support for Web Push for Home Screen web apps, Badging API, Manifest ID, and more.

Source: Web Push for Web Apps on iOS and iPadOS | WebKit

programming security

NIST Retires SHA-1 Cryptographic Algorithm | NIST

NIST, the US standards body in charge of cybersecurity, is recommending phasing out the use of SHA-1 due to vulnerabilities in the algorithm and possibility of attacks by powerful machines. Modern browsers have already sunset support for SHA-1. However, older applications may still be using it as a form of checksum.

The venerable cryptographic hash function has vulnerabilities that make its further use inadvisable.

Source: NIST Retires SHA-1 Cryptographic Algorithm | NIST


Image which displays its own MD5 hash

This is very impressive. Someone managed to create an image whose MD5 hash (also known as MD5 sum) is in the image. Why is it impressive? Well, for one, hashes are computed based on the contents of the input, and even making a one bit change creates wildly different hashes. Let me illustrate this with an example:

$ echo -n 1234 | md5sum
81dc9bdb52d04dc20036dbd8313ed055 *-

$ echo -n 1235 | md5sum
9996535e07258a7bbfd8b132435c5962 *-

1234 and 1235 differs by just 1 bit. In binary:

1234 is 00110001 00110010 00110011 00110100
1235 is 00110001 00110010 00110011 00110101

and yet the MD5 hash is totally different. Imagine if you’re trying to create a text document with this content:

The MD5 sum of this is: 12345678901234567890123456789012

Obviously the MD5 hash is not going to be 12345678901234567890123456789012.
It computes to b8aca742ffc52d6bea85fd87a92d3ede.

So you tweak the contents to this:

The MD5 sum of this is: b8aca742ffc52d6bea85fd87a92d3ede

Now the MD5 sum becomes 878da8ba44a3938fdce6da2191f221a1.

The minute you attempt to tweak the contents, the MD5 hash changes.

So it is possible to engineer a text such that the MD5 hash is contained in the text? The answer is probably yes. But you will have to test 2^128 combinations to find it.

Which brings us to the image. Creating an image like this is difficult, due to the encoding and checksums that a valid PNG image needs to have. Yes it’s not the first image-based hash quine (or file that show their own hash), but what makes this even more impressive is the deliberate choice of choosing 1337 to appear at the front and back of the MD5 sum.

Kudos to the author for this achievement.

The image in this post displays its own MD5 hash. You can download and hash it yourself, and it should still match – 1337e2ef42b9bee8de06a4d223a51337 I think this is the first PNG/MD5 hashquine.

Source: Retr0id

3D programming

Data Management OSS (Object Storage Service) migrating to Direct-to-S3 approach

It’s no secret that Autodesk Forge uses AWS. But now they made it explicit. Some of the API will be exposing AWS services – in particular S3 – directly.

Source: Data Management OSS (Object Storage Service) migrating to Direct-to-S3 approach

programming security

BIG sabotage: Famous npm package deletes files to protest Ukraine war

Oh dear. Yet another npm author went rouge. This time it appears that the npm package deletes files for users with Russian/Belarus IP addresses. Time to take package pinning more seriously.

This week, the developer of the popular npm package ‘node-ipc’ released sabotaged versions of the library in protest of the ongoing Russo-Ukrainian War. The ‘node-ipc’ package, which gets downloaded over a million times weekly, began deleting files on developer’s machines, in addition to creating new text files with “peace” messages.

Source: BIG sabotage: Famous npm package deletes files to protest Ukraine war

bug programming security

Dev corrupts NPM libs ‘colors’ and ‘faker’ breaking thousands of apps

Previously we had attackers using hijacked npm libraries to steal credentials. In this case the libraries or the maintainer wasn’t compromised. In fact it was the maintainer who deliberately introduced bugs into his libraries, thereby breaking thousands of apps that depends on it. There’s no easy solution to this dependency problem. For now use pinned versions and manually approve upgrades.

Users of popular open-source libraries ‘colors’ and ‘faker’ were left stunned after they saw their applications, using these libraries, printing gibberish data and breaking. Some surmised if the NPM libraries had been compromised, but it turns out there’s more to the story.

Source: Dev corrupts NPM libs ‘colors’ and ‘faker’ breaking thousands of apps

programming security

RCE 0-day exploit found in log4j, a popular Java logging package | LunaSec

log4j is a common logging library for Java applications. This vulnerability is extremely easy to exploit, and allows the attacker to run arbitrary code in the server. IOW, very bad. For now, set log4j.formatMsgNoLookups=true to mitigate the issue, until an official patch is out.

Given how ubiquitous this library is, the impact of this vulnerability is quite severe. Learn how to patch it, why it’s bad, and more in this post.

Source: RCE 0-day exploit found in log4j, a popular Java logging package | LunaSec

gis programming

OneMap API

Do you know that you can do this? No API key or token is required to do simple geocoding via OpenMap API.

Notice the response returns LONGTITUDE and LONGITUDE containing the same values. This is due a misspelling in the earlier API and a decision not to break the earlier API.

Find out more here: