Daniel Green's Homepage

About Me

My name is Daniel Green.  I am a programmer from the western USA.  In 2008, I created camelcamelcamel.com, the Internet's most camel-themed Amazon price tracker, and here in the far-distant future I am very gratefully still running the site.... continue reading


The dangers of restoring RDS instances from snapshots

When restoring an RDS instance from a snapshot, AWS unavoidably lazy loads the entire database on-demand. This leads to a huge performance penalty until the entire database has been accessed at least once. This is fine if your database is... continue reading


Avoiding "Storage-optimization" state when restoring RDS instance from snapshot

This seems somewhat obvious in hindsight, but the always lovely AWS support taught me that the "Storage-optimization" state occurs – during restore, anyway – when the snapshot's IO provisioning doesn't match that of the newly restored instance.  That is, the IO configuration... continue reading


AWS RDS MySQL restore stalling

When working with RDS, I would occasionally find my web console-initiated restores-from-s3 stall out indefinitely with no errors of any kind.  Confused, I ran to AWS Support, who told me that I should not specify "root" as the admin user... continue reading


Restoring a MySQL backup to AWS RDS without the automated post-restore snapshot

While getting our CDK-based AWS infrastructure going, I often found myself restoring a MySQL backup to RDS.  This went fairly smoothly, except that I could not avoid the end-of-restore snapshot that the AWS web console requires.  This adds several hours... continue reading


Speeding up and compressing MySQL backups using xtrabackup

I love Percona and its MySQL tools.  They have really enabled me to pretend I'm a DBA for much longer than I probably should have. One of the main Percona tools we use is called "xtrabackup", an online / hot backup... continue reading

MySQL Percona xtrabackup  

Optimizing EC2-S3 transfer throughput / bandwidth utilization

The base "aws s3" commands like "sync" seem to work sequentially, uploading one file at a time.  S3 also seems to have a per-transfer bandwidth limit, which significantly restricts the performance of 10+ gbit instances, and increases the amount of... continue reading