Daniel Green's Homepage

About Me

My name is Daniel Green.  I am a programmer from the western USA.  In 2008, I created camelcamelcamel.com, the Internet's most camel-themed Amazon price tracker, and here in the far-distant future I am very gratefully still running the site.... continue reading

 

The dangers of restoring RDS instances from snapshots

When restoring an RDS instance from a snapshot, AWS unavoidably lazy loads the entire database on-demand. This leads to a huge performance penalty until the entire database has been accessed at least once. This is fine if your database is... continue reading

AWS RDS MySQL  

Avoiding "Storage-optimization" state when restoring RDS instance from snapshot

This seems somewhat obvious in hindsight, but the always lovely AWS support taught me that the "Storage-optimization" state occurs – during restore, anyway – when the snapshot's IO provisioning doesn't match that of the newly restored instance.  That is, the IO configuration... continue reading

AWS RDS  

AWS RDS MySQL restore stalling

When working with RDS, I would occasionally find my web console-initiated restores-from-s3 stall out indefinitely with no errors of any kind.  Confused, I ran to AWS Support, who told me that I should not specify "root" as the admin user... continue reading

AWS RDS MySQL  

Restoring a MySQL backup to AWS RDS without the automated post-restore snapshot

While getting our CDK-based AWS infrastructure going, I often found myself restoring a MySQL backup to RDS.  This went fairly smoothly, except that I could not avoid the end-of-restore snapshot that the AWS web console requires.  This adds several hours... continue reading

AWS MySQL RDS  

Speeding up and compressing MySQL backups using xtrabackup

I love Percona and its MySQL tools.  They have really enabled me to pretend I'm a DBA for much longer than I probably should have. One of the main Percona tools we use is called "xtrabackup", an online / hot backup... continue reading

MySQL Percona xtrabackup  

Optimizing EC2-S3 transfer throughput / bandwidth utilization

The base "aws s3" commands like "sync" seem to work sequentially, uploading one file at a time.  S3 also seems to have a per-transfer bandwidth limit, which significantly restricts the performance of 10+ gbit instances, and increases the amount of... continue reading

AWS EC2 S3