Daniel Green's Homepage

Posts tagged under AWS

DynamoDB Storage Overhead and Billing

Today we learned that DynamoDB has a per-item storage overhead of at least 100 bytes (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/CapacityUnitCalculations.html).  Depending on your item size, this can significantly increase your storage costs.  And, conveniently, AWS... continue reading

AWS DynamoDB Billing Storage Overhead  

RDS Gotcha - Backup Retention

If you have your RDS backup retention period set to a non-zero value, then change it to zero, all of your automatic snapshots get deleted...at least when deploying via CDK. Easy to imagine that causing trouble!... continue reading


RDS Gotcha - Replication

When we migrated from our datacenter rack to AWS, most of our stress revolved around our 2TB MySQL database.  With a database this size, one does not simply mysqldump your way into Mordor.  Our final migration plan required us to... continue reading


The dangers of restoring RDS instances from snapshots

When restoring an RDS instance from a snapshot, AWS unavoidably lazy loads the entire database on-demand. This leads to a huge performance penalty until the entire database has been accessed at least once. This is fine if your database is... continue reading


Avoiding "Storage-optimization" state when restoring RDS instance from snapshot

This seems somewhat obvious in hindsight, but the always lovely AWS support taught me that the "Storage-optimization" state occurs – during restore, anyway – when the snapshot's IO provisioning doesn't match that of the newly restored... continue reading


AWS RDS MySQL restore stalling

When working with RDS, I would occasionally find my web console-initiated restores-from-s3 stall out indefinitely with no errors of any kind.  Confused, I ran to AWS Support, who told me that I should not specify "root" as the... continue reading


Restoring a MySQL backup to AWS RDS without the automated post-restore snapshot

While getting our CDK-based AWS infrastructure going, I often found myself restoring a MySQL backup to RDS.  This went fairly smoothly, except that I could not avoid the end-of-restore snapshot that the AWS web console requires.  This adds several hours... continue reading


Optimizing EC2-S3 transfer throughput / bandwidth utilization

The base "aws s3" commands like "sync" seem to work sequentially, uploading one file at a time.  S3 also seems to have a per-transfer bandwidth limit, which significantly restricts the performance of 10+ gbit instances, and... continue reading