Slow AWS machine but you're not running out of credit?

Slow AWS machine, but you're not running out of credit?

You might be running a t2 or t3 instance. Those instances are very cheap. They let you spin up a webserver, a database if you want, but they work with a CPU credit system.

CPU credits

If your application is using the CPU of the machine at 100%, they use credits. Those credits replenish whenever your CPU load goes down and is calculated every minute. Depending on the instance size (small, medium, etc.) it comes with a different baseline.

The baseline, for example, for a t2.nano is 5% CPU usage per hour. The baseline, the accrued credits and their expiration is well documented here.

Once you understand the concept of the credit system, it's pretty easy to adapt your needs and your instances accordingly. The monitor tab or cloudwatch are your best friends.

So, when your instance starts slowing down (you're trying to connect to ssh, it takes over 10 minutes...), your first reflex should be to check the number of CPU credits.

If those CPU credits are still pretty high though... turns out there's also a credit system for the hard drives!

As you can see from the last picture, in my case those credits are not showing at the instance level, for our account anyway.

Hard drive IOPS

AWS calculates a baseline for your hard drive performance based on the type of drive (gp2, gp3, io1, io2) and the size of it. The bigger the size of the drive, the bigger will be your IOPS and the expected throughput. If you're running a web server though, you're likely to be using a small hard drive with either gp2 or gp3. When using gp2 and gp3, your hard drive has the possibility of going over those limit. This is called the burst capacity (or burst balance).

Burst balance

Just like CPU credit, this balance goes up when the activity goes down. Here's the official documentation to help you understand and monitor your burst balance.

The quick fix is to switch from gp2/gp3 to io2. It will cost you more, but it will fix the slow crawl right away. The proper fix is probably to look into why your server is suddenly generating so many IOPS.

Conclusion

If you're having performance issues on your server, check your CPU usage, then check your CPU credits, then make sure to remember to also check your hard drive burst balance.