Scaling MySQL writes through partitioning

MySQL is fairly fast for most applications, but every now and then you might come across an application that needs fast writes at a very large scale. The problem with this is that if your inserts are not ordered according to your primary key, then once you cross a limit known as the innodb_buffer_pool_size (for InnoDB tables), write performance starts to degrade because you're now hitting disk a lot. This limit is dependent on the amount of RAM you have, and that in turn is limited by how much money you can throw at the problem. At some point it makes sense to close your pocket and start thinking of a way to hack around the problem.

Faced with a similar problem, we figured out a way to hack it with MySQL 5's partitioning to scale writes to a consistently high rate.

In this talk, I'll cover all the steps we went through to get to this solution in the hopes that either the solution itself or the thought processes behind it will help others solve their own scaling problems.