Asynchronous processing with RabbitMQ

Developers of modern web applications strive for fast response times and efficiency. One of the ways to achieve them is to postpone performing costly and potentially failing operations like sending an e-mail after the HTTP request is complete and the user has seen that his intended action has been successful. This is called asynchronous task processing.

In the past it was usually achieved with periodically executed scripts by Cron. This solution requires inserting data about tasks into a persistent storage like a relational database and lock the data to prevent duplicate execution. Tasks are not performed instantly but within the next run of a script. It’s also not easy to scale task processing to multiple executors at the same time. This approach became popular because of limited capabilities of shared webhosts. But in the last years it has been more and more difficult to make excuses for preferring Cron over alternative approaches thanks to decreasing prices of VPSes.

Message queues do not share the problems of Cron-executed scripts – they offer instant task processing and easy scalability. But at the same time this concept can be more difficult to grasp and creates new troubles in different areas – mainly deployment and integration into existing codebases.

In this talk, I will dive into specifics, advantages and disadvantages of developing a web application with the help of RabbitMQ or a similar technology, and share everything we had to do to be able to produce and consume hundreds of thousands messages a day within a large legacy PHP codebase of an application that serves >200k daily visitors.

Comments

Comments are closed.