Developers of modern web applications strive for fast response times and efficiency. One of the ways to achieve them is to postpone performing costly and potentially failing operations like sending an e-mail after the HTTP request is complete and the user has seen that his intended action has been successful. This is called asynchronous task processing. In the past it was usually achieved with periodically executed scripts by Cron. This solution requires inserting data about tasks into a persistent storage like a relational database and lock the data to prevent duplicate execution. Tasks are not performed instantly but within the next run of a script. It's also not easy to scale task processing to multiple executors at the same time. This approach became popular because of limited capabilities of shared webhosts. But in the last years it has been more and more difficult to make excuses for preferring Cron over alternative approaches thanks to decreasing prices of VPSes. Message queues do not share the problems of Cron-executed scripts - they offer instant task processing and easy scalability. But at the same time this concept can be more difficult to grasp and creates new troubles in different areas - mainly deployment and integration into existing codebases. In this talk, I will dive into specifics, advantages and disadvantages of developing a web application with the help of RabbitMQ or a similar technology, and share everything we had to do to be able to produce and consume hundreds of thousands messages a day within a large legacy PHP codebase of an application that serves >200k daily visitors.


Comments are closed.

I enjoyed this talk. It seemed like an introduction to RabbitMQ as it started off showing how a cronjob isn't the right way of going about things. The reason I gave 4 instead of 5 (4 is still "Great info & insight", btw), is because I would've liked to hear a lot more on the clustering and master / slave side of things and we could perhaps have skipped the the long introduction to queues and moved to this faster instead. Great talk and looking forward to hearing more on the topic.

I wondered if it was easy or difficult to make it work and it half convinced me as it demonstrated that it implies some complexity. In addition, it shows that it's useful for high trafic application so I'm not concerned yet :)