Building a Firehose


Comments are closed.

Great talk - I think the slide deck will form my todo list for the next few weeks.

I thought the talk would have benefitted from a better description of the problem and the challenges faced in order to put the examples into better context.

wow! Heavy going at times to keep up with what was happening, but a very interesting and enjoyable talk, well devlivered!

Excellent talk, nice and indepth but at times it left me trying to catch up

Impressive, it's not easy to deliver such a technical talk without losing the audience but it seems Ian was able to pull it off.

Chose this talk at random from the last 3, not really sure what it would be about (and not remembering the abstract).

Turned out to be a really useful and informative talk (happy accident!).

Very well presented, well spoken and well informed speaker, lots of appropriate references to real life examples and covered a wide variety of different software that could be used to accomplish the same end-game (felt very unbiased which is refreshing).

Pick a different name next time. It was kind of based on the assumption people knew what a "firehose" is. I didn't (at least not in this context), and it would be a shame for that small detail to put people off any future presentations on the subject.


Very interesting description of how much complexity is involved in a real-time data stream. There were a lot of ideas and concepts presented in a short amount of time, and it wasn't easy to keep up with it all. Having said that, I could have quite happily listened to the same amount of content with more explanation in a double-length slot.

Excellent talk, and nice to hear a project I helped build get a mention!

Great talk, well presented and excellent reference material to back it up. Big data pipes like the ones described are never easy to implement and it's nice to know we were on the right track in our implementation!

Well structuctured, in depth information nicely presented.

Ian at his best as usual.

In depth and well structured. Good examples and very clear presentation.

Ian's presentation provoked quite a lot of thought for me about how we typically deal with large volumes of data. The analogy of a firehose actually describes it very well. Defining real-time as the time for data to propagate through a system was interesting and was backed up with some really good examples.

The only downside for me was the sheer amount of information being presented. Whilst it was well structured and well presented there was very little time to actually ponder many of the concepts being discussed.

Good content, great slides. I'm not giving it 5 stars as parts of it were rushed because of time constraints. I will need to read up on a few more things again now.

The speaker seemed very experienced and comfortable on stage. A lot of detail in a short time. The talk was a firehose of data in itself. Many very interesting technologys mentioned. I learned a lot. Thanks.

In typical fashion Ian filled the room and left the audience with tons to ponder. This session was rammed with useful advice on processing real time data, and demonstrated some interesting architectures for applying / managing synchronisation. Being tasked to loosen up a very ridged data flow on a current project meant I found this talk very valuable.

However, I do feel some aspects were compromised to fit the time slot. I would have liked:
An elaboration on the problem with less haste moving into solutions.
Some background on the technologies used (Zookeeper / ZMQ / Redis) in the solutions (knowledge was sometimes assumed).
An occasional pause to allow absorption (specifically when screen sliding code). I think a fair part of the audience got dazzled on this one, hence the "no questions".

+1 for a double-length slot.

Always nice when a meaty topic with lots of code is well presented. There was a lot of information in the slides which would make a good reference. Perhaps too much for the time allowed but felt engaged throughout.

My first introduction to firehoses so I was lost within the first 30 seconds, but the slides kept things clear and I was able to follow the key principles and problems. Certainly something to look in to at a later date.

Anonymous at 17:58 on 8 Oct 2012

Great talk and relevant to things I've been doing at work.

Great talk, and a great demonstration of the power of open source tools when combined in a novel way. Really made me think and spurred discussions throughout the night.

Really loved this talk - so much meat on the bones! Some may say too much technical content, but that's how I like it - talks like this can be springboards, laying the groundwork which you can then research and experiment a bit more with and thus end up getting your head round some pretty big new concepts.

There's always an issue of time with these short conference talks - should the speaker keep from going into much detail so that the content remains manageable and the audience can keep up at all times? Alternatively the speaker will choose to give a little bit more than can easily be digested at the time, so the audience go away with new ideas still swirling around their heads not yet fully understood.

Both can work, and I think Ian was a great example of the second type of speaker. Thank you!

Anonymous at 11:44 on 9 Oct 2012

In depth talk and I really enjoyed it. Well done!

One of my favourite talks. Case study really helped me to understand the subject. Thanks.

A really well presented talk on a potentially complicated subject. Lost me several times and I'm going to have to read back through the slides more than once, but Ian made it seem really interesting and created a drive to learn beyond the session. Also introduced a couple of tools that I think I will have a real use for in the future.

Very good talk by Ian.

I've never looked into firehoses before so this was my introduction into the topic. A lot of complicated stuff mentioned in the talk however the slides and diagrams were very good so I was kept interested without feeling too lost.
Had to take lots of notes during this talk as most of the stuff that was mentioned I had never heard of before.

I'm not sure I knew quite what I was letting myself in for by attending this talk, but it covered a great array of tools and techniques that could be used to solve various problems you might come across when dealing with making your data accessible.
I can't pretend to have followed all of it and Ian did go quite fast at points, but it was a good if flying introduction to tools and issues you might have to consider in your own application