Dealing with continuous data processing

Michael Peacock (29.Feb.2012 at 13:15)
Talk at ConFoo 2012 (English - UK)

Rating: 4 of 5

At Smith Electric Vehicles we process billions of pieces of data every week into our telematics system, all coming from remote collection devices continually sending us data. With that data, we need to be able to continually add it to our database, report against it, display recent data, maintain it, and most importantly - ensure we receive every byte of data. No easy challenge!

This talk will discuss some of the concepts, tools and technologies available to help deal with continually processing and managing data through de-coupled systems. From the services needed to ensure you can safely perform maintenance without loosing data, to reporting, storing, managing, displaying and reacting upon this data from within your web application, your database systems while keeping your hair in the process.

Who are you?

Claim talk

By clicking this button you are declaring that you are the speaker responsible for it and a claim request will be sent to the administrator of the event.

If the claim is approved you will be able to edit the information for this talk.

Are you sure?

 
Comments closed.

Comments

Rating: 4 of 5

29.Feb.2012 at 14:16 by Anonymous

I don't think it's useful to show us that much about your company you work for... No need to tell us the details of how fast a vehicule can go or what payload they can support. We are here for technology. The reasons to get that telemetry data is fine it sets the table for the amount of data. The amount of data too is fine and great, sets the table for the wow factor...

Apart from that, great historical review of the problems you had, serisouly, good work. Your slides where very complete and precise.

The last problem though is about you being tired and very monotone. We all pay to get into this convention, the fact that you had 4 different timezones in the last 2 weeks is not valid. You or whoever planned your over-extenuating schedule is the one to blame, we merit all the same energy and attention.

Still great job, keep up the good work...

Rating: 4 of 5

29.Feb.2012 at 14:52 by Pierre Voisin (27 comments)

Very interesting! We've got the big picture and you successfully made us dive into that scaliability adventure. Excellent use case and real examples for a great pleasure!

As a side note, like anonymous previoulsy said, I think you could fly a bit higher over numbers about your company. It's interesting but not sure it is 100% relevant here.

Rating: 4 of 5

29.Feb.2012 at 16:25 by Jean-Marc Fontaine (18 comments)

Very nice talk. A bit fast paced but informative.

Rating: 5 of 5

29.Feb.2012 at 22:31 by Hugo Hamon (124 comments)

I really liked this presentation and was impressed by the huge amount of data you have to process every single day. Very instructive use case that gives us another look at we can do with a PHP environment for a non web application.

Rating: 4 of 5

29.Feb.2012 at 23:36 by Anonymous


Rating: 4 of 5

01.Mar.2012 at 09:40 by Olivier Gauthier (3 comments) via api

A bit too fast. Too much detail. Should focus on the most important things. But very interesting with lots of useful information.

© Joind.in 2014