magpiebrain

We have lots of information available about our systems. CPU, disk IO, orders placed, error rates, users logged in. But typically all these pieces of information are collected, aggregated and stored in very different ways making correlation difficult and increasing the operational overhead of our systems. What if we could treat all of this information as events? What if we could aggregate, store, and report on all of this information as a uniform event stream?

This talk will look at emerging trends in the space of log aggregation, monitoring and event streaming to paint a picture for how you too can start to make real use of the information already available to you using nothing more complex than some free, off the shelf Open Source software.

I presented this talk at the joint ThoughtWorks and BBC Develop:North conference in October 2012, and again at GeeCon 2013 in Poland. A video should be available soon, but until then you can see the slides.