Logs and traces generated by applications are valuable sources of information that can help detect issues and improve performance. However, they are often treated separately from other data, even though they are no different from the data an application works with. In this talk, we will explore a different approach: treating logs and traces as part of a scalable cloud storage repository that can be analyzed with the same techniques used for big data. By keeping all the data together, we can apply machine learning models to detect situations of interest and alert us in real-time when unwanted behavior is occurring or brewing. This approach enables intelligent monitoring that goes beyond simple threshold-based alerts and can help identify complex issues that would otherwise go unnoticed. We will discuss how to harness existing technologies to implement this approach, providing attendees with practical tips and insights that they can apply to their own projects.

Comments

Please login to leave a comment

Very interesting talk. For sure realtime/online ML is a really interesting topic related to observability and large scale application. The talk focused on the platform and it was a bit difficult for me to switch from the general approach presented and the concrete examples. I think I'll re-watch the video!

Luca Bovo at 10:50 on 19 Mar 2024

A very interesting even if complex speech.
The observability topic in software and microservices architectures is a must, but this talk is also a deep dive into ML processing of logs and traces so I really appreciated it.

Interesting and certainly technically intriguing, but perhaps the explanation was difficult to follow.