Data Dog and Sysdig were both reporting a 502 error from the HAproxy frontend.
On the App Dynamics side they were seeing a Java exception.
They are a record of every system call that passes through your host.
They are really just a flat file, with one record per line.
We need a quick, easy way to go deeper into our processes and understand what’s happening.
And we’ve got one tool within our Sysdig toolbox that we haven’t used yet: sysdig captures.
The information is really rich, allowing you to see everything that a process is doing, including network activity, file activity, container-related activity and more.
If you’re familiar with Wireshark, captures are similar in format and use, but sysdig captures are much richer. This is an illegal HTTP response and HAPROXY was rejecting it. Monitoring tools are typically great for alerting you to a problem, and in many cases getting you very close to what might be the root cause.If you want to read more about captures, read Troubleshooting Containers after they’re long gone. But many times, you have to get knee deep into it before you can really solve the problem.You can independently use sysdig captures with open source sysdig, but using Sysdig’s monitoring tool allows you to store captures centrally and even trigger captures on alerts. If you want to give your troubleshooting a leg up, add open source sysdig to your toolbox!Too often monitoring and performance management tools are good at giving you the “what” of your software but don’t help with the “why.” But, it’s usually the “why” that eats up hours or days of your time.Tracking down bugs and performance issues can be incredibly tricky, no matter how many good tools you have at your disposal.Interestingly, the customer had three commercial tools that could help with this issue.