> I look at the reddest part of the chart, I look at the peaks
Neither of these are really the places I look at when examining flame graphs. I tend to look at the bottom, and work my way up.
The key thing (imo) to look for are wide pieces that are not a core part of what the code you're profiling is supposed to do.
In the first example of your first link, you have a flame graph of code that seems to draw an image. There are a bunch of functions like 'CopyPixels' and 'RasterizeEdges'. Those seem to be a core part of drawing an image. In contrast, there's a 'KiPageFault' function that's pretty wide that does not seem to be important to the code's overall goal. I think a bit more context is needed to arrive at the author's conclusion, though.
I think flame graphs, like all graphs, are more helpful when the reader has a lot of context about what's supposed to happen, or some intuition about how the chart is supposed to look.
> I think flame graphs, like all graphs, are more helpful when the reader has a lot of context about what's supposed to happen, or some intuition about how the chart is supposed to look.
Yeah, and then your comment... just ends? So, what I get here is that, in a flame graph, the reddest part isn't the most interesting, and neither is the widest part, nor the part with the most peaks.
So, what, exactly, am I looking for? "You know what this graph should look like in case all was OK" is not exactly helpful, right, because I might just lack that baseline?
This can’t be answered in general. Flamegraphs are measurements of what happened. But just like a ruler doesn’t tell you whether a given human is atypically short or tall for its species, a flamegraph can’t tell you which portion of the program takes too long a time. You need to have prior knowledge about data structures, algorithms, memory bandwidth etc in order to confront your justified expectations with the reality and be surprised with something. And it will all depend on the particular program you profile.
Width in a flame graph is directly proportional to runtime. Optimizing a block that covers x% of the graph will only speed up the program by x% or less, so probably dont bother with blocks less than 0.5% wide.
This by itself should already tell you what NOT to optimize.
But really, you should be looking for operations that take a long time but shouldn't (wide blocks that should be thin). To find it you need to have an intuitive idea of how fast things should be beforehand.
If you have no idea how fast things should be, no amount of graphs will help you with this.
You're looking for a wide part for a function that you don't expect to have a wide part.
So start at the widest part and work your way up each hill. If you see a function that's still relatively wide that you think should not be wide, that's when you go look into why it's wide.
As I understand it, flame graphs add to normal x-axis-as-time ones by merging repeated calls to the same function, so if one function is called a lot it shows up as one wide chunk and not many small chunks spread out.
So yes, height doesn't matter much, you read bottom to top to get context, and width is what you're looking for.
But like any chart, it can't tell you what to optimise, or what Can be optimized. It just tells you what parts of the code are taking up the most time, in a hierarchical way. Asking for a complete description of what to see here is like asking for a complete description of what to look for in a histogram. Like, ofc it matters what the histogram is of!
> As I understand it, flame graphs add to normal x-axis-as-time ones by merging repeated calls to the same function, so if one function is called a lot it shows up as one wide chunk and not many small chunks spread out.
Not if the graph is sorted on time axis. Not if the callstacks look different in different cases. Not if the program is recursive and thus have different depth all over the place. There is a lot of places that merging can fall over and not happen.
If you sort it on a time axis, its not really a flame graph any more. The whole point of a flame graph is to group same stacks together, even if they happen at multiple disparate times.
Neither of these are really the places I look at when examining flame graphs. I tend to look at the bottom, and work my way up.
The key thing (imo) to look for are wide pieces that are not a core part of what the code you're profiling is supposed to do.
In the first example of your first link, you have a flame graph of code that seems to draw an image. There are a bunch of functions like 'CopyPixels' and 'RasterizeEdges'. Those seem to be a core part of drawing an image. In contrast, there's a 'KiPageFault' function that's pretty wide that does not seem to be important to the code's overall goal. I think a bit more context is needed to arrive at the author's conclusion, though.
I think flame graphs, like all graphs, are more helpful when the reader has a lot of context about what's supposed to happen, or some intuition about how the chart is supposed to look.