Using perfetto for network traffic

[yehuday]

Hello perfettoers :)

i am trying to use perfetto to visualize traffic flow on a network device. so i basically have many complete events (packet submit/complete) which overlap but aren't really related.
I am trying to display all of them in a single track (pid,tid AFAIK) and while perfetto UI does expand to track to 8 rows, to avoid visual overlapping, it does eventually display overlapping events one on top of the other.
i can;t really figure why some overlap in display and other get separate rows.

i tried assigning different categories to overlapping events but that didn't help.
i am using chrome tracing format simple because it's super easy to generate so would ideally like a sollution withing this format.
Alternatively, if there is a simple library/python module to generate perfetto format traces (other than the C++ sdk) i can move to perfetto.

[LalitMaganti]

Please read https://docs.google.com/document/u/0/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/mobilebasic

The very important part of this is this quote:

It is not possible to have non-nested duration events for a given thread. If you need to have durations that do not nest property you should use Async events instead.

Given you mention tids it seems very very likely to me that you are emitting X or B/E events. You should follow the advice there and instead emit async events instead as it says.

Alternatively you could emit the Perfetto native protobuf format instead which supports this sort of thing much more explicitly than hacking things using pids:
https://perfetto.dev/docs/reference/synthetic-track-event

[yehuday]

Thanks Lalit I am indeed using X events. I will see how async events work for that. I think i skipped them since I couldn't put them in different tracks which was important to me. Are you familiar with a python friendly method to write protobuff formatted events ?

…

On Thu, 13 Mar 2025, 22:22 Lalit Maganti, ***@***.***> wrote: Please read https://docs.google.com/document/u/0/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/mobilebasic The very important part of this is this quote: It is not possible to have non-nested duration events for a given thread. If you need to have durations that do not nest property you should use Async events instead. Given you mention tids it seems very very likely to me that you are emitting X or B/E events. You should follow the advice there and instead emit async events instead as it says. Alternatively you could emit the Perfetto native protobuf format instead which supports this sort of thing much more explicitly than hacking things using pids: https://perfetto.dev/docs/reference/synthetic-track-event — Reply to this email directly, view it on GitHub <#1030 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHX7LGYBIVTM72TTIN2JPC32UHSIPAVCNFSM6AAAAABY7ETSNSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENBZGM2DAMQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[LalitMaganti]

I am indeed using X events. I will see how async events work for that. I
think i skipped them since I couldn't put them in different tracks which
was important to me.

In your original post you say you want everything in the same track, not you mention you want different tracks. Can you please clarify what you want?

Are you familiar with a python friendly method to write protobuff formatted events ?

You can follow the guide at https://protobuf.dev/reference/python/python-generated/ and run it on https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/protos/perfetto/trace/perfetto_trace.proto;l=1?q=f:perfetto_trace

[LalitMaganti]

Though I've been increasingly feeling like we should just expose the protos in our python library. And there's a long standing idea for us to provide nice wrapper APIs to write these protos.

[yehuday]

> In your original post you say you want everything in the same track, not

you mention you want different tracks. Can you please clarify what you want? Let me clarify in more detail. So basically I need 2 levels of separations for displaying the packets in rows. THe first can be the device they were sent on (enps0, etc) and the second level is the packet type. but it could also be the address of remote peer and then packet type. I tried using async events but they are all grouped under "Global legacy events" and below that there is only one level of separation where the event name defines the row. That's why I tried to hack the pid/tid into 2 levels of separation.

> Though I've been increasingly feeling like we should just expose the

protos in our python library. And there's a long standing idea for us to provide nice wrapper APIs to write these protos. That would be awesome. a simple python API to leverage the perfetto format advantages would really be useful.

…

On Thu, Mar 13, 2025 at 11:21 PM Lalit Maganti ***@***.***> wrote: I am indeed using X events. I will see how async events work for that. I think i skipped them since I couldn't put them in different tracks which was important to me. In your original post you say you want everything in the same track, not you mention you want different tracks. Can you please clarify what you want? Are you familiar with a python friendly method to write protobuff formatted events ? You can follow the guide at https://protobuf.dev/reference/python/python-generated/ and run it on https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/protos/perfetto/trace/perfetto_trace.proto;l=1?q=f:perfetto_trace — Reply to this email directly, view it on GitHub <#1030 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHX7LG5KSBUVQIWBKUAGCSD2UHZFZAVCNFSM6AAAAABY7ETSNSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENBZGM4TEMI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[LalitMaganti]

Ah that makes sense. Yeah I don't think there's a good way to do what you want in Chrome format. Maybe you can hack it by making the top level of nesting the process? (Afaik async tracks can still be process scoped).

[yehuday]

AFAICT from the spec of chrome tracing (1), async events don't support tid/pid :( [1] https://docs.google.com/document/d/1CvAClvFfyA5R-PhYUmn5OOQtYMH4h6I0nSsKchNAySU/preview?tab=t.0#heading=h.nso4gcezn7n1)

…

On Sat, Mar 15, 2025 at 8:25 PM Lalit Maganti ***@***.***> wrote: Ah that makes sense. Yeah I don't think there's a good way to do what you want in Chrome format. Maybe you can hack it by making the top level of nesting the process? (Afaik async tracks can still be process scoped). — Reply to this email directly, view it on GitHub <#1030 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHX7LG6YOIPZRYIMTGBB7J32URWBJAVCNFSM6AAAAABY7ETSNSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENJRGE2DKMI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[LalitMaganti]

It might not be ab official part but unofficially it is and we support it: https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/src/trace_processor/importers/json/json_trace_parser_impl.cc;drc=7ac77be59009c916d8a5e2fb08090b298a7928d9;l=188

[yehuday]

Thanks a lot for that reference !! I switched to using "id2 : {local : <my-id>}" instead of "id" and it works perfectly.

…

On Sat, Mar 15, 2025 at 8:59 PM Lalit Maganti ***@***.***> wrote: It might not be ab official part but unofficially it is and we support it: https://cs.android.com/android/platform/superproject/main/+/main:external/perfetto/src/trace_processor/importers/json/json_trace_parser_impl.cc;drc=7ac77be59009c916d8a5e2fb08090b298a7928d9;l=188 — Reply to this email directly, view it on GitHub <#1030 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHX7LG2ZK4HGZ7EEU3HIC332UR2CLAVCNFSM6AAAAABY7ETSNSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTENJRGE3DGMY> . You are receiving this because you authored the thread.Message ID: ***@***.***>