[Intel-wired-lan] [PATCH next-queue v1 0/3] igc: Add support for multiple TX tstamp requests

[Vinicius Costa Gomes]

Vladimir Oltean <vladimir.oltean at nxp.com> writes:

> On Mon, Feb 27, 2023 at 09:45:31PM -0800, Vinicius Costa Gomes wrote:
>> Patch 3 - More of an optimization. Use the ptp_aux_work kthread to do
>>       the work, and also try to do the work "inline" if the timestamp
>>       is ready already. Suggested by Vladimir Oltean and Kurt
>>       Kanzenbach.
>> 
>> Evaluation
>> ----------
>> 
>> To do the evaluation I am using a simple application that sends
>> packets (and waits for the timestamp to be received before sending the
>> next packet) and takes two measurements:
>
> If the application never generates multiple requests in flight, then
> this evaluation is only testing patch 3 (and patches 1 and 2 only to the
> extent that they don't cause a regression), right?
>

That's right. I was more interested in not causing a regression. I could
run the same test with two (or more) applications and give some numbers,
but those numbers couldn't be directly compared with the current version
of the code.

But good idea. I will change the application to send "batches" of
packets, so I can configure the number of "in flight" requests.

>>   1. from the HW timestamp value and the time the application
>>   retrieves the timestamps (called "HW to Timestamp";
>>   2. from just before the sendto() being called in the application to
>>   the time the application retrieves the timestamp (called "Send to
>>   Timestamp"). I think this measurement is useful to make sure that
>>   the total time to send a packet and retrieve its timestamp hasn't
>>   degraded.
>> 
>> (all tests were done for 1M packets, and times are in nanoseconds)
>> 
>> Before:
>> 
>> HW to Timestamp
>> 	min: 9130
>> 	max: 143183
>
> what margin of error did phc2sys have here? Tens, hundreds, thousands of
> ns, more? Was it a controlled variable? "HW to Timestamp" implies a
> comparison of 2 times from 2 different time sources, kept in sync with
> each other.
>

Should have provided these numbers, sorry. Yes, I was using phc2sys to
keep those different clocks (CLOCK_TAI and the NIC phc) synchronized,
and the phc2sys measured offset was in the order of tens of nanoseconds,
usually less than 20.

>> 	percentile 99: 10379
>> 	percentile 99.99: 11510
>> Send to Timestamp
>> 	min: 18431
>> 	max: 196798
>> 	percentile 99: 19937
>> 	percentile 99.99: 26066
>> 
>> After:
>> 
>> HW to Timestamp
>> 	min: 7933
>> 	max: 31934
>
> so the reduction of the max "HW to Timestamp" from 143 us to 32 us all
> the way to user space is mostly due to the inline processing of the TX
> timestamp, within the hardirq handler, right? Can you measure how much
> it is due to that, and how much due to the PTP kthread (simplest way
> would be to keep the kthread, but remove the inline processing)? How
> many reschedules of the kthread there are per TX timestamp? Even a
> single set of 4 numbers, denoting the maximum numbers of reschedules per
> timestamp request, would be useful information.
>

I will get these numbers, it will be useful for answering the questions
raised by that other patch.

>> 	percentile 99: 8690
>> 	percentile 99.99: 10598
>> Send to Timestamp
>> 	min: 17291
>> 	max: 46327
>> 	percentile 99: 18268
>> 	percentile 99.99: 21575
>> 
>> The minimum times are not that different
>
> right, probably because the time to do a context switch to user space
> dominates

Yep. Context switches and reading the PCIe registers account for most of
it.

>
>> , but we can see a big improvement in the 'maximum' time.


Cheers,
-- 
Vinicius