[Intel-wired-lan] [PATCH net v2] i40e: Fix incorrect netdev's real number of RX/TX queues

[Jagielski, Jedrzej]

>> There was a wrong queues representation in sysfs during
>> driver's reinitialization in case of online cpus number is
>> less than combined queues. It was caused by stopped
>> NetworkManager, which is responsible for calling vsi_open
>> function during driver's initialization.
>> In specific situation (ex. 12 cpus online) there were 16 queues
>> in /sys/class/net/<iface>/queues. In case of modifying queues with
>> value higher, than number of online cpus, then it caused write
>> errors and other errors.
>> Add updating of sysfs's queues representation during driver
>> initialization.
>
>Description is not very clear to me - how did you get the value of 16
>queues? Do you have 16 cpu system and then modified the count of cpus
>being online?
>

When You have less than 16 available/online cpus and network manager
is disabled, the maximum preset number of combined queues of the
interface goes to 16. So you can set combined queues up to 16.
Then there is sysfs representation of TX/RX queues index greater than
available/online cpus. Then trying to write some value to (for example)
/sys/class/net/ethx/queues/tx-15/xps_cpus causes Invalid argument error.

>> 
>> Fixes: 41c445ff0f48 ("i40e: main driver core")
>> Signed-off-by: Lukasz Cieplicki <lukaszx.cieplicki at intel.com>
>> Signed-off-by: Jedrzej Jagielski <jedrzej.jagielski at intel.com>
>> ---
>> v2:Fix "CHECK: Lines should not end with a '('" warning
>> ---
>>  drivers/net/ethernet/intel/i40e/i40e_main.c | 19 +++++++++++++++++++
>>  1 file changed, 19 insertions(+)
>> 
>> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
>> index 4ff1c9b9217b..1b6f03cc41da 100644
>> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
>> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
>> @@ -14074,6 +14074,22 @@ static struct i40e_vsi *i40e_vsi_reinit_setup(struct i40e_vsi *vsi)
>>  	return NULL;
>>  }
>>  
>> +/**
>> + * i40e_netif_set_realnum_tx_rx_queues - Update number of tx/rx queues
>> + * @vsi: vsi structure
>> + *
>> + * This updates netdev's number of tx/rx queues
>> + *
>> + * Returns status of setting tx/rx queues
>> + **/
>> +static int i40e_netif_set_realnum_tx_rx_queues(struct i40e_vsi *vsi)
>> +{
>> +	netif_set_real_num_rx_queues(vsi->netdev, vsi->num_queue_pairs);
>
>Why return value of above is not checked?
>
>Also would be good to convert the i40e_vsi_open() to make use of this
>function you're introducing, I guess.
>
Sure, good point.
>> +
>> +	return netif_set_real_num_tx_queues(vsi->netdev,
>> +					    vsi->num_queue_pairs);
>> +}
>> +
>>  /**
>>   * i40e_vsi_setup - Set up a VSI by a given type
>>   * @pf: board private structure
>> @@ -14203,6 +14219,9 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,
>>  	case I40E_VSI_MAIN:
>>  	case I40E_VSI_VMDQ2:
>>  		ret = i40e_config_netdev(vsi);
>> +		if (ret)
>> +			goto err_netdev;
>> +		ret = i40e_netif_set_realnum_tx_rx_queues(vsi);
>>  		if (ret)
>>  			goto err_netdev;
>>  		ret = register_netdev(vsi->netdev);
>> -- 
>> 2.27.0

Thanks for your suggestions,
Jedrzej


-----Original Message-----
From: Fijalkowski, Maciej <maciej.fijalkowski at intel.com> 
Sent: środa, 15 grudnia 2021 11:58
To: Jagielski, Jedrzej <jedrzej.jagielski at intel.com>
Cc: intel-wired-lan at lists.osuosl.org; Cieplicki, LukaszX <lukaszx.cieplicki at intel.com>
Subject: Re: [Intel-wired-lan] [PATCH net v2] i40e: Fix incorrect netdev's real number of RX/TX queues

On Wed, Dec 15, 2021 at 10:45:54AM +0000, Jedrzej Jagielski wrote:
> There was a wrong queues representation in sysfs during
> driver's reinitialization in case of online cpus number is
> less than combined queues. It was caused by stopped
> NetworkManager, which is responsible for calling vsi_open
> function during driver's initialization.
> In specific situation (ex. 12 cpus online) there were 16 queues
> in /sys/class/net/<iface>/queues. In case of modifying queues with
> value higher, than number of online cpus, then it caused write
> errors and other errors.
> Add updating of sysfs's queues representation during driver
> initialization.

Description is not very clear to me - how did you get the value of 16
queues? Do you have 16 cpu system and then modified the count of cpus
being online?

> 
> Fixes: 41c445ff0f48 ("i40e: main driver core")
> Signed-off-by: Lukasz Cieplicki <lukaszx.cieplicki at intel.com>
> Signed-off-by: Jedrzej Jagielski <jedrzej.jagielski at intel.com>
> ---
> v2:Fix "CHECK: Lines should not end with a '('" warning
> ---
>  drivers/net/ethernet/intel/i40e/i40e_main.c | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
> 
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c
> index 4ff1c9b9217b..1b6f03cc41da 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_main.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c
> @@ -14074,6 +14074,22 @@ static struct i40e_vsi *i40e_vsi_reinit_setup(struct i40e_vsi *vsi)
>  	return NULL;
>  }
>  
> +/**
> + * i40e_netif_set_realnum_tx_rx_queues - Update number of tx/rx queues
> + * @vsi: vsi structure
> + *
> + * This updates netdev's number of tx/rx queues
> + *
> + * Returns status of setting tx/rx queues
> + **/
> +static int i40e_netif_set_realnum_tx_rx_queues(struct i40e_vsi *vsi)
> +{
> +	netif_set_real_num_rx_queues(vsi->netdev, vsi->num_queue_pairs);

Why return value of above is not checked?

Also would be good to convert the i40e_vsi_open() to make use of this
function you're introducing, I guess.

> +
> +	return netif_set_real_num_tx_queues(vsi->netdev,
> +					    vsi->num_queue_pairs);
> +}
> +
>  /**
>   * i40e_vsi_setup - Set up a VSI by a given type
>   * @pf: board private structure
> @@ -14203,6 +14219,9 @@ struct i40e_vsi *i40e_vsi_setup(struct i40e_pf *pf, u8 type,
>  	case I40E_VSI_MAIN:
>  	case I40E_VSI_VMDQ2:
>  		ret = i40e_config_netdev(vsi);
> +		if (ret)
> +			goto err_netdev;
> +		ret = i40e_netif_set_realnum_tx_rx_queues(vsi);
>  		if (ret)
>  			goto err_netdev;
>  		ret = register_netdev(vsi->netdev);
> -- 
> 2.27.0
> 
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan at osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan