* This is only my own learning note *

NAPI (New API) and Parallel Processing with Softirqs

The polling mechanism in NAPI (New API) is implemented using softirqs, which allows for efficient and parallel packet processing across multiple CPUs. This design leverages the softirq infrastructure in the Linux kernel to handle network packet processing with minimal latency and high throughput.

How NAPI Utilizes Softirqs for Polling

1. Initial Packet Handling:

   • When packets arrive at the network interface card (NIC), the NIC generates a hardware interrupt.

   • The interrupt handler in the network driver performs minimal work, typically disabling further NIC interrupts and scheduling the NAPI poll function.

2. Scheduling the Poll Function:

   • The interrupt handler schedules the NAPI poll function by raising a NET_RX_SOFTIRQ.

   • This involves calling napi_schedule() or __napi_schedule(), which adds the napi_struct to a list of scheduled softirqs.

3. Softirq Execution:

   • Softirqs are processed by the kernel at a high priority, just after handling hardware interrupts.

   • Multiple CPUs can handle different softirqs simultaneously, allowing for parallel processing.

4. NAPI Poll Function Execution:

   • The NAPI poll function is executed in the context of the softirq.

   • This function processes incoming packets in batches, up to a specified budget, and passes them up the network stack.

5. Completion and Re-enabling Interrupts:

   • If the NAPI poll function processes all available packets (or reaches the budget), it calls napi_complete().

   • This function re-enables NIC interrupts if there are no more packets to process, allowing the NIC to generate new interrupts for subsequent packets.

Parallel Processing with Softirqs

Softirqs can run on multiple CPUs in parallel, which is a key advantage for handling high network traffic:

• Multiple CPUs Handling Softirqs: The Linux kernel can process different softirqs on multiple CPUs concurrently. This parallel processing capability helps distribute the load of packet processing across all available CPUs, enhancing performance and scalability.

• Efficient Load Balancing: By leveraging multiple CPUs, the system can efficiently balance the network processing load, ensuring that no single CPU becomes a bottleneck.

• High Throughput: The ability to process packets in parallel across multiple CPUs results in higher network throughput, making it suitable for high-performance networking environments.

Example Code: Scheduling and Handling Softirqs

Interrupt Handler

static irqreturn_t my_netdev_interrupt(int irq, void *dev_id)
{
    struct net_device *netdev = dev_id;
    struct my_netdev_priv *priv = netdev_priv(netdev);

    // Disable further interrupts from the NIC
    disable_device_interrupts(netdev);

    // Schedule the NAPI poll function
    if (likely(napi_schedule_prep(&priv->napi))) {
        __napi_schedule(&priv->napi);
    }

    return IRQ_HANDLED;
}

NAPI Poll Function

static int my_netdev_poll(struct napi_struct *napi, int budget)
{
    struct my_netdev_priv *priv = container_of(napi, struct my_netdev_priv, napi);
    struct net_device *netdev = priv->netdev;
    int work_done = 0;

    // Process packets up to the budget limit
    while (work_done < budget) {
        struct sk_buff *skb;

        // Fetch a packet from the device
        skb = fetch_packet_from_device(netdev);
        if (!skb)
            break;

        // Process the packet
        netif_receive_skb(skb);
        work_done++;
    }

    // If we processed fewer packets than the budget, we are done
    if (work_done < budget) {
        napi_complete(napi);
        // Re-enable interrupts
        enable_device_interrupts(netdev);
    }

    return work_done;
}

Benefits of Using Softirqs for NAPI

1. Minimal Latency: Softirqs ensure minimal latency for processing network packets, as they are handled immediately after hardware interrupts.

2. Low Overhead: Softirqs have lower overhead compared to tasklets and workqueues, making them suitable for high-frequency packet processing.

3. Parallel Processing: The ability to run softirqs on multiple CPUs enables parallel packet processing, improving scalability and performance.

4. Efficient CPU Utilization: By processing packets in batches, softirqs help reduce the number of context switches, leading to more efficient CPU utilization.

References

1. Linux Kernel Documentation: Provides detailed information on NAPI and softirqs.

   • NAPI Documentation

   • Softirqs Documentation

2. Understanding Linux Network Internals by Christian Benvenuti: Comprehensive insights into the Linux networking stack and NAPI.

3. Linux Device Drivers by Jonathan Corbet, Alessandro Rubini, and Greg Kroah-Hartman: Practical guide to writing network drivers, including NAPI.

Using softirqs to trigger NAPI ensures that network packet processing is efficient, scalable, and capable of handling high traffic loads without overwhelming the CPU, making it an ideal solution for high-performance networking.