如何为 IOCP 构建工作线程逻辑

Question

我正在创建一个客户端程序，它与通过 LAN 连接到我的 PC 的设备进行通信。

我的程序和设备之间的典型通信如下：

Program -> Device   1616000D  08 02 00 00 00 21 11 A1 00 01 22 08 00 // Sender sends data (a specific command to the device) to Receiver
Program <- Device   16160002  80 00 // Receiver sends ACK to sender
Program <- Device   16160005  08 20 00 00 00 // Receiver sends command response to sender
Program -> Device   16160002  80 00 // Sender sends ACK to receiver

第一个字节序列的最后一个十六进制数字表示后面的数据大小（D = 13 字节）。

我的发送例程如下所示：

bool TcpConnection::SendCommand(const Command& rCommand, const std::vector<BYTE>& rvecCommandOptions)
{
    std::vector<BYTE> vecCommandData;
    m_commandBuilder.BuildCommand(rCommand, rvecCommandOptions, vecCommandData);

    if (vecCommandData.empty())
        return false;

    PerIoData *pPerIoData = new PerIoData;
    if (!pPerIoData)
        return false;

    SecureZeroMemory(&(pPerIoData->m_overlapped), sizeof(WSAOVERLAPPED));

    pPerIoData->m_socket = m_socket.Get();
    pPerIoData->m_overlapped.hEvent = WSACreateEvent();
    pPerIoData->m_vecBuffer.assign(vecCommandData.begin(), vecCommandData.end());
    pPerIoData->m_wsaBuf.buf = (CHAR*)(&(pPerIoData->m_vecBuffer[0]));
    pPerIoData->m_wsaBuf.len = pPerIoData->m_vecBuffer.size();
    pPerIoData->m_dwFlags = 0;
    pPerIoData->m_dwNumberOfBytesSent = 0;
    pPerIoData->m_dwNumberOfBytesToSend = pPerIoData->m_wsaBuf.len;
    pPerIoData->m_operationType = OP_TYPE_SEND;

    if (!m_socket.Send(pPerIoData))
        return false;

    return true;
}

我的工作线程例程如下所示：

DWORD WINAPI TcpConnection::WorkerThread(LPVOID lpParameter)
{
    HANDLE hCompletionPort = (HANDLE)lpParameter;
    DWORD dwNumberOfBytesTransferred;
    ULONG ulCompletionKey;
    PerIoData *pPerIoData;

    DWORD dwNumberOfBytesReceived;
    DWORD dwNumberOfBytesSent;
    DWORD dwFlags;

    while (GetQueuedCompletionStatus(hCompletionPort, &dwNumberOfBytesTransferred, &ulCompletionKey, (LPOVERLAPPED*)&pPerIoData, INFINITE)) 
    {
        if (!pPerIoData)
            continue;

        if ((dwNumberOfBytesTransferred == 0) && ((pPerIoData->m_operationType  == OP_TYPE_SEND) || (pPerIoData->m_operationType  == OP_TYPE_RECEIVE)))
        {
            closesocket(pPerIoData->m_socket);
            delete pPerIoData;
            continue;
        }

        if (pPerIoData->m_operationType == OP_TYPE_SEND)
        {
            pPerIoData->m_dwNumberOfBytesSent += dwNumberOfBytesTransferred;
            if (pPerIoData->m_dwNumberOfBytesSent < pPerIoData->m_dwNumberOfBytesToSend)
            {
                pPerIoData->m_wsaBuf.buf = (CHAR*)(&(pPerIoData->m_vecBuffer[pPerIoData->m_dwNumberOfBytesSent]));
                pPerIoData->m_wsaBuf.len = (pPerIoData->m_dwNumberOfBytesToSend - pPerIoData->m_dwNumberOfBytesSent);

                if (WSASend(pPerIoData->m_socket, &(pPerIoData->m_wsaBuf), 1, &dwNumberOfBytesTransferred, 0, &(pPerIoData->m_overlapped), NULL) == 0)
                    continue;

                if (WSAGetLastError() == WSA_IO_PENDING)
                    continue;
            }
            else if (pPerIoData->m_dwNumberOfBytesSent == pPerIoData->m_dwNumberOfBytesToSend)
            {
                delete pPerIoData;
            }

            // Q1. Do I create a new instance of PerIoData here before calling WSARecv() or reuse pPerIoData?

            // QA. If I did do "PerIoData pPerIoData = new PerIoData" here, how do I handle if this momory allocation request has failed?  Should I simply "continue" or "return -1"?

            // QB. Or is this a wrong place to do this memory allocation to achive the typical communication between my program and the device?

            SecureZeroMemory(&(pPerIoData->m_overlapped), sizeof(WSAOVERLAPPED));

            pPerIoData->m_overlapped.hEvent = WSACreateEvent();
            pPerIoData->m_wsaBuf.buf = (CHAR*)(&(pPerIoData->m_vecBuffer[0]));
            pPerIoData->m_wsaBuf.len = pPerIoData->m_vecBuffer.size();
            pPerIoData->m_operationType = OP_TYPE_RECEIVE;

            if (WSARecv(pPerIoData->m_socket, &(pPerIoData->m_wsaBuf), 1, &dwNumberOfBytesReceived, &(pPerIoData->m_dwFlags), &(pPerIoData->m_overlapped), NULL) == 0)
                continue;

            if (WSAGetLastError() == WSA_IO_PENDING)
                continue;
        }
        else if (pPerIoData->m_operationType == OP_TYPE_RECEIVE)
        {
            if ((pPerIoData->m_vecBuffer[0] == 0x16) && (pPerIoData->m_vecBuffer[1] == 0x16))
            {
                // Q2. Do I need to do SecureZeroMemory(&(pPerIoData->m_overlapped), sizeof(WSAOVERLAPPED)); here?

                // Q3. Or do I new PerIoData?

                pPerIoData->m_wsaBuf.buf = (CHAR*)(&(pPerIoData->m_vecBuffer[0]));
                pPerIoData->m_wsaBuf.len = pPerIoData->m_vecBuffer.size();
                pPerIoData->m_operationType = OP_TYPE_RECEIVE;

                // QC. At this point two syn bytes (0x16) are received.  I now need to receive two more bytes of data (000D = 13 bytes) to find out the size of the actual command response data.
                // If I clear my m_vecBuffer here and try to resize its size to two, I get this debug assertion: "vector iterators incompatible" at runtime.  Do you know how I can fix this problem?

                if (WSARecv(pPerIoData->m_socket, &(pPerIoData->m_wsaBuf), 1, &dwNumberOfBytesReceived, &(pPerIoData->m_dwFlags), &(pPerIoData->m_overlapped), NULL) == 0)
                    continue;

                if (WSAGetLastError() == WSA_IO_PENDING)
                    continue;
            }

            // QD. I'm not sure how to structure this if clause for when m_operationType is OP_TYPE_RECEIVE.  I mean how do I distinguish one receive operation for getting two syn bytes from another for getting data size?
            // One way I can think of doing is to create more receive operation types such as OP_TYPE_RECEIVE_DATA_SIZE or OP_TYPE_RECEIVE_DATA?  So you can have something like below.
            // Is this how you would do it?
        }
        //else if (pPerIoData->m_operationType == OP_TYPE_RECEIVE_DATA_SIZE)
        //{
            // Call WSARecv() again to get command response data
        //}
    }

    return 0;
}

请在上面的代码中查看我的问题。

非常感谢

Answer 1

1. 正如您的 PerIoData 类型的名称所指，您需要一个数据结构每个不完整的 I/O 请求。从您使用 WSASend 或 WSARecv 启动异步 I/O 到您使用 GetQueuedCompletionStatus 从 I/O 完成端口检索该请求的完成数据包时，PerIoData 结构应该一直存在。
2. 当您即将开始一个新请求时，您应该始终重新初始化您的 OVERLAPPED 结构。
3. 只要 I/O 请求完成，您就可以重新使用 PerIoData 结构。鉴于您已从 I/O 完成端口检索到 pPerIoData，您可以将其重用于后续请求。只需确保您已重置该结构中的所有适用字段，使其处于适合新 I/O 请求的状态。

编辑回答后续问题：

A.我会continue，因为即使您无法发起其他请求，您也想继续处理 I/O 事件。如果您不continue，那么您将无法处理更多的 I/O 完成。在continue 之前，您可能需要调用某种错误处理程序。

B.我认为不必分配“正确”或“错误”的位置，但请记住，当您在那里分配 PerIoData 时，您实际上最终要做的是重复分配和删除相同的数据结构并在一个循环中结束。当我使用 I/O 完成端口编写代码时，我会预先分配一个 PerIoData 等效池并重新使用它们。

C.我没有足够的上下文来知道答案。显示执行此操作的代码以及断言命中的行，我可能会提供帮助。

D.您可以按照您的建议将操作类型分解为更细粒度的组件，例如 OP_TYPE_RECEIVE_DATA_SIZE 操作。作为警告，在每个WSARecv 调用上读取几个字节不会像您希望的那样执行。 Winsock 调用很昂贵；请求几个字节的开销很大。我建议您在一个WSARecv 中将更大的数据块读入PerIoData 缓冲区。然后将您的尺寸信息从该缓冲区中提取出来，然后开始将您的数据从该缓冲区中复制出来。如果到达的数据多于缓冲区无法容纳的数据，那么您可以进行额外的WSARecv 调用，直到您读完其余数据。