C 令牌解析

Question

所以我正在尝试实现一个不使用任何 C 库函数（如 strtok() 等）的令牌解析器，但我遇到了一些访问冲突问题，并且在阅读了这里的几个类似问题后仍然没有搞定了。有人愿意提供一些指点吗？

int main(int argc, char* argv[])
{
    int maxTokens = 10;
    char* tokens[10];

    int i;
    for(i = 0; i < maxTokens; i++)
    {
        tokens[i] = NULL;
    }

    char* str = "This,is,a,test,string";

    int result = parseLine(str, ',', tokens, maxTokens);

    printf("%d tokens were found!", result);
    system("PAUSE");
    return 0;
}

int parseLine(char* str, char delimeter, char* tokens[], int maxTokens)
{
    char* srcStr = str;

    int strlen = 0;
    int tokenCount = 0;

    if(srcStr[strlen] != delimeter && srcStr[strlen] != '\0')
    {
        tokens[tokenCount] = (char*) malloc(sizeof(char)*strlen+1);
        tokens[tokenCount] = &srcStr[strlen];
        tokenCount++;
    }

    while(srcStr[strlen] != '\0')
    {
        if(srcStr[strlen] == delimeter)
        {
            tokens[tokenCount-1][strlen] = '\0';
            if(srcStr[strlen+1] != '\0')
            {
                tokens[tokenCount] = (char*) malloc(sizeof(char)*strlen+1);
                tokens[tokenCount] = &srcStr[++strlen];
                tokenCount++;
            }
        }
        else
        {
            strlen++;
        }
    }

    return tokenCount;
}

Answer 1

1) 艰难的路：

char* tokens[10];

int i;
for(i = 0; i < maxTokens; i++)
{
    tokens[i] = NULL;
}

简单的方法：

char tokens[10] = { NULL };

2) 此行不会复制字符串（只会创建另一个对其的引用）

char* srcStr = str;

这将：

char* srcStr = (char*) malloc ( strlen(str) + 1 );
strcpy( srcStr , str );

3) 不要重新发明轮子，除非你真的必须这样做。我已经学会了这个艰难的方式。相信我。你在你的功能中犯了很多错误。如果您真的想出于“教育”目的或其他目的这样做，请先获取有关指针和字符串的一些信息

Answer 2

“有人愿意提供一些指点吗？”

不过，严肃地说，考虑使用调试器（如果您使用的是 Windows，我建议使用 Visual Studio）或 Valgrind（仅限 Linux）来捕获访问冲突。

即使根本没有阅读您的代码，我也能够获得有关发生段错误的行号的有用信息：

==8272== Memcheck, a memory error detector
==8272== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==8272== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==8272== Command: ./a.out
==8272== 
==8272== 
==8272== Process terminating with default action of signal 11 (SIGSEGV)
==8272==  Bad permissions for mapped region at address 0x400868
==8272==    at 0x40069E: parseLine (asdf.c:22)
==8272==    by 0x400790: main (asdf.c:52)
==8272== 
==8272== HEAP SUMMARY:
==8272==     in use at exit: 1 bytes in 1 blocks
==8272==   total heap usage: 1 allocs, 0 frees, 1 bytes allocated
==8272== 
==8272== LEAK SUMMARY:
==8272==    definitely lost: 1 bytes in 1 blocks
==8272==    indirectly lost: 0 bytes in 0 blocks
==8272==      possibly lost: 0 bytes in 0 blocks
==8272==    still reachable: 0 bytes in 0 blocks
==8272==         suppressed: 0 bytes in 0 blocks
==8272== Rerun with --leak-check=full to see details of leaked memory
==8272== 
==8272== For counts of detected and suppressed errors, rerun with: -v
==8272== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
Segmentation fault

在您的代码中，这对应于 tokens[tokenCount-1][strlen] = '\0'; 行。