【问题标题】:Reader/Writer problem in C using pthreadsC中使用pthreads的读写器问题
【发布时间】:2023-04-08 07:52:01
【问题描述】:

当我在 C 中遇到读取器/写入器问题时,我正在学习 pthreads。问题非常简单,“写入器”线程将从外部源访问数据,而“读取器”线程将从共享缓冲区,然后伪处理数据。 reader 和 writer 线程需要在 while 循环中连续运行。

我正在尝试在具有 POSIX 接口的标准 unix 系统上实现和编译它。

我浏览了一些堆栈溢出问题:

Reader Writer program in C using mutexes and pthreads

reader/writer lock in pthread

而我没有得到这些。

这是我目前所拥有的:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <pthread.h>
#include <semaphore.h>
#include <assert.h>

#define BUFF_SIZE 50
#define M 10
#define N 20

int get_external_data(char *buffer, int bufferSizeInBytes);
void process_data(char *buffer, int bufferSizeInBytes);

int get_external_data(char *buffer, int bufferSizeInBytes){
    int status;
    int val;
    char srcString[] = "0123456789abcdefghijklmnopqrstuvwxyxABCDEFGHIJKLMNOPQRSTUVWXYZ";

    val = (int)(rand() % 62);
    if (bufferSizeInBytes < val){
        return (-1);
    }

    strncpy(buffer, srcString, val);
    return val;
}

void process_data(char *buffer, int bufferSizeInBytes){
    int i;
    if(buffer) {
        printf("thread %li - ", pthread_self());

        for(i = 0; i < bufferSizeInBytes; i++) {
            printf("%c", buffer[i]);
        }

        printf("\n");
        memset(buffer, 0, bufferSizeInBytes);
    } else {
        printf("error in process data - %li\n", pthread_self());
    }

    return;
}


pthread_mutex_t data_lock = PTHREAD_MUTEX_INITIALIZER;
pthread_mutex_t print_lock = PTHREAD_MUTEX_INITIALIZER;
sem_t data_count;

typedef struct node {
    struct node *next;
    char *data;
    int length;
} node_t;

node_t *head, *tail;

/**
 * This thread is responsible for pulling data off of the shared data 
 * area and processing it using the process_data() API.
 */

void *reader_thread(void *arg) {
    int rc;
    node_t *removed_node;

    while(1) {
        rc = sem_wait(&data_count);
        if (0 != rc) {
            return NULL;
        }

        pthread_mutex_lock(&data_lock);

        assert(NULL != head);
        removed_node = head;
        head = head->next;

        pthread_mutex_unlock(&data_lock);

        //Adding this lock for sake of readability at the cost of reduced consumption rate...will run out of memory eventually.

        pthread_mutex_lock(&print_lock);

        process_data(removed_node->data, removed_node->length);

        pthread_mutex_unlock(&print_lock);

        free(removed_node->data);
        free(removed_node);
    }

    return NULL;
}


/**
 * This thread is responsible for pulling data from a device using
 * the get_external_data() API and placing it into a shared area
 * for later processing by one of the reader threads.
 */

void *writer_thread(void *arg) {
    int length;
    char *buffer;
    node_t *new_node;

    new_node = (node_t*) malloc(sizeof(*new_node));
    buffer = (char*) malloc(sizeof(*buffer) * BUFF_SIZE);

    while(1) {

        length = get_external_data(buffer, BUFF_SIZE);

        if (length == -1) {
            //data too big, discard it and try again;
            continue;
        }

        new_node->next = NULL;
        new_node->length = length;
        new_node->data = buffer;

        pthread_mutex_lock(&data_lock);


        if (head == NULL) { //The linked list is completely empty   
            head = new_node;
            tail = new_node;        
        } else { //There are items in the list and we're appending  
            tail->next = new_node;
            tail = new_node;
        }

        pthread_mutex_unlock(&data_lock);

        pthread_mutex_lock(&print_lock);

        printf("thread %ld wrote - %s \n", pthread_self(), buffer);

        pthread_mutex_unlock(&print_lock);

        sem_post(&data_count);

        buffer = (char*) malloc(sizeof(*buffer) * BUFF_SIZE);
        new_node = (node_t*) malloc(sizeof(*new_node));
    }

    return NULL;
}

int main(int argc, char **argv) {
    int i = sem_init(&data_count, 0, 0);
    pthread_t dummy; //creating a dummy thread

    for(i = 0; i < N; i++) { 
        pthread_create(&dummy, NULL, reader_thread, NULL);
    }

    for(i = 0; i < M; i++) { 
        pthread_create(&dummy, NULL, writer_thread, NULL);
    }

    sleep(100);
    return 0;   
}

线程同步没有任何编译错误,但我的程序在写入器线程在缓冲区上写入几次后停止。

读取器和写入器线程需要在 while 循环中连续运行,但在我的情况下不需要。

知道如何解决这个问题吗?

【问题讨论】:

  • gcc .... -lpthread.
  • 好吧,在一个返回void * 的函数中有一个没有值的返回值。返回一些东西,。
  • 对于 sem_* 函数,您可能还需要 -lrt - 取决于平台
  • 您的程序不会停止,因为它永远不会启动。您的构建失败并出现一些编译器警告和链接错误。

标签: c pthreads posix mutex


【解决方案1】:

使用 gcc 添加选项 -lpthread 以链接 pthread 库,我也鼓励您添加选项 -pedantic -Wextra -Wall

你的程序有几个问题,有些是编译器指出的

在:

printf("thread %i - ", pthread_self());
printf("error in process data - %i\n", pthread_self());

在 POSIX 标准中,pthread_t 不需要是算术类型,因此它可以是结构等,您不能将其写为 int。在您的情况下,它可能是一个 int(更可能是一个 unsigned long),但这不是可移植的,最好自己管理与每个线程关联的标识符。

reader_thread

return;

但是函数返回一个void*,例如用return NULL;替换它

writer_thread

printf("thread %d wrote - %s", buffer);

有三个问题:

  • 缺少 int 类型的参数(可能您想要pthread_self()
  • bufferget_external_data 使用strncpy 设置,因此空结束字符 存在但printf %s 需要(当缺少的参数将被添加或删除%d
  • buffer 可以被reader_thread 释放,可能你认为它受到信号量的保护,但事实并非如此

get_external_data 状态未使用

 usleep(100);

给线程工作的时间很短,其实你只是想被阻塞,所以可以换成pthread_join(dummy, NULL);


一个线程标识符的提议:在堆中分配一个int,用一个唯一的数字设置它,并将它作为参数提供给创建的线程:

int main() {
    ...
    for(i = 0; i < N; i++) { 
      int * m = malloc(sizeof(int));

      *m = i;
      pthread_create(&dummy, NULL, reader_thread, m);
    }

    for(i = 0; i < M; i++) { 
      int * m = malloc(sizeof(int));

      *m = 100 + i;
      pthread_create(&dummy, NULL, writer_thread, m);
    }
    ...
}

void *writer_thread(void *arg) {
    int id = *((int*) arg);
    ...
    free((int*) arg);
    ...
    printf("thread %d wrote - %s", id, buffer);
    ...
}

void *reader_thread(void *arg) {
    int id = *((int*) arg);
    ...
    free((int*) arg);
    ...
    process_data(removed_node->data, removed_node->length, id);
    ...
}

void process_data(char *buffer, int bufferSizeInBytes, int id){
    ...
        printf("thread %i - ", id);
...
        printf("error in process data - %i\n", id);
...
}

在前面的更改之后,valgrind 下的执行由于其他问题而出现错误:

pi@raspberrypi:/tmp $ gcc -g -pedantic -Wextra -Wall t.c -lpthread
pi@raspberrypi:/tmp $ valgrind ./a.out
==3847== Memcheck, a memory error detector
==3847== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==3847== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==3847== Command: ./a.out
==3847== 
==3847== Thread 22:
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x484B20C: strlen (vg_replace_strmem.c:458)
==3847==    by 0x48FD68F: vfprintf (vfprintf.c:1637)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
thread 100 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABCDEthread 103 wrote - 0123456789abcdefghijklmnopqrstuvwxyxthread 1 - 0123456789abcdefghijklmnopqrstuvwxyxABCDE
thread 0 - 0123456789abcdefghijklmnopqrstuvwxyx
thread 102 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABthread 103 wrote - 0123456thread 2 - 012
thread 101 wrote - 0123456789abcdefghithread 105 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFGHIJKLMthread 3 - 0123456
thread 109 wrote - 0123456789abcdefghijklmnopqrstuvwxyxAthread 3 - 0123456789abcdefghi
==3847== Invalid read of size 1
==3847==    at 0x484B1EC: strlen (vg_replace_strmem.c:458)
==3847==    by 0x48FD68F: vfprintf (vfprintf.c:1637)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847==  Address 0x49fcd18 is 0 bytes inside a block of size 50 free'd
==3847==    at 0x4848B8C: free (vg_replace_malloc.c:530)
==3847==  Block was alloc'd at
==3847==    at 0x4847568: malloc (vg_replace_malloc.c:299)
==3847== 
thread 107 wrote - 0123456789abcdefgthread 100 wrote - thread 100 wrote - 0123456789athread 101 wrote - 0123456789abcthread 7 - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFGHIJKLM
thread 7 - 0123456789abcdefghijklmnopqrstuvwxyxAB
thread 9 - 0123456789abc
thread 9 - 0123456789abcdefghijklmnopqrstuvwxyxA
thread 108 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABthread 105 wrote - 01thread 8 - 0123456789ab
thread 108 wrote - 0123456789abcdefghithread 12 - 0123
thread 103 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFGthread 101 wrote - 0123456789abcdefghijklmnopqrstuvthread 103 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFGthread 101 wrote - 0123456789abcdefghijklmnopqrstuvthread 14 - 0123456789a
thread 101 wrote - 0123456789abcdefghijklthread 101 wrote - 0123456789abcdefghijklmnopqrstuvwxyxthread 101 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFGHthread 101 wrote - 0123456789abcdefghijklmnopqthread 101 wrote - 0123456789abcdefghijklmnopqrstuthread 101 wrote - 0123thread 102 wrote - thread 2 - 0123456789abcdefghijklmnopqrstuv
thread 2 - 0123456789abcdefghijklmnopqr
thread 2 - 0123456789abcdefghi
thread 2 - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFG
thread 100 wrote - 0123456789abcthread 100 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFGHIJKthread 4 - 0123456789abcdefghijkl
thread 109 wrote - thread 13 - 0123456789abcdefghijklmnopqrstuvwxyxAB
thread 108 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFGHIthread 19 - 0123456789abc
thread 101 wrote - 0thread 18 - 01
thread 108 wrote - 0123456thread 108 wrote - 01thread 108 wrote - 0123456789abcdefghijthread 10 - 0123456789abcdefghijklmnopq
thread 8 - 0123456789abcdefghijklmnopqrstu
==3847== Thread 30:
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x484B1F4: strlen (vg_replace_strmem.c:458)
==3847==    by 0x48FD68F: vfprintf (vfprintf.c:1637)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x48FBEEC: vfprintf (vfprintf.c:1637)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x48FBF0C: vfprintf (vfprintf.c:1637)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x49245B8: _IO_file_xsputn@@GLIBC_2.4 (fileops.c:1294)
==3847==    by 0x48FBF7B: vfprintf (vfprintf.c:1637)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x48FBF80: vfprintf (vfprintf.c:1637)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x48FBF90: vfprintf (vfprintf.c:1637)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x48FBE24: vfprintf (vfprintf.c:1668)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
==3847== Conditional jump or move depends on uninitialised value(s)
==3847==    at 0x48FBE6C: vfprintf (vfprintf.c:1668)
==3847==    by 0x4902ADF: printf (printf.c:33)
==3847==    by 0x10B83: writer_thread (t.c:144)
==3847==    by 0x4898FC3: start_thread (pthread_create.c:458)
==3847==    by 0x498D037: ??? (clone.S:76)
==3847== 
thread 108 wrote - thread 108 wrote - 0123456789abcdefghijkthread 12 - 0123456789abcdefghijklmnopqrstuvwxyxABCDEFGHIJKL
thread 108 wrote - 0123456789abcdefghijthread 106 wrote - thread 5 - 0123456789abcdefg
thread 106 wrote - thread 106 wrote - 0123456789abcdefghijklmnopqrstuvwxyxABCDthread 16 - 0123456789abcdefghijklmnopqrstuvwx
thread 16 - 0123456789abcdefghijklmnopqrstuv
==3847== 
==3847== HEAP SUMMARY:
==3847==     in use at exit: 5,444 bytes in 74 blocks
==3847==   total heap usage: 155 allocs, 81 frees, 8,138 bytes allocated
==3847== 
==3847== LEAK SUMMARY:
==3847==    definitely lost: 0 bytes in 0 blocks
==3847==    indirectly lost: 0 bytes in 0 blocks
==3847==      possibly lost: 4,080 bytes in 30 blocks
==3847==    still reachable: 1,364 bytes in 44 blocks
==3847==         suppressed: 0 bytes in 0 blocks
==3847== Rerun with --leak-check=full to see details of leaked memory
==3847== 
==3847== For counts of detected and suppressed errors, rerun with: -v
==3847== Use --track-origins=yes to see where uninitialised values come from
==3847== ERROR SUMMARY: 51 errors from 10 contexts (suppressed: 6 from 3)

【讨论】:

  • 感谢您的回答!!它在一定程度上解决了这个问题。我用我遇到的问题更新了我的代码。你是对的:usleep(100) 我把它改成了sleep(100),它工作正常。但是你能检查一下我刚刚编辑的代码,以确定它现在是否正常吗?
  • @arsalunic612 您的新版本代码并不能解决所有问题,我在 valgrind 下执行编辑了我的答案,您可以在其中看到很多随机错误(取决于关于线程并发)
  • 我将尝试使用您提出的解决方案来获得线程标识符。如果需要我去任何地方,我会告诉你的。
  • GCC 的许多端口都需要-pthread(这意味着-lpthread),但是设置了额外的编译器开关,定义了多(p)线程应用程序正常工作所必需的任何东西。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2021-04-09
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多