【问题标题】:Moving array of smaller structs into array of larger structs in C在C中将较小结构的数组移动到较大的结构数组中
【发布时间】:2014-12-09 10:10:13
【问题描述】:

今天我正在研究将较小结构数组直接移动到较大结构数组中的问题 (arrayNew)(本质上是升级较小结构以存储更多信息)。较小的结构需要在一次 读取操作中从 HDD 读取到新的“升级”较大结构的数组中,将调用一个函数来执行“升级”。此外,从硬盘驱动器读取的结构中的所有新字段都将设置为'0'。 我尝试过的其他更简单的解决方案是:

  • 创建旧结构的本地数组 (arrayOld),将结构从 HDD 加载到其中,然后简单地遍历新结构的空数组 (arrayNew) 并手动移动每个结构内容从arrayOldarrayNew。 (例如arrayNew[i].x = arrayOld[i].x;) 这样做的问题是,在我的情况下,我正在使用的数组非常大并且对于堆栈来说太大(每个数组大约 1mb)导致在调用升级函数时出现分段错误。

  • 另一个可行的解决方案是创建旧结构的动态数组 (arrayDy) 并将旧结构加载到 arrayDy,然后再次手动将每个结构内容从 arrayDy 移动到 arrayNew。 (例如 arrayNew[i].y = arrayDy[i].y; )这解决了堆栈内存不足的问题。

实施第二个解决方案后。我决定试验并开发一种解决方案,它不使用动态分配的内存,并在一次读取操作中将旧结构数组从 HHD 直接加载到更大的结构数组arrayNew 中,并在内存中操作arrayNew 的内容以填补由于数组更大而存在的缺失值。

我将在下面以我实施的缩小版本发布我的解决方案,在我的示例中使用以下结构:

typedef struct INNER_STRUCT_ {

    int i_item1;
    int i_item2;
    char i_item3;

} INNER_STRUCT;

typedef struct SMALL_STRUCT_ {

    int item1;
    char item2;
    INNER_STRUCT item3;

} SMALL_STRUCT;

typedef struct BIG_STRUCT_ {

    int item1;
    char item2;
    INNER_STRUCT item3;
    INNER_STRUCT item4;

} BIG_STRUCT;

【问题讨论】:

  • 您是通过 1 次调用还是 100 次调用从磁盘加载这 100 个结构?
  • 如果您的堆栈空间不足,请检查变量的对齐方式并首先分配最严格的。您的空间计算假设您正在打包结构
  • 如果堆栈空间有限,为什么不将数组存储在其他地方(例如使用动态内存分配)?
  • 纯粹在 C 中,您必须分别执行每个复制操作(即,迭代 100 次)。根据您的处理器,一些(如 DSP)专门为此目的指定了操作。但这当然不是 C 语言标准的一部分。
  • 从磁盘加载struct的函数调用一次,加载不成功会报错。我目前正在研究使用动态内存的解决方案,但正在考虑是否可以使用其他选项。 @TimChild 我需要在某处阅读变量对齐的信息吗?谢谢

标签: c data-structures struct casting padding


【解决方案1】:

是的,这是可能的 - 您可以为此使用 union。 C99 标准提供了可用于实现您的要求的特殊保证:

6.5.2.3-5:为了简化联合的使用,提供了一项特殊保证:如果联合包含多个共享相同初始序列的结构(见下文),并且联合对象当前包含其中之一结构,允许在任何可见联合类型声明的地方检查它们中任何一个的公共初始部分。

您的 structA_structB_ 确实共享一个共同的初始序列,因此创建一个 union 并通过它访问结构就可以了:

union {
    structA a;
    structB b;
} u;
memset(&u.b, 0, sizeof(structB)); // Zero out the bigger structB
loadFromHdd(&u.a); // Load structA part into the union
// At this point, u.b is valid, with its structA portion filled in
// and structB part zeroed out.

请注意,您不能对数组执行此操作(当然,除非您创建了一个 unions 数组)。每个structA 都需要单独加载到union 中,然后可以从中读取为structB

【讨论】:

  • 如果 HHD 上的数据被“压缩”为较小结构的数组,并且 RAM 中的数据需要“扩展”以便每个较小的结构后跟 20字节的附加数据?
  • @barakmanos 我添加了一条注释,提到这不可能与数组有关。我假设 OP 可以一一读取structs。
  • 好的,但我很确定 OP 已经知道如何做到这一点,不管有没有 union(无论如何都会产生相同的运行时性能)。
【解决方案2】:

我提出并用作解决方案的方法基本上是将 HDD 的较小结构(在本例中为文件)加载到新的较大结构的数组中,然后重新排列内存块,以便可以正确访问每个字段。说明这一点的代码如下,是mcve

#include <stdio.h>
#include <string.h>

typedef struct INNER_STRUCT_ {

    int i_item1;
    int i_item2;
    char i_item3;

} INNER_STRUCT;

typedef struct SMALL_STRUCT_ {

    int item1;
    char item2;
    INNER_STRUCT item3;

} SMALL_STRUCT;

typedef struct BIG_STRUCT_ {

    int item1;
    char item2;
    INNER_STRUCT item3;
    INNER_STRUCT item4;
    /* 
    Note that the big struct is exactly the same as the small 
    struct with one extra field - Key to this method working 
    is the fact that the extension to the struct is appended
    at the end, in an array of the structs will be placed one 
    after the other in memory with no gaps*/

} BIG_STRUCT;

void printSmallStruct (SMALL_STRUCT *inStruct, int count) {
    // Print everything inside given small struct
    printf("\n\n Small struct %d, item1: %d \n",count,inStruct->item1);
    printf(" Small struct %d, item2: %c \n",count,inStruct->item2);
    printf(" Small struct %d, item3.i_item1: %d \n",count,inStruct->item3.i_item1);
    printf(" Small struct %d, item3.i_item2: %d \n",count,inStruct->item3.i_item2);
    printf(" Small struct %d, item3.i_item3: %c \n",count,inStruct->item3.i_item3);
}

void printBigStruct (BIG_STRUCT *inStruct, int count) {
    // Print everything inside given big struct
    printf("\n\n Big struct %d, item1: %d \n",count,inStruct->item1);
    printf(" Big struct %d, item2: %c \n",count,inStruct->item2);
    printf(" Big struct %d, item3.i_item1: %d \n",count,inStruct->item3.i_item1);
    printf(" Big struct %d, item3.i_item2: %d \n",count,inStruct->item3.i_item2);
    printf(" Big struct %d, item3.i_item3: %c \n",count,inStruct->item3.i_item3);
    printf(" Big struct %d, item4.i_item1: %d \n",count,inStruct->item4.i_item1);
    printf(" Big struct %d, item4.i_item1: %d \n",count,inStruct->item4.i_item2);
    printf(" Big struct %d, item4.i_item1: %c \n",count,inStruct->item4.i_item3);
}

int main() {


    SMALL_STRUCT smallStructArray[5];       // The array of small structs that we will write to a file then read

    BIG_STRUCT   loadedBigStructArray[5];   // The large array of structs that we will read the data from the file into

    int i;  // Counter that we will use

    FILE *pfile;    // pointer to our file stream

    void *secondary_ptr;    // void pointer that we will use to 'chop' memory into the size we want

    /* Fill the array of structs (smallStructArray) */
    for (i = 0; i < 5; i++) {
    /* We fill each field with different data do we can ID that the right data is in the right fields */
        smallStructArray[i].item1 = 111;
        smallStructArray[i].item2 = 'S';
        INNER_STRUCT*    temp = &smallStructArray[i].item3;
        temp->i_item1 = 777;
        temp->i_item2 = 999;
        temp->i_item3 = 'I';
    }


    /* Write the contents of smallStructArray to binary file then display it */
    pfile = fopen("test.dat","wb");
    if (pfile!=NULL){
    for (i = 0; i < 5; i++) {
        fwrite(&smallStructArray[i],sizeof(SMALL_STRUCT),1,pfile);
    }
    fclose(pfile);
    }
    else{
    printf("Unable to open file!");
    return 1;
    }

    for (i = 0; i < 5; i++) {
         printSmallStruct(&smallStructArray[i],i);
    }

    /* Clear array of big structs using memset  */
    memset(&loadedBigStructArray[0],0,sizeof(loadedBigStructArray));

    /* Here we read from the smallStructArray that was aved to file into the  loadedBigStructArray */
    pfile = fopen("test.dat","rb");
    if (pfile !=NULL){
    /*
    He we pass fread the following:     size_t fread(void *args1, size_t args2, size_t args3, FILE *args4)
    args1   - a pointer to the beginning of a block of memory, in our case the beginning of the 
          array loadedBigStructArray.

    args2   - the size of the ammout of bytes we wish to read, in our case the size of a SMALL_STRUCT, 
          the size one of the elments in the array saved to the file.

    args3   - the ammount of elements to read, in our case five (which is the number of elements the 
          array saved to the file has. 

    args4   - a pointer to a FILE that specifies our input stream.

    Essentially what fread will do here is read a block of bytes the size of the array we saved to 
    the file (smallStructArray) into the array in memory loadedBigStructArray from the 
    beggining of loadedBigStructArray. Fig 1 illustrates what this will look like in memory.
    */
    fread(&loadedBigStructArray,sizeof(SMALL_STRUCT),5,pfile);
    fclose(pfile);
    }
    else{
    printf("Unable to open file!");
    return 1;
    }
    /* 
    Due to the way the array on the file has been read into the array in memory, if we try 
    to access the data in loadedBigStructArray only the first 5 values will be valid, due to 
    the  memory not being in the order we want. We need to re-arrange the data in loadedBigStructArray
    */

    /* 
    Here we use a void pointer to point to  the beggining of the loadedBigStructArray.
    we will use this pointer to 'chop' the data loadedBigStructArray into SMALL_STRUCT 
    sized 'chunks' we can read from.

    Due to the way pointers and arrays work in C we can cast the void pointer to any type we want
    and get a chunk of memory that size begginnig from the pointer and its off set.
    E.g. : int temp = ((int *)void_ptr)[i];  
    This example above will give us an integer 'temp' that was taken from memory beggining from position
    void_ptr in memory and its offset i. ((int *)void_ptr) casts the pointer to type int and [i] dereferances
    the pointer to location i.
    */
    secondary_ptr = &loadedBigStructArray;

    /* 
    Not we are going through the array backwards so that we can rearange the data with out overwriting 
    data in a location that has data which we havent moved yet. As the bottom end of the loadedBigStructArray
    is essentially empty we can shift data down that way.
    */
    for (i = 5; i > -1; i=i-1) {


    SMALL_STRUCT temp = ((SMALL_STRUCT *)secondary_ptr)[i]; // dereference pointer to SMALL_STRUCT [i] inside loadedBigStructArray call it 'temp'

    /*
    Now that we have dereferenced a pointer a given SMALL_STRUCT inside loadedBigStructArray called 'temp'
    we can use temp to move the data inside temp to its corresponding position in loadedBigStructArray 
    which rearragnes the data.
    */
    loadedBigStructArray[i].item1 = temp.item1;
    loadedBigStructArray[i].item2 = temp.item2;
    loadedBigStructArray[i].item3.i_item1 = temp.item3.i_item1;
    loadedBigStructArray[i].item3.i_item2 = temp.item3.i_item2;
    loadedBigStructArray[i].item3.i_item3 = temp.item3.i_item3;

    /* We then fill the new field to be blank */
    loadedBigStructArray[i].item4.i_item1 = 0;
    loadedBigStructArray[i].item4.i_item2 = 0;
    loadedBigStructArray[i].item4.i_item3 = '0';
    }

    /* Print our new structures */
    for (i = 0; i < 5; i++) {
         printBigStruct(&loadedBigStructArray[i],i);
    }

    return 0;
}

技术可视化:

当 fread 对保存在磁盘上的数组进行单次读取操作时,由于它较小,它将占用内存中数组的第一部分,但“底部”部分可以是任何东西,如果我们尝试使用我们对数据的当前句柄访问新数组中的数据,我们将获得不准确的信息或内存损坏。我们必须重新排列这些数据,然后才能对数组中的结构使用任何句柄。

【讨论】:

  • 如有任何不妥之处请发表评论
猜你喜欢
  • 2015-11-12
  • 1970-01-01
  • 2016-03-06
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多