将数组划分为子序列数组的数组

Question

我有一个字节数组：

byte[] bytes;  // many elements

我需要将它分成 X 元素的字节数组的子序列。例如，x = 4。

如果 bytes.Length 不乘以 X，则将 0 添加到最后一个子序列数组，因此 所有子序列的长度必须为 X。

Linq 可用。

PS：我的尝试

static void Main(string[] args)
{
    List<byte> bytes = new List<byte>() { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 };

    int c = bytes.Count / 4;

    for (int i = 0; i <= c; i+=4)
    {
        int diff = bytes.Count - 4;

        if (diff < 0)
        {

        }
        else
        {
            List<byte> b = bytes.GetRange(i, 4);
        }
    }

    Console.ReadKey();
}

Answer 1

这很可爱：

static class ChunkExtension
{
    public static IEnumerable<T[]> Chunkify<T>(
        this IEnumerable<T> source, int size)
    {
        if (source == null) throw new ArgumentNullException("source");
        if (size < 1) throw new ArgumentOutOfRangeException("size");
        using (var iter = source.GetEnumerator())
        {
            while (iter.MoveNext())
            {
                var chunk = new T[size];
                chunk[0] = iter.Current;
                for (int i = 1; i < size && iter.MoveNext(); i++)
                {
                    chunk[i] = iter.Current;
                }
                yield return chunk;
            }
        }
    }
}
static class Program
{
    static void Main(string[] args)
    {
        List<byte> bytes = new List<byte>() {
              1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 };
        var chunks = bytes.Chunkify(4);
        foreach (byte[] chunk in chunks)
        {
            foreach (byte b in chunk) Console.Write(b.ToString("x2") + " ");
            Console.WriteLine();
        }
    }
}

Answer 2

如果您总是得到source.Length % size != 0，则投票答案有效，尽管它太冗长了。这是一个更好的实现：

public static IEnumerable<T[]> AsChunks<T>(IEnumerable<T> source, int size)
{
    var chunk = new T[size];
    var i = 0;
    foreach(var e in source)
    {
        chunk[i++] = e;
        if (i==size)
        {
            yield return chunk;
            i=0;
        }
    }
    if (i>0) // Anything left?
    {
        Array.Resize(ref chunk, i);
        yield return chunk;
    }
}

void Main()
{
    foreach(var chunk in AsChunks("Hello World!",5))
        Console.WriteLine(new string(chunk));
}

制作：

1. 你好
2. 世界
3. d!

Answer 3

这个怎么样：

var bytes = new List<byte>() { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 };

var result = Chunkify(bytes, 4);

IEnumerable<IEnumerable<T>> Chunkify<T>(IEnumerable<T> source, int chunkSize)
{
    var indicies = 
        Enumerable.Range(0, source.Count()).Where(i => i%chunkSize==0);

    var chunks = 
            indicies
            .Select( i => source.Skip(i).Take(chunkSize) )
            .Select( chunk => new { Chunk=chunk, Count=chunk.Count() } )
            .Select( c => c.Count < chunkSize ? c.Chunk.Concat( Enumerable.Repeat( default(T), chunkSize - c.Count ) ) : c.Chunk )
            ;

    return chunks;      
}

Answer 4

这很好：

    public static IEnumerable<IEnumerable<T>> GetBatches<T>(this IEnumerable<T> items, int batchsize) {
        var itemsCopy = items;
        while (itemsCopy.Any()) {
            yield return itemsCopy.Take(batchsize);
            itemsCopy = itemsCopy.Skip(batchsize);
        }
    }

Answer 5

    const int x = 4;
var bytes = new List<byte>() { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 };
var groups = bytes.Select((b, index) => new { b, index }).GroupBy(obj => obj.index / x).Select(group => new List<byte>(group.Select(i => i.b)));
var last = groups.Last();   
while (last.Count < x)
{
    last.Add(0);
}

Answer 6

你可以试试这个：

    List<byte> bytes = new List<byte>() { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 };

    int partLength = 4;
    int c = bytes.Count / partLength;

    if((c % partLength) != 0)
        c++; // we need one last list which will have to be filled with 0s

    List<List<byte>> allLists = new List<List<byte>>();

    for (int i = 0; i <= c; i++)
        allLists.Add(bytes.Take(partLength).ToList());

    int zerosNeeded = partLength - allLists.Last().Count;

    for (int i = 0; i < zerosNeeded; i++)
        allLists.Last().Add(0);

询问是否有不清楚的地方。

Answer 7

您当然会想采用 Marc Gravell 的解决方案，但我忍不住拼凑了一个纯 LINQ 版本，只是想看看是否可以做到：

static IEnumerable<T[]> LinqChunks<T>(IEnumerable<T> input, int chunkSize)
{
  return input
    //assign chunk numbers to elements by integer division
    .Select((x, index) => new {ChunkNr = index / chunkSize, Value = x})

    //group by chunk number
    .GroupBy(item => item.ChunkNr)

    //convert chunks to arrays, and pad with zeroes if necessary
    .Select(group =>
              {
                var block = group.Select(item => item.Value).ToArray();

                //if block size = chunk size -> return the block
                if (block.Length == chunkSize) return block;

                //if block size < chunk size -> this is the last block, pad it
                var lastBlock= new T[chunkSize];
                for (int i = 0; i < block.Length; i++) lastBlock[i] = block[i];
                return lastBlock;
              });
}

Answer 8

如果有人想要纯粹的功能解决方案 -

static IEnumerable<T[]> Chunkify<T>(IEnumerable<T> input, int size)
{
    return input    
        .Concat(Enumerable.Repeat(default(T), size - input.Count() % size))
        .Select((x, i) => new { Value = x, Chunk = i / size })
        .GroupBy(x => x.Chunk, x => x.Value)
        .Select(x => x.ToArray());
}

Answer 9

/// <summary>
/// Splits an array of bytes into a List<byte[]> holding the
/// chunks of the original array. If the size of the chunks is bigger than
/// the array it will return the original array to be split.
/// </summary>
/// <param name="array">The array to split</param>
/// <param name="size">the size of the chunks</param>
/// <returns></returns>
public static List<byte[]> SplitArray(byte[] array, int size)
{
    List<byte[]> chunksList = new List<byte[]>();
    int skipCounter = 0;

    while (skipCounter < array.Length)
    {
        byte[] chunk = array.Skip(skipCounter).Take(size).ToArray<byte>();
        chunksList.Add(chunk);
        skipCounter += chunk.Length;
    }
    return chunksList;
}

Answer 10

//without LINQ

List<byte> bytes = new List<byte>() { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 };
int x = 4;
int initialLength = bytes.Count;
for (int i = 0; i < (x - (initialLength % x)); i++) // adds enough 0's to list
{
    bytes.Add(0);
}

List<byte[]> byteList= new List<byte[]>(); // contains answers

for (int i=0;i<bytes.Count;i+=4)
{
    byteList.Add(bytes.GetRange(i,4).ToArray());
}   

Answer 11

static IEnumerable<T[]> Chunkify<T>(IEnumerable<T> items, int size)
    {
    var chunk = new List<T>(size);
    foreach (T item in items)
        {
        chunk.Add(item);
        if (chunk.Count == size)
            {
            yield return chunk.ToArray();
            chunk.Clear();
            }
        }
    if (chunk.Count > 0)
        {
        yield return chunk.ToArray();
        }
    }

Answer 12

这个答案更适用于 IEnumerable 的情况，但 question 被标记为与此重复。

有很多解决方案，但没有一个对我来说足够懒惰。这个可以解决问题：

  private class CachedEnumeration<T> : IEnumerable<T>  
  {  
    /// <summary>  
    /// enumerator for the cachedEnumeration class  
    /// </summary>  
    class CachedEnumerator : IEnumerator<T>  
    {  
      private readonly CachedEnumeration<T> m_source;  
      private int m_index;  
      public CachedEnumerator(CachedEnumeration<T> source)  
      {  
        m_source = source;  
        // start at index -1, since an enumerator needs to start with MoveNext before calling current  
        m_index = -1;  
      }  
      public T Current { get { return m_source.m_items[m_index]; } }  
      public void Dispose() { }  
      object System.Collections.IEnumerator.Current { get { return Current; } } 
      public bool MoveNext()  
      {  
        // if we have cached items, just increase our index  
        if (m_source.m_items.Count > m_index + 1)  
        {  
          m_index++;  
          return true;  
        }  
        else 
        {  
          var result = m_source.FetchOne();  
          if (result) m_index++;  
          return result;  
        }  
      }  
      public void Reset()  
      {  
        m_index = -1;  
      }  
    }  
    /// <summary>  
    /// list containing all the items  
    /// </summary>  
    private readonly List<T> m_items;  
    /// <summary>  
    /// callback how to fetch an item  
    /// </summary>  
    private readonly Func<Tuple<bool, T>> m_fetchMethod;  
    private readonly int m_targetSize;  
    public CachedEnumeration(int size, T firstItem, Func<Tuple<bool, T>> fetchMethod)  
    {  
      m_items = new List<T>(size);  
      m_items.Add(firstItem);  
      m_fetchMethod = fetchMethod;  
      m_targetSize = size;  
    }  
    public IEnumerator<T> GetEnumerator()  
    {  
      return new CachedEnumerator(this);  
    }  
    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()  
    {  
      return GetEnumerator();  
    }  
    private bool FetchOne()  
    {  
      if (IsFull) return false;  
      var result = m_fetchMethod();  
      if (result.Item1) m_items.Add(result.Item2);  
      return result.Item1;  
    }  
    /// <summary>  
    /// fetches all items to the cached enumerable  
    /// </summary>  
    public void FetchAll()  
    {  
      while (FetchOne()) { }  
    }  
    /// <summary>  
    /// tells weather the enumeration is already full  
    /// </summary>  
    public bool IsFull { get { return m_targetSize == m_items.Count; } }  
  }  
  /// <summary>  
  /// partitions the <paramref name="source"/> to parts of size <paramref name="size"/>  
  /// </summary>  
  public static IEnumerable<IEnumerable<T>> Partition<T>(this IEnumerable<T> source, int size)  
  {  
    if (source == null) throw new ArgumentNullException("source");  
    if (size < 1) throw new ArgumentException(string.Format("The specified size ({0}) is invalid, it needs to be at least 1.", size), "size");  
    var enumerator = source.GetEnumerator();  
    while (enumerator.MoveNext())  
    {  
      var lastResult = new CachedEnumeration<T>(size, enumerator.Current, () => Tuple.Create(enumerator.MoveNext(), enumerator.Current));  
      yield return lastResult;  
      lastResult.FetchAll();  
    }  
  }  

你可以找到单元测试和源代码here

Answer 13

我一直在解决我的项目中类似的问题，我想出了这个漂亮的解决方案：

dataAsIEnumerable => 您要分批的来源

BatchSize => 你的批次大小

            var batchSize = dataAsIEnumerable.Count / BatchSize;

            // not enought items, create at least one batch
            if (batchSize < 1)
                batchSize = 1;

            var dataAsList = dataAsIEnumerable.ToList();
            var batchAsSplit = new List<List<Model>>();

            for (int j = 0; j < batchSize; j++)
            {
                batchAsSplit.Add(dataAsList.GetRange(j * BatchSize, (dataAsList.Count - (j * BatchSize)) - BatchSize > 0 ? BatchSize : dataAsList.Count - (j * BatchSize)));
            }

            Parallel.ForEach(batchAsSplit, item =>
            {
                lock (MyContent)
                    MyContent.InsertBulk(item);
            });

代码将IEnumerate类型的集合枚举成List，其中有一个GetRange操作，之后生成批次的集合。然后执行批量保存到 MyContent (db)。