Parallel.Foreach SQL 查询有时会导致连接

Question

我需要加快在我的应用程序中执行 12 个查询。我从常规 foreach 切换到 Parallel.ForEach。但有时我会收到一条错误消息，提示“ExecuteReader 需要一个打开且可用的连接连接的当前状态为正在连接。”据我了解，由于 12 个查询中的许多查询都使用相同的 InitialCatalog，因此这 12 个查询中并没有真正的新连接，这可能是问题所在？我怎样才能解决这个问题？ “sql”是“Sql”类型的列表——一个类只是一个字符串名称、字符串连接a和一个查询列表。代码如下：

 /// <summary>
    /// Connects to SQL, performs all queries and stores results in a list of DataTables
    /// </summary>
    /// <returns>List of data tables for each query in the config file</returns>
    public List<DataTable> GetAllData()
    {
        Stopwatch sw = new Stopwatch();
        sw.Start();
        List<DataTable> data = new List<DataTable>();

         List<Sql> sql=new List<Sql>();

        Sql one = new Sql();
         one.connection = "Data Source=XXX-SQL1;Initial Catalog=XXXDB;Integrated Security=True";
         one.name = "Col1";
         one.queries.Add("SELECT Name FROM [Reports]");
         one.queries.Add("SELECT Other FROM [Reports2]");
         sql.Add(one);

        Sql two = new Sql();
         two.connection = "Data Source=XXX-SQL1;Initial Catalog=XXXDB;Integrated Security=True";
         two.name = "Col2";
         two.queries.Add("SELECT AlternateName FROM [Reports1]");
         sql.Add(two);

         Sql three = new Sql();
         three.connection = "Data Source=YYY-SQL2;Initial Catalog=YYYDB;Integrated Security=True";
         three.name = "Col3";
         three.queries.Add("SELECT Frequency FROM Times");
         sql.Add(three);


        try
        {
            // ParallelOptions options = new ParallelOptions();
            //options.MaxDegreeOfParallelism = 3;
            // Parallel.ForEach(sql, options, s =>
            Parallel.ForEach(sql, s =>
            //foreach (Sql s in sql)
            {
                foreach (string q in s.queries)
                {
                    using (connection = new SqlConnection(s.connection))
                    {
                        connection.Open();
                        DataTable dt = new DataTable();
                        dt.TableName = s.name;
                        command = new SqlCommand(q, connection);
                        SqlDataAdapter adapter = new SqlDataAdapter();
                        adapter.SelectCommand = command;
                        adapter.Fill(dt);
                        //adapter.Dispose();

                        lock (data)
                        {
                            data.Add(dt);
                        }
                    }
                }
            }
            );
        }
        catch (Exception ex)
        {
            MessageBox.Show(ex.ToString(), "GetAllData error");
        }

        sw.Stop();
        MessageBox.Show(sw.Elapsed.ToString());

        return data;
    }

这是我创建的你需要的 Sql 类：

/// <summary>
/// Class defines a SQL connection and its respective queries
/// </summary>
public class Sql
{
    /// <summary>
    /// Name of the connection/query
    /// </summary>
    public string name { get; set; }
    /// <summary>
    /// SQL Connection string
    /// </summary>
    public string connection { get; set; }
    /// <summary>
    /// List of SQL queries for a connection
    /// </summary>
    public List<string> queries = new List<string>();
}

Answer 1

我会重构你的业务逻辑（连接到数据库）。

public class SqlOperation
{
    public SqlOperation()
    {
        Queries = new List<string>();
    }

    public string TableName { get; set; }
    public string ConnectionString { get; set; }
    public List<string> Queries { get; set; }
}

public static List<DataTable> GetAllData(IEnumerable<SqlOperation> sql)
{
    var taskArray =
        sql.SelectMany(s =>
            s.Queries
             .Select(query =>
                Task.Run(() => //Task.Factory.StartNew for .NET 4.0
                    ExecuteQuery(s.ConnectionString, s.TableName, query))))
            .ToArray();

    try
    {
        Task.WaitAll(taskArray);
    }
    catch(AggregateException e)
    {
        MessageBox.Show(e.ToString(), "GetAllData error");
    }

    return taskArray.Where(t => !t.IsFaulted).Select(t => t.Result).ToList();
}

public static DataTable ExecuteQuery(string connectionString, string tableName, string query)
{
    DataTable dataTable = null;

    using (var connection = new SqlConnection(connectionString))
    {
        dataTable = new DataTable();
        dataTable.TableName = tableName;
        using(var command = new SqlCommand(query, connection))
        {
            connection.Open();

            using(var adapter = new SqlDataAdapter())
            {
                adapter.SelectCommand = command;
                adapter.Fill(dataTable);
            }
        }
    }

     return dataTable;
}

Answer 2

Ado.Net 有一个非常聪明的连接池，所以通常你应该只打开连接和关闭每个命令的连接，让池处理它们是否真的被打开或关闭。

所以每个命令一个连接：

  Parallel.ForEach(sql, s=>
            //foreach (Sql s in sql)
            {
                foreach (string q in s.queries)
                {
                    using (connection = new SqlConnection(s.connection))
                    {
                        connection.Open();
                        DataTable dt = new DataTable();
                        dt.TableName = s.name;
                        command = new SqlCommand(q, connection);
                        SqlDataAdapter adapter = new SqlDataAdapter();
                        adapter.SelectCommand = command;
                        adapter.Fill(dt);
                        //adapter.Dispose();

                        lock(data){
                            data.Add(dt);
                        }
                    }
                }
            }

Answer 3

您还可以在连接字符串中使用 MultipleActiveResultSets=true; 来支持多个阅读器