【问题标题】:Efficiency in SQL querySQL查询的效率
【发布时间】:2011-10-12 10:51:24
【问题描述】:

我为我的蛋糕应用程序创建了一个搜索功能。它由多个选择框构建而成,您可以在其中选择数据,然后循环选择选项并将它们实现为 SQL 语法。

这个函数基本上是这样的:

$selectedFilters = $this->data;
        $selectSQL =    'SELECT
                        agencies.agency, agencies.website_url, agencies.status, agencies.size, agencies.id, OfficeData.id, ContactData.name, ContactData.surname, ContactData.job_title, ContactData.email, 
                        ContactData.mobile, OfficeCountryData.country
                        FROM agencies
                        LEFT JOIN (SELECT agencies_industries.agency_id, agencies_industries.industry_id FROM agencies_industries) AS IndustryData ON agencies.id = IndustryData.agency_id
                        LEFT JOIN (SELECT agencies_professions.agency_id, agencies_professions.profession_id FROM agencies_professions) AS ProfessionData ON agencies.id = ProfessionData.agency_id
                        LEFT JOIN (SELECT agencies_sectors.agency_id, agencies_sectors.sector_id FROM agencies_sectors) AS SectorData ON agencies.id = SectorData.agency_id
                        LEFT JOIN (SELECT agencies_seniorities.agency_id, agencies_seniorities.seniority_id FROM agencies_seniorities) AS SeniorityData ON agencies.id = SeniorityData.agency_id
                        LEFT JOIN (SELECT agencies_zones.agency_id, agencies_zones.zone_id FROM agencies_zones) AS ZonesData ON agencies.id = ZonesData.agency_id
                        LEFT JOIN (SELECT agencies_countries.agency_id, agencies_countries.country_id FROM agencies_countries) AS CountryData ON agencies.id = CountryData.agency_id
                        LEFT JOIN (SELECT agencies_regions.agency_id, agencies_regions.region_id FROM agencies_regions) AS RegionData ON agencies.id = RegionData.agency_id
                        LEFT JOIN (SELECT agencies_cities.agency_id, agencies_cities.city_id FROM agencies_cities) AS CityData ON agencies.id = CityData.agency_id
                        LEFT JOIN (SELECT agencies_specialisms.agency_id, agencies_specialisms.specialism_id FROM agencies_specialisms) AS SpecialismData ON agencies.id = SpecialismData.agency_id
                        LEFT JOIN (SELECT offices.id, offices.agency_id, offices.hq FROM offices WHERE offices.hq = "1") AS OfficeData ON agencies.id = OfficeData.agency_id
                        LEFT JOIN (SELECT countries.id, countries.country FROM countries) AS OfficeCountryData ON OfficeData.hq = OfficeCountryData.id
                        LEFT JOIN (SELECT contacts.name, contacts.surname, contacts.agency_id, contacts.job_title, contacts.email, contacts.mobile FROM contacts) AS ContactData ON agencies.id = ContactData.agency_id
                        ';
        $whereSQL = ' WHERE 1 = 1 ';
            foreach($selectedFilters as $key)
                foreach($key as $name=>$value){
                    if(is_array($key))
                        foreach($key as $key=>$value){
                            $i = 0;
                            $connector = 'AND';
                            if(is_array($value)){
                                foreach($value as $value){
                                    if($i > 0)
                                        $connector = 'OR';
                                    $i++;
                                    switch($key){
                                        case 'Profession': $whereSQL .= $connector.' ProfessionData.profession_id = ' . $value . ' ';
                                        break;
                                        case 'Specialism': $whereSQL .= $connector.' SpecialismData.specialism_id = ' . $value . ' ';
                                        break;
                                        case 'SubSpecialism': $whereSQL .= ''; //$whereSQL .= $connector.' SubData.sub_specialism_id = ' . $value . ' ';
                                        break;
                                        case 'Seniority': $whereSQL .= $connector.' SeniorityData.seniority_id = ' . $value . ' ';
                                        break;
                                        case 'Industry': $whereSQL .= $connector.' IndustryData.industry_id = ' . $value . ' ';
                                        break;
                                        case 'Zone': $whereSQL .= $connector.' ZonesData.zone_id = ' . $value . ' ';
                                        break;
                                        case 'Country': $whereSQL .= $connector.' CountryData.country_id = ' . $value . ' ';
                                        break;
                                        case 'Region': $whereSQL .= $connector.' RegionData.region_id = ' . $value . ' ';
                                        break;
                                        case 'City': $whereSQL .= $connector.' CityData.city_id = ' . $value . ' ';
                                        break;
                                        case 'Sector': $whereSQL .= $connector.' SectorData.sector_id = ' . $value . ' ';
                                        break;
                                        case 'status': $whereSQL .= $connector.' agencies.status = "' . $value . '" ';
                                        break;
                                        case 'size': $whereSQL .= $connector.' agencies.size = "' . $value . '" ';
                                        break;
                                    }
                                }
                            }
                            else
                                if(!isBlank($value) && $key != 'Search')
                                    $whereSQL .= $connector.' agencies.'.$key.' = "'.$value.'" ';
                        }
                }
        $groupBySQL = 'GROUP BY agencies.id ORDER BY agencies.id ASC';
        $resultAgencies = $this->Agency->query($selectSQL . $whereSQL . $groupBySQL);
        $this->set(compact('resultAgencies'));

我在搜索时遇到的问题是它的运行速度非常慢。这是因为使用了太多的LEFT JOIN 命令。每个LEFT JOIN 从不同的表中选择数据并将它们全部收集起来创建另一个表。然后显示数据。

我需要有人给我一个提示,如何在不使用这么多 LEFT JOINs 的情况下做到这一点。

干杯。

【问题讨论】:

  • 杀死查询的不是 JOINS,而是子查询的数量!为什么加入子查询而不是表?

标签: php mysql


【解决方案1】:

试试这个:

$selectSQL =    'SELECT
                        agencies.agency, agencies.website_url, agencies.status, agencies.size, agencies.id, OfficeData.id, ContactData.name, ContactData.surname, ContactData.job_title, ContactData.email, 
                        ContactData.mobile, OfficeCountryData.country
                        FROM agencies
                        LEFT JOIN agencies_industries AS IndustryData ON agencies.id = IndustryData.agency_id
                        LEFT JOIN  agencies_professions AS ProfessionData ON agencies.id = ProfessionData.agency_id
                        LEFT JOIN  agencies_sectors AS SectorData ON agencies.id = SectorData.agency_id
                        LEFT JOIN  agencies_seniorities AS SeniorityData ON agencies.id = SeniorityData.agency_id
                        LEFT JOIN  agencies_zones AS ZonesData ON agencies.id = ZonesData.agency_id
                        LEFT JOIN agencies_countries AS CountryData ON agencies.id = CountryData.agency_id
                        LEFT JOIN  agencies_regions AS RegionData ON agencies.id = RegionData.agency_id
                        LEFT JOIN  agencies_cities AS CityData ON agencies.id = CityData.agency_id
                        LEFT JOIN  agencies_specialism AS SpecialismData ON agencies.id = SpecialismData.agency_id
                        LEFT JOIN  offices  AS OfficeData ON (agencies.id = OfficeData.agency_id AND OfficeData.hq = "1")
                        LEFT JOIN countries AS OfficeCountryData ON OfficeData.hq = OfficeCountryData.id
                        LEFT JOIN  contacts AS ContactData ON agencies.id = ContactData.agency_id
                        ';

但即便如此,它也可能会很慢,因为您加入了太多的表。但是,如果不了解您的数据以及您将返回的行数,就很难说清楚。如果您只返回几行,您可能希望将一些 JOINS 移动到子查询(如国家/地区)。或者,您可以在单独的查询中添加该信息。

编辑: 在不知道您的数据和数据库结构的情况下很难分辨。影响查询速度的因素有很多。首先重写您的查询,以便查询中不使用未用于您选择的表(即 WHERE)或您要显示的字段。因此,如果您不进行选择 (emtpy $selectedFilters),则您不必包括行业、专业、部门、资历等表。:

$selectedFilters = $this->data;
        $selectSQL =    'SELECT
                        agencies.agency, agencies.website_url, agencies.status, agencies.size, agencies.id, OfficeData.id, ContactData.name, ContactData.surname, ContactData.job_title, ContactData.email, 
                        ContactData.mobile, OfficeCountryData.country
                        FROM agencies';


        $sql2='                LEFT JOIN  offices  AS OfficeData ON (agencies.id = OfficeData.agency_id AND OfficeData.hq = "1")
                        LEFT JOIN countries AS OfficeCountryData ON OfficeData.hq = OfficeCountryData.id
                        LEFT JOIN  contacts AS ContactData ON agencies.id = ContactData.agency_id
                        ';

        $whereSQL = ' WHERE 1 = 1 ';
            foreach($selectedFilters as $key)
                foreach($key as $name=>$value){
                    if(is_array($key))
                        foreach($key as $key=>$value){
                            $i = 0;
                            $connector = 'AND';
                            if(is_array($value)){
                                foreach($value as $value){
                                    if($i > 0)
                                        $connector = 'OR';
                                    $i++;
                                    switch($key){
                                        case 'Profession': $whereSQL .= $connector.' ProfessionData.profession_id = ' . $value . ' ';
                                        $sql2.=' LEFT JOIN  agencies_professions AS ProfessionData ON agencies.id = ProfessionData.agency_id ';
                                        break;
                                        case 'Specialism': $whereSQL .= $connector.' SpecialismData.specialism_id = ' . $value . ' ';
                                        $sql2.=' LEFT JOIN  agencies_specialism AS SpecialismData ON agencies.id = SpecialismData.agency_id ';
                                        break;
                                        case 'SubSpecialism': $whereSQL .= ''; //$whereSQL .= $connector.' SubData.sub_specialism_id = ' . $value . ' ';
                                        break;
                                        case 'Seniority': $whereSQL .= $connector.' SeniorityData.seniority_id = ' . $value . ' ';
                                        $sql2.=' LEFT JOIN  agencies_seniorities AS SeniorityData ON agencies.id = SeniorityData.agency_id ';
                                        break;
                                        case 'Industry': $whereSQL .= $connector.' IndustryData.industry_id = ' . $value . ' ';
                                        $sql2=' LEFT JOIN agencies_industries AS IndustryData ON agencies.id = IndustryData.agency_id ';
                                        break;
                                        case 'Zone': $whereSQL .= $connector.' ZonesData.zone_id = ' . $value . ' ';
                                        $sql2.=' LEFT JOIN  agencies_zones AS ZonesData ON agencies.id = ZonesData.agency_id ';
                                        break;
                                        case 'Country': $whereSQL .= $connector.' CountryData.country_id = ' . $value . ' ';
                                        $sql2.=' LEFT JOIN agencies_countries AS CountryData ON agencies.id = CountryData.agency_id ';
                                        break;
                                        case 'Region': $whereSQL .= $connector.' RegionData.region_id = ' . $value . ' ';
                                        $sql2.=' LEFT JOIN  agencies_regions AS RegionData ON agencies.id = RegionData.agency_id ';
                                        break;
                                        case 'City': $whereSQL .= $connector.' CityData.city_id = ' . $value . ' ';
                                        $sql2.=' LEFT JOIN  agencies_cities AS CityData ON agencies.id = CityData.agency_id ';
                                        break;
                                        case 'Sector': $whereSQL .= $connector.' SectorData.sector_id = ' . $value . ' ';
                                        $sql2.='LEFT JOIN  agencies_sectors AS SectorData ON agencies.id = SectorData.agency_id ';
                                        break;
                                        case 'status': $whereSQL .= $connector.' agencies.status = "' . $value . '" ';
                                        break;
                                        case 'size': $whereSQL .= $connector.' agencies.size = "' . $value . '" ';
                                        break;
                                    }
                                }
                            }
                            else
                                if(!isBlank($value) && $key != 'Search')
                                    $whereSQL .= $connector.' agencies.'.$key.' = "'.$value.'" ';
                        }
                }
        $groupBySQL = 'GROUP BY agencies.id ORDER BY agencies.id ASC';
        $resultAgencies = $this->Agency->query($selectSQL . $sql2 . $whereSQL . $groupBySQL);
        $this->set(compact('resultAgencies'));

其次,仔细查看每个表的索引。确保您在 JOINS 中使用的字段上有一个索引。

第三,看你使用的字段类型。如果 SMALLINT 足够大,则不要使用 INT。

最后:规范化很棒,但有时将一些东西结合起来会更好,即使这意味着你有重复的数据。

【讨论】:

  • Unknown column 'offices.hq' in 'on clause'(是的,office 表中有一个列名 hq
  • 将办公室更改为 OfficeData(在上面的代码中也进行了更改)
  • 仍然需要等待一分钟才能得到结果(即使没有选择任何数据)。
  • 对我的回答进行了一些修改。最重要的是,没有一个快速的答案可以加快速度。
  • 感谢您的代码。我会在一小时内检查并留下反馈:) 如果它有效,我会接受:)
【解决方案2】:

您应该使用联接而不是子查询。 你也可能并不总是需要所有的左连接;我可以看到您的 WHERE 语句是动态的,因此作为 switch 语句的一部分,您可以决定需要加入哪些额外的表。

因此,首先只加入您需要列的表;

$selectSQL = "
    SELECT agencies.agency, 
       agencies.website_url, 
       agencies.status, 
       agencies.size, 
       agencies.id, 
       OfficeData.id, 
       ContactData.name, 
       ContactData.surname, 
       ContactData.job_title, 
       ContactData.email,               
       ContactData.mobile, 
       OfficeCountryData.country
  FROM agencies
  LEFT JOIN offices AS OfficeData          ON ( agencies.id = OfficeData.agency_id )
  LEFT JOIN contacts AS ContactData        ON ( agencies.id = ContactData.agency_id )
  LEFT JOIN countries AS OfficeCountryData ON ( OfficeData.hq = OfficeCountryData.id ) "

然后在构建 where 语句时,您可以评估是否需要加入表以使该子句有效。

$whereSQL = 'WHERE OfficeData.hq = "1"';
$joinSQL ='';

# Loop though your filter options and build up the where and joins
foreach(...){
    switch($key){
        case 'Profession': 
            $whereSQL .= $connector.' ProfessionData.profession_id = ' . $value . ' ';
            $joinSQL .= 'LEFT JOIN agencies_professions AS ProfessionData ON (agencies.id = ProfessionData.agency_id)'
        break;
        ....
    }
}

然后构建您的最终查询

$sql = $selectSQL.' '.$joinSQL.' '.$whereSQL;

【讨论】:

  • 在这种情况下,选择所有可能无法正常工作的选项时。
  • 我必须更多地了解可能的选项如何工作才能理解这一点。
  • 它们只是多选/选择框。
【解决方案3】:

学习使用MySQL's EXPLAIN 语法。编辑您的问题,并包含您的 EXPLAIN 计划的输出。

除其他问题外,您在许多未从中选择的表上进行左连接。试试这个。

SELECT agencies.agency, agencies.website_url, agencies.status, agencies.size, 
       agencies.id, 
       OfficeData.id, 
       ContactData.name, ContactData.surname, ContactData.job_title, 
       ContactData.email, ContactData.mobile, OfficeCountryData.country
FROM agencies
LEFT JOIN (SELECT offices.id, offices.agency_id, offices.hq 
           FROM offices 
           WHERE offices.hq = "1") AS OfficeData 
       ON agencies.id = OfficeData.agency_id
LEFT JOIN countries AS OfficeCountryData 
       ON OfficeData.hq = OfficeCountryData.id
LEFT JOIN contacts AS ContactData 
       ON agencies.id = ContactData.agency_id

这对性能有何影响?

可能没有令人信服的理由来识别具有 ID 号的国家、城市和地区;他们以他们的名义携带他们的身份。测试用正确的名称替换 ID 号。 (ID 号总是需要一个连接操作来获取有用的数据;自然键通常会消除连接。)

您评论说,没有不需要的连接的性能很好,switch 语句不应该受到责备。如果这是真的,那么您需要减少连接数。幸运的是,减少连接既简单又直接。

如果您必须“报告全域”,您可以尝试拆分查询并异步提交多个查询。首先返回并显示代理和联系人数据,您将大大提高应用程序的明显速度。 dbms 可以在渲染第一个查询时处理第二个查询。通常表观速度比实际速度更重要。

【讨论】:

  • 我需要加入所有表,因为最终结果是动态的。
  • 没有连接的性能是应该的。运行速度非常快。
  • 当我注释掉 switch 语句和循环时,SQL 语句的加载时间是一样的。
  • @PiotrChabros:当您说“SQL 语句需要相同的加载时间”时,是指 SQL 语句运行快还是运行慢?
  • @PiotrChabros:更新了答案。见最后两段。
【解决方案4】:

在不知道您实际在做什么的情况下,很难判断您的查询是否可以简化。假设您需要来自所有表的信息并且所有 id 都是主键,我将分析 WHERE 子句 - 您是否正确定义了索引?与大型数据库索引产生巨大差异并且可以大大提高性能。

【讨论】:

    【解决方案5】:

    连接很慢,尝试子查询,编写更短的查询。它们可能意味着更多代码,但如果您在查询前后收集时间戳,您会看到巨大的差异。

    【讨论】:

      【解决方案6】:

      加入可能会很慢,但这不是您的问题。一个快速解决方法:删除那些子查询,为什么要创建子查询而不是整个表?它使一切都变慢了。

      第二:确保你用来连接的所有键都标记为索引,它可以让一切变得更快。

      【讨论】:

      • 如何删除这些子查询?
      • 只加入全表,而不是子查询。
      猜你喜欢
      • 1970-01-01
      • 2021-08-19
      • 2015-10-20
      • 1970-01-01
      • 1970-01-01
      • 2013-04-14
      • 1970-01-01
      • 2013-10-21
      相关资源
      最近更新 更多