【问题标题】:PostgreSQL slightly slow in simple connection and query on PHPPostgreSQL 在 PHP 上的简单连接和查询稍慢
【发布时间】:2012-05-03 20:51:17
【问题描述】:

在 PHP 5.4.1 中,我使用本机 PostgreSQL 驱动程序连接到 SQL 服务器。即使我尝试 PDO 方式也是一样的。这里只有一个例外。如果我使用 pg_pconnect (persistent),它会像我预期的那样非常快。

我从 PHP 到 PostgreSQL 的连接很慢。另外,一些查询也很慢。

例如,请看一下这张表。

id  type    active  serverip
1   1   0   127.0.0.1
2   10  1   127.0.0.1
3   4   1   127.0.0.1
4   7   1   127.0.0.1
5   13  1   127.0.0.1
6   2   1   127.0.0.1
7   11  1   127.0.0.1
8   6   1   127.0.0.1
9   9   1   127.0.0.1
10  3   0   192.168.1.202
11  5   1   127.0.0.1
12  8   1   127.0.0.1
13  12  1   127.0.0.1
14  14  1   127.0.0.1

我的查询是select * from mytable order by id desc。所以,

此表在id 上有索引和主键。此表在 MySQL 5.5.23 和 PostgreSQL 9.1.3 中相同 (1:1)

MySQL 连接这个数据库并在0.0019 sec. 中读取这个表(整个数据)但是PostgreSQL 需要0.0069 sec。这可能看起来像往常一样,但如果 MySQL 可以非常快地做到这一点,为什么不 PostgreSQL 呢?

在我的大桌子上,它需要的时间比这长。在大表中 MySQL 连接并在 0.09601 sec, 中获取 PostgreSQL 连接并在 0.11260 秒 0.14201 sec. 中获取它(现已测试)

如您所见,在这个非常简单的数据库表中,PostgreSQL 简直太慢了。这是我的 postgresql.conf 文件:

    listen_addresses = '*'      # what IP address(es) to listen on;
                    # comma-separated list of addresses;
                    # defaults to 'localhost', '*' = all
                    # (change requires restart)
port = 5432             # (change requires restart)
max_connections = 100           # (change requires restart)
# Note:  Increasing max_connections costs ~400 bytes of shared memory per
# connection slot, plus lock space (see max_locks_per_transaction).
#superuser_reserved_connections = 3 # (change requires restart)
#unix_socket_directory = ''     # (change requires restart)
#unix_socket_group = ''         # (change requires restart)
#unix_socket_permissions = 0777     # begin with 0 to use octal notation
                    # (change requires restart)
#bonjour = off              # advertise server via Bonjour
                    # (change requires restart)
#bonjour_name = ''          # defaults to the computer name
                    # (change requires restart)

# - Security and Authentication -

authentication_timeout = 20s        # 1s-600s
ssl = off               # (change requires restart)
#ssl_ciphers = 'ALL:!ADH:!LOW:!EXP:!MD5:@STRENGTH'  # allowed SSL ciphers
                    # (change requires restart)
#ssl_renegotiation_limit = 512MB    # amount of data between renegotiations
#password_encryption = on
#db_user_namespace = off

# Kerberos and GSSAPI
#krb_server_keyfile = ''
#krb_srvname = 'postgres'       # (Kerberos only)
#krb_caseins_users = off

# - TCP Keepalives -
# see "man 7 tcp" for details

#tcp_keepalives_idle = 0        # TCP_KEEPIDLE, in seconds;
                    # 0 selects the system default
#tcp_keepalives_interval = 0        # TCP_KEEPINTVL, in seconds;
                    # 0 selects the system default
#tcp_keepalives_count = 0       # TCP_KEEPCNT;
                    # 0 selects the system default


#------------------------------------------------------------------------------
# RESOURCE USAGE (except WAL)
#------------------------------------------------------------------------------

# - Memory -

shared_buffers = 1GB            # min 128kB
                    # (change requires restart)
temp_buffers = 16MB         # min 800kB
max_prepared_transactions = 0       # zero disables the feature
                    # (change requires restart)
# Note:  Increasing max_prepared_transactions costs ~600 bytes of shared memory
# per transaction slot, plus lock space (see max_locks_per_transaction).
# It is not advisable to set max_prepared_transactions nonzero unless you
# actively intend to use prepared transactions.
work_mem = 64MB             # min 64kB
maintenance_work_mem = 32MB     # min 1MB
max_stack_depth = 8MB           # min 100kB

# - Kernel Resource Usage -

max_files_per_process = 1000        # min 25
                    # (change requires restart)
shared_preload_libraries = ''       # (change requires restart)

# - Cost-Based Vacuum Delay -

vacuum_cost_delay = 0ms     # 0-100 milliseconds
vacuum_cost_page_hit = 100      # 0-10000 credits
vacuum_cost_page_miss = 100     # 0-10000 credits
vacuum_cost_page_dirty = 200        # 0-10000 credits
vacuum_cost_limit = 2000        # 1-10000 credits

# - Background Writer -

bgwriter_delay = 200ms          # 10-10000ms between rounds
bgwriter_lru_maxpages = 100     # 0-1000 max buffers written/round
bgwriter_lru_multiplier = 2.0       # 0-10.0 multipler on buffers scanned/round

# - Asynchronous Behavior -

#effective_io_concurrency = 1       # 1-1000. 0 disables prefetching


#------------------------------------------------------------------------------
# WRITE AHEAD LOG
#------------------------------------------------------------------------------

# - Settings -

#wal_level = minimal            # minimal, archive, or hot_standby
                    # (change requires restart)
#fsync = on             # turns forced synchronization on or off
#synchronous_commit = on        # synchronization level; on, off, or local
#wal_sync_method = fsync        # the default is the first option
                    # supported by the operating system:
                    #   open_datasync
                    #   fdatasync (default on Linux)
                    #   fsync
                    #   fsync_writethrough
                    #   open_sync
#full_page_writes = on          # recover from partial page writes
#wal_buffers = -1           # min 32kB, -1 sets based on shared_buffers
                    # (change requires restart)
#wal_writer_delay = 200ms       # 1-10000 milliseconds

commit_delay = 0            # range 0-100000, in microseconds
commit_siblings = 5         # range 1-1000

# - Checkpoints -

#checkpoint_segments = 3        # in logfile segments, min 1, 16MB each
#checkpoint_timeout = 5min      # range 30s-1h
#checkpoint_completion_target = 0.5 # checkpoint target duration, 0.0 - 1.0
#checkpoint_warning = 30s       # 0 disables

# - Archiving -

#archive_mode = off     # allows archiving to be done
                # (change requires restart)
#archive_command = ''       # command to use to archive a logfile segment
#archive_timeout = 0        # force a logfile segment switch after this
                # number of seconds; 0 disables


#------------------------------------------------------------------------------
# REPLICATION
#------------------------------------------------------------------------------

# - Master Server -

# These settings are ignored on a standby server

#max_wal_senders = 0        # max number of walsender processes
                # (change requires restart)
#wal_sender_delay = 1s      # walsender cycle time, 1-10000 milliseconds
#wal_keep_segments = 0      # in logfile segments, 16MB each; 0 disables
#vacuum_defer_cleanup_age = 0   # number of xacts by which cleanup is delayed
#replication_timeout = 60s  # in milliseconds; 0 disables
#synchronous_standby_names = '' # standby servers that provide sync rep
                # comma-separated list of application_name
                # from standby(s); '*' = all

# - Standby Servers -

# These settings are ignored on a master server

#hot_standby = off          # "on" allows queries during recovery
                    # (change requires restart)
#max_standby_archive_delay = 30s    # max delay before canceling queries
                    # when reading WAL from archive;
                    # -1 allows indefinite delay
#max_standby_streaming_delay = 30s  # max delay before canceling queries
                    # when reading streaming WAL;
                    # -1 allows indefinite delay
#wal_receiver_status_interval = 10s # send replies at least this often
                    # 0 disables
#hot_standby_feedback = off     # send info from standby to prevent
                    # query conflicts


#------------------------------------------------------------------------------
# QUERY TUNING
#------------------------------------------------------------------------------

# - Planner Method Configuration -

enable_bitmapscan = on
enable_hashagg = on
enable_hashjoin = on
enable_indexscan = on
enable_material = on
enable_mergejoin = on
enable_nestloop = on
enable_seqscan = on
enable_sort = on
enable_tidscan = on

# - Planner Cost Constants -

seq_page_cost = 1.0         # measured on an arbitrary scale
random_page_cost = 4.0          # same scale as above
cpu_tuple_cost = 0.01           # same scale as above
cpu_index_tuple_cost = 0.005        # same scale as above
cpu_operator_cost = 0.0025      # same scale as above
effective_cache_size = 512MB

# - Genetic Query Optimizer -

geqo = on
geqo_threshold = 12
geqo_effort = 5         # range 1-10
geqo_pool_size = 0          # selects default based on effort
geqo_generations = 0            # selects default based on effort
geqo_selection_bias = 2.0       # range 1.5-2.0
geqo_seed = 0.0         # range 0.0-1.0

# - Other Planner Options -

#default_statistics_target = 100    # range 1-10000
#constraint_exclusion = partition   # on, off, or partition
#cursor_tuple_fraction = 0.1        # range 0.0-1.0
#from_collapse_limit = 8
#join_collapse_limit = 8        # 1 disables collapsing of explicit
                    # JOIN clauses


#------------------------------------------------------------------------------
# ERROR REPORTING AND LOGGING
#------------------------------------------------------------------------------

# - Where to Log -

log_destination = 'stderr'      # Valid values are combinations of
                    # stderr, csvlog, syslog, and eventlog,
                    # depending on platform.  csvlog
                    # requires logging_collector to be on.

# This is used when logging to stderr:
logging_collector = on          # Enable capturing of stderr and csvlog
                    # into log files. Required to be on for
                    # csvlogs.
                    # (change requires restart)

# These are only used if logging_collector is on:
log_directory = 'pg_log'        # directory where log files are written,
                    # can be absolute or relative to PGDATA
log_filename = 'postgresql-%a.log'  # log file name pattern,
                    # can include strftime() escapes
log_file_mode = 0777            # creation mode for log files,
                    # begin with 0 to use octal notation
log_truncate_on_rotation = on       # If on, an existing log file with the
                    # same name as the new log file will be
                    # truncated rather than appended to.
                    # But such truncation only occurs on
                    # time-driven rotation, not on restarts
                    # or size-driven rotation.  Default is
                    # off, meaning append to existing files
                    # in all cases.
log_rotation_age = 1d           # Automatic rotation of logfiles will
                    # happen after that time.  0 disables.
log_rotation_size = 0           # Automatic rotation of logfiles will
                    # happen after that much log output.
                    # 0 disables.

# These are relevant when logging to syslog:
#syslog_facility = 'LOCAL0'
#syslog_ident = 'postgres'

#silent_mode = off          # Run server silently.
                    # DO NOT USE without syslog or
                    # logging_collector
                    # (change requires restart)


# - When to Log -

#client_min_messages = notice       # values in order of decreasing detail:
                    #   debug5
                    #   debug4
                    #   debug3
                    #   debug2
                    #   debug1
                    #   log
                    #   notice
                    #   warning
                    #   error

#log_min_messages = warning     # values in order of decreasing detail:
                    #   debug5
                    #   debug4
                    #   debug3
                    #   debug2
                    #   debug1
                    #   info
                    #   notice
                    #   warning
                    #   error
                    #   log
                    #   fatal
                    #   panic

#log_min_error_statement = error    # values in order of decreasing detail:
                    #   debug5
                    #   debug4
                    #   debug3
                    #   debug2
                    #   debug1
                    #   info
                    #   notice
                    #   warning
                    #   error
                    #   log
                    #   fatal
                    #   panic (effectively off)

#log_min_duration_statement = -1    # -1 is disabled, 0 logs all statements
                    # and their durations, > 0 logs only
                    # statements running at least this number
                    # of milliseconds


# - What to Log -

#debug_print_parse = off
#debug_print_rewritten = off
#debug_print_plan = off
#debug_pretty_print = on
log_checkpoints = off
#log_connections = off
log_disconnections = off
log_duration = off
log_error_verbosity = default       # terse, default, or verbose messages
log_hostname = off
#log_line_prefix = ''           # special values:
                    #   %a = application name
                    #   %u = user name
                    #   %d = database name
                    #   %r = remote host and port
                    #   %h = remote host
                    #   %p = process ID
                    #   %t = timestamp without milliseconds
                    #   %m = timestamp with milliseconds
                    #   %i = command tag
                    #   %e = SQL state
                    #   %c = session ID
                    #   %l = session line number
                    #   %s = session start timestamp
                    #   %v = virtual transaction ID
                    #   %x = transaction ID (0 if none)
                    #   %q = stop here in non-session
                    #        processes
                    #   %% = '%'
                    # e.g. '<%u%%%d> '
log_lock_waits = off            # log lock waits >= deadlock_timeout
#log_statement = 'none'         # none, ddl, mod, all
#log_temp_files = -1            # log temporary files equal or larger
                    # than the specified size in kilobytes;
                    # -1 disables, 0 logs all temp files
#log_timezone = '(defaults to server environment setting)'


#------------------------------------------------------------------------------
# RUNTIME STATISTICS
#------------------------------------------------------------------------------

# - Query/Index Statistics Collector -

track_activities = on
track_counts = on
track_functions = none          # none, pl, all
track_activity_query_size = 1024    # (change requires restart)
update_process_title = on
stats_temp_directory = 'pg_stat_tmp'


# - Statistics Monitoring -

#log_parser_stats = off
#log_planner_stats = off
#log_executor_stats = off
#log_statement_stats = off


#------------------------------------------------------------------------------
# AUTOVACUUM PARAMETERS
#------------------------------------------------------------------------------

autovacuum = on         # Enable autovacuum subprocess?  'on'
                    # requires track_counts to also be on.
log_autovacuum_min_duration = -1    # -1 disables, 0 logs all actions and
                    # their durations, > 0 logs only
                    # actions running at least this number
                    # of milliseconds.
autovacuum_max_workers = 3      # max number of autovacuum subprocesses
                    # (change requires restart)
autovacuum_naptime = 1min       # time between autovacuum runs
autovacuum_vacuum_threshold = 50    # min number of row updates before
                    # vacuum
autovacuum_analyze_threshold = 50   # min number of row updates before
                    # analyze
autovacuum_vacuum_scale_factor = 0.2    # fraction of table size before vacuum
autovacuum_analyze_scale_factor = 0.1   # fraction of table size before analyze
autovacuum_freeze_max_age = 200000000   # maximum XID age before forced vacuum
                    # (change requires restart)
autovacuum_vacuum_cost_delay = 20ms # default vacuum cost delay for
                    # autovacuum, in milliseconds;
                    # -1 means use vacuum_cost_delay
autovacuum_vacuum_cost_limit = -1   # default vacuum cost limit for
                    # autovacuum, -1 means use
                    # vacuum_cost_limit


#------------------------------------------------------------------------------
# CLIENT CONNECTION DEFAULTS
#------------------------------------------------------------------------------

# - Statement Behavior -

search_path = '"$user",public'      # schema names
default_tablespace = ''     # a tablespace name, '' uses the default
temp_tablespaces = ''           # a list of tablespace names, '' uses
                    # only default tablespace
check_function_bodies = on
default_transaction_isolation = 'read committed'
default_transaction_read_only = off
default_transaction_deferrable = off
session_replication_role = 'origin'
statement_timeout = 0           # in milliseconds, 0 is disabled
vacuum_freeze_min_age = 50000000
vacuum_freeze_table_age = 150000000
bytea_output = 'hex'            # hex, escape
xmlbinary = 'base64'
xmloption = 'content'

# - Locale and Formatting -

datestyle = 'iso, mdy'
#intervalstyle = 'postgres'
#timezone = '(defaults to server environment setting)'
#timezone_abbreviations = 'Default'     # Select the set of available time zone
                    # abbreviations.  Currently, there are
                    #   Default
                    #   Australia
                    #   India
                    # You can create your own file in
                    # share/timezonesets/.
#extra_float_digits = 0         # min -15, max 3
#client_encoding = sql_ascii        # actually, defaults to database
                    # encoding

# These settings are initialized by initdb, but they can be changed.
lc_messages = 'en_US.UTF-8'         # locale for system error message
                    # strings
lc_monetary = 'en_US.UTF-8'         # locale for monetary formatting
lc_numeric = 'en_US.UTF-8'          # locale for number formatting
lc_time = 'en_US.UTF-8'             # locale for time formatting

# default configuration for text search
default_text_search_config = 'pg_catalog.english'

# - Other Defaults -

#dynamic_library_path = '$libdir'
#local_preload_libraries = ''


#------------------------------------------------------------------------------
# LOCK MANAGEMENT
#------------------------------------------------------------------------------

#deadlock_timeout = 1s
max_locks_per_transaction = 64      # min 10
                    # (change requires restart)
# Note:  Each lock table slot uses ~270 bytes of shared memory, and there are
# max_locks_per_transaction * (max_connections + max_prepared_transactions)
# lock table slots.
max_pred_locks_per_transaction = 64 # min 10
                    # (change requires restart)

#------------------------------------------------------------------------------
# VERSION/PLATFORM COMPATIBILITY
#------------------------------------------------------------------------------

# - Previous PostgreSQL Versions -

#array_nulls = on
#backslash_quote = safe_encoding    # on, off, or safe_encoding
#default_with_oids = off
#escape_string_warning = on
#lo_compat_privileges = off
#quote_all_identifiers = off
#sql_inheritance = on
#standard_conforming_strings = on
#synchronize_seqscans = on

# - Other Platforms and Clients -

#transform_null_equals = off


#------------------------------------------------------------------------------
# ERROR HANDLING
#------------------------------------------------------------------------------

exit_on_error = off             # terminate session on any error?
restart_after_crash = on            # reinitialize after backend crash?


#------------------------------------------------------------------------------
# CUSTOMIZED OPTIONS
#------------------------------------------------------------------------------

custom_variable_classes = ''        # list of custom variable class names

这里是 pg_hba.conf 文件:

local   all             postgres                                trust

# TYPE  DATABASE        USER            ADDRESS                 METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
host    all             postgres             192.168.1.0/24         trust
host    all             all                  127.0.0.0/24          trust

我尝试启用/禁用 autovacuum、禁用日志记录等。没有成功。 M 系统有 8 个 CPU 和 3GB 内存,我有 1 TB 的专用空间用于这个 SQL 服务器。

感谢任何想法和帮助。

注意:This page 不能解决我的问题。

结论:获得 PGPool-II 3.x。非常容易设置(如答案中所述)并且非常快!

【问题讨论】:

  • @MarcusAdams 是的,它是 InnoDB。但是当我测试这个时,我也测试了 MyISAM 和它几乎相同。
  • 如果某些连接器速度快而不是其他连接器速度快,我首先要检查的两件事是慢速连接器是否为每个请求连接和断开连接。这可以解释差异。另一个问题是他们是否使用 TCP_NODELAY。默认情况下,如果您向 TCP 缓冲区写入内容,它会等待一段时间再发送,以防您可能想要发送更多内容。这对于流式传输大文件很有用,但对于数据库协议的来回传输不利。
  • 您的 work_mem * max_connections 设置太高。 (同样,但不太重要:shared_buffers 可以降低,effective_cachesize 增加)
  • @kgrittn 实际上 PHP 和 PgAdmin III 都很慢,但我只讲 PHP。所以,我认为问题出在配置上?
  • 你真的每次都重新连接吗?为什么不使用连接池。您不应为运行的每个查询打开和关闭连接。

标签: php mysql postgresql


【解决方案1】:

有趣的是,MySQL 建立连接的速度更快。在 PHP 这样的环境中,连接的建立和拆除速度非常快,而不是在 Rails 或 JVM 等应用程序环境中运行时间更长的连接。

在这种情况下,为 Postgres 使用连接池可能是有意义的。想到两个:

PGPool-II:http://www.pgpool.net/mediawiki/index.php/Main_Page

PgBouncer:http://pgfoundry.org/projects/pgbouncer/

我使用过 PGPool-II,它的设置非常简单。

【讨论】:

    【解决方案2】:

    嗯....我不是这里的专家,但是您不想将您的 hba 绑定到 NIC 或 localhost 而不是网络吗?

    在 Windows 机器上,

    # IPv4 local connections:
    host    all             all             127.0.0.1/32            md5
    # IPv6 local connections:
    host    all             all             ::1/128                 md5
    

    你有多少内存?多少虚拟内存?如果 VM 变得太大,它会减慢机器的速度。但有时你只是需要它。

    【讨论】:

    • 我有 12 GB RAM,其中只有 3GB 专用于 VM。 VM 不是太大,它是小型 CentOS VPS。但这仅用于测试。我有一台配备 i3 处理器和 4GB RAM 的专用服务器。
    • 看看如果你禁用虚拟内存,如果本地主机没有为你做,你会得到多少性能。你没有说你是在本地还是通过网络使用 pgsql。如果仅在本地,则使用我上面列出的本地主机设置。如果它是远程的,则使用您列出的 IP 设置,但要知道,如果这样做,您需要在 NIC 上使用静态 IP。你不能使用 DHCP。
    • 如果您只是连接到路由器并且这是本地流量,那么这应该不是问题,请将您的 NIC 设置为 192.168.1.5 或设置网站上列出的 DHCP 范围之外的内容路由器页面。那么您需要将 pgsql 绑定到该地址。获得 /32 或任何正确的数字不是我的专业领域。我把它留给 pgsql 安装程序。当你在[windows-logo-flag-key]-[pause-break] Advanced,Performance,settings,advanced,virtualmemory,change,调整设置中禁用VM时,点击SET和OK多次,重启。跨度>
    【解决方案3】:

    我对 PostgreSQL 了解不多,但我可以告诉你为什么 MySQL 对这个特定的查询速度很快。

    在 InnoDB 中,主键索引是聚集索引,这意味着页面以与主键相同的物理顺序存储。

    最后,当你请求ORDER BY DESC时,MySQL只是反向读取索引,所以性能和ASC一样。

    【讨论】:

    • 我明白了。我还将主主键设置为唯一且聚集的,但我猜它不像 innodb :)
    • 这不是正确的答案。 OP 正在创建与 Postgres 的新连接,这是一个缓慢且昂贵的过程。 Postgres 有两种连接类型。简单(纯文本)和高级(二进制)模式。默认为高级模式,需要两次往返和更多内部设置步骤才能启动。这是一个更快的连接,但需要更长的时间来创建。 OP 正在为每个查询创建连接,而不是重用相同的连接,这不是应该使用 postgres 连接的方式。在这种情况下,pooler 是解决方案。它“缓存”连接。
    • 此外,为每个查询创建新连接对于并行查询是不可扩展的,因为与 Postgres 服务器的连接数受配置 (The default is typically 100 connections)、系统资源和内核的限制。跨度>
    猜你喜欢
    • 2012-12-09
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2021-07-13
    • 1970-01-01
    • 1970-01-01
    • 2019-07-21
    相关资源
    最近更新 更多