Comint 模式每 4096 个字符插入换行符答案

【问题标题】：Comint Mode Inserts Line Break Every 4096 CharactersComint 模式每 4096 个字符插入换行符
【发布时间】：2013-04-06 03:22:49
【问题描述】：

在 Ubuntu Lucid 上使用 Emacs 23.2.1，任何基于 Comint 的模式都会偶尔插入换行符以获得更大的输出（参见下面的示例 Shell 和 SQL 模式输出）。我在 SQL 模式和 Shell 模式下都试过了，两种情况下的结果都是一样的。在普通终端模拟器中运行类似的命令不会导致这些问题（对于 shell 模式和 mysql 模式命令）。

我尝试过的事情：

在 SQL 模式下使用 MySQL，添加以下标志：-A、-C、-t、-f、-n，并将 max_allowed_packet 设置为 16MB。
将 comint-buffer-maximum-size 设置为 10240。

这些都不会对此行为产生任何影响。

如果我向上滚动到有问题的行并删除换行符，则输出会正确显示，因此该问题的可能解决方案可能涉及删除每 4096 个字符的钩子（如果可能的话）。

注意：在终端示例中，输出似乎在每 4096 个字符以外的点处被截断。在 SQL 模式下，正好是每 4096 个（确实是一个可疑的数字）。

这是一些示例输出：

brent@battlecruiser:/$ for i in {1..4096}; do echo -n 0; done; echo;

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

在这种情况下，它应该打印出一行0，但实际上在904个字符之后插入了一个换行符。

还有一个使用 MySQL 的 SQL 模式示例：

mysql> show variables like '%n%';

+-----------------------------------------+----------------------------------+
| Variable_name                           | Value                            |
+-----------------------------------------+----------------------------------+
| auto_increment_increment                | 1                                |
| auto_increment_offset                   | 1                                |
| binlog_cache_size                       | 32768                            |
| binlog_format                           | STATEMENT                        |
| bulk_insert_buffer_size                 | 8388608                          |
| character_set_client                    | utf8                             |
| character_set_connection                | utf8                             |
| collation_connection                    | utf8_general_ci                  |
| collation_database                      | latin1_swedish_ci                |
| collation_server                        | latin1_swedish_ci                |
| completion_type                         | 0                                |
| concurrent_insert                       | 1                                |
| connect_timeout                         | 10                               |
| delayed_insert_limit                    | 100                              |
| delayed_insert_timeout                  | 300                              |
| div_precision_increment                 | 4                                |
| engine_condition_pushdown               | ON                               |
| error_count                             | 0                                |
| event_scheduler                         | OFF                              |
| foreign_key_checks                      | ON                               |
| ft_boolean_syntax                       | + -><()~*:""&|                   |
| ft_max_word_len                         | 84                               |
| ft_min_word_len                         | 4                                |
| ft_query_expansion_limit                | 20                               |
| general_log                             | OFF                              |
| general_log_file                        | /var/lib/mysql/battlecruiser.log |
| group_concat_max_len                    | 1024                             |
| have_community_features                 | YES                              |
| have_dynamic_loading                    | YES                              |
| have_innodb                             | YES                              |
| have_ndbcluster                         | NO                               |
| have_openssl                            | DISABLED                         |
| have_partitioning                       | YES                              |
| have_symlink                            | YES                              |
| hostname                                | battlecruiser                    |
| identity                                | 0                                |
| ignore_builtin_innodb                   | OFF                              |
| init_connect                            |                                  |
| init_file                               |                                  |
| init_slave                              |                                  |
| innodb_adaptive_hash_index              | ON                               |
| innodb_additional_mem_pool_size         | 1048576                          |
| innodb_autoextend_increment             | 8                                |
| innodb_autoinc_lock_mode                | 1                                |
| innodb_buffer_pool_size                 | 8388608                          |
| innodb_checksums                        | ON                               |
| innodb_commit_concurrency               | 0                                |
| innodb_concurrency_tickets              | 500                              |
| innodb_data_file_path                   | ibdata1:10M:autoextend
           |
| innodb_data_home_dir                    |                                  |
| innodb_doublewrite                      | ON                               |
| innodb_fast_shutdown                    | 1                                |
| innodb_file_io_threads                  | 4                                |
| innodb_file_per_table                   | OFF                              |
| innodb_flush_log_at_trx_commit          | 1                                |
| innodb_flush_method                     |                                  |
| innodb_force_recovery                   | 0                                |
| innodb_lock_wait_timeout                | 50                               |
| innodb_locks_unsafe_for_binlog          | OFF                              |
| innodb_log_buffer_size                  | 1048576                          |
| innodb_log_file_size                    | 5242880                          |
| innodb_log_files_in_group               | 2                                |
| innodb_log_group_home_dir               | ./                               |
| innodb_max_dirty_pages_pct              | 90                               |
| innodb_max_purge_lag                    | 0                                |
| innodb_mirrored_log_groups              | 1                                |
| innodb_open_files                       | 300                              |
| innodb_rollback_on_timeout              | OFF                              |
| innodb_stats_on_metadata                | ON                               |
| innodb_support_xa                       | ON                               |
| innodb_sync_spin_loops                  | 20                               |
| innodb_table_locks                      | ON                               |
| innodb_thread_concurrency               | 8                                |
| innodb_thread_sleep_delay               | 10000                            |
| innodb_use_legacy_cardinality_algorithm | ON                               |
| insert_id                               | 0                                |
| interactive_timeout                     | 28800                            |
| join_buffer_size                        | 131072                           |
| keep_files_on_create                    | OFF                              |
| key_cache_division_limit                | 100                              |
| language                                | /usr/share/mysql/english/        |
| last_insert_id                          | 0                                |
| lc_time_names                           | en_US                            |
| license                                 | GPL                              |
| local_infile                            | ON                               |
| locked_in_memory                        | OFF                              |
| log_bin                                 | OFF                              |
| log_bin_trust_function_creators         | OFF                              |
| log_bin_trust_routine_creators          | OFF                              |
| log_queries_not_using_indexes           | OFF                              |
| log_warnings                            | 1                                |
| long_query_time                         | 10.000000                        |
| lower_case_table_names                  | 0                                |
| max_binlog_cache_size                   | 4294963200                       |
| max_binlog_size                         | 104857600                        |
| max_connect_errors                      | 10                               |
| max_connections                         | 151                              |
| max_error_count                         | 64                               |
| max_insert_delayed_threads              | 20                               |
| max_join_size                           | 18446744073709551615             |
| max_length_for_sort_data                | 1024     
                        |
| max_prepared_stmt_count                 | 16382                            |
| max_sort_length                         | 1024                             |
| max_sp_recursion_depth                  | 0                                |
| max_user_connections                    | 0                                |
| max_write_lock_count                    | 4294967295                       |
| min_examined_row_limit                  | 0                                |
| multi_range_count                       | 256                              |
| myisam_data_pointer_size                | 6                                |
| myisam_recover_options                  | BACKUP                           |
| net_buffer_length                       | 16384                            |
| net_read_timeout                        | 30                               |
| net_retry_count                         | 10                               |
| net_write_timeout                       | 60                               |
| new                                     | OFF                              |
| open_files_limit                        | 1024                             |
| optimizer_prune_level                   | 1                                |
| plugin_dir                              | /usr/lib/mysql/plugin            |
| profiling                               | OFF                              |
| profiling_history_size                  | 15                               |
| protocol_version                        | 10                               |
| query_cache_min_res_unit                | 4096                             |
| query_cache_wlock_invalidate            | OFF                              |
| rand_seed1                              |                                  |
| rand_seed2                              |                                  |
| range_alloc_block_size                  | 4096                             |
| read_only                               | OFF                              |
| read_rnd_buffer_size                    | 262144                           |
| relay_log_index                         |                                  |
| relay_log_info_file                     | relay-log.info                   |
| rpl_recovery_rank                       | 0                                |
| skip_external_locking                   | ON                               |
| skip_networking                         | OFF                              |
| slave_net_timeout                       | 3600                             |
| slave_transaction_retries               | 10                               |
| slow_launch_time                        | 2                                |
| sql_auto_is_null                        | ON                               |
| sql_log_bin                             | ON                               |
| sql_max_join_size                       | 18446744073709551615             |
| sql_notes                               | ON                               |
| sql_slave_skip_counter                  |                                  |
| sql_warnings                            | OFF                              |
| storage_engine                          | MyISAM                           |
| sync_binlog                             | 0                                |
| sync_frm                                | ON                               |
| system_time_zone                        | EDT                              |
| table_definition_cache                  | 256                              |
| table_open_cache                        | 64                               |
| thread_handling                         | one-thread-per-connection        |
| time_zone                               | SYSTEM                           |
| transaction_alloc_block_size            | 8192                             |
| transaction_prealloc_size               | 4096                             |
| tx_isolation                          
  | REPEATABLE-READ                  |
| unique_checks                           | ON                               |
| version                                 | 5.1.41-3ubuntu12.10              |
| version_comment                         | (Ubuntu)                         |
| version_compile_machine                 | i486                             |
| version_compile_os                      | debian-linux-gnu                 |
| warning_count                           | 0                                |
+-----------------------------------------+----------------------------------+
159 rows in set (0.00 sec)

这里的输出总是以 4096 个字符的精确间隔被换行符打断。

除了可能的解决方案之外，如果有任何新方法可以找到有关正在发生的事情的更多信息，我们将不胜感激。

【问题讨论】：

标签： emacs

【解决方案1】：

我遇到了类似的问题，尽管我的休息时间似乎是 1024 个字符（啊哈！ 在 21_1 版本中就是这种情况）。这对我来说没什么大不了的，但我确实写了一些正确连接结果的东西，这样我就可以对它们进行后期处理。但这并没有影响输出，所以不会有太大帮助。

问题的根源在于process.c中的read_process_output，它硬编码了4096：

/* Read pending output from the process channel,
   starting with our buffered-ahead character if we have one.
   Yield number of decoded characters read.

   This function reads at most 4096 characters.
   If you want to read all available subprocess output,
   you must call it repeatedly until it returns zero.

   The characters read are decoded according to PROC's coding-system
   for decoding.  */

static int
read_process_output (proc, channel)
     Lisp_Object proc;
     register int channel;
{
  // ... snip
  int readmax = 4096;

就像你在问题中提到的那样，一个非常可能的解决方案是编写一个函数（调用它，clean-up-comint-output-at-4096-chars），并将其添加到comint-output-filter-functions。像这样的东西。 注意：未经测试的代码。

(add-hook 'comint-output-filter-functions 'clean-up-comint-output-at-4096-chars)
(defun clean-up-comint-output-at-4096-chars (&optional str)
  "look for string of 4096 length and remove newline in the buffer"
  (let ((magic-block-size 4096))
    (save-match-data
      (when (= magic-block-size (length str))
        ;; at the magic block size, look for a newline
        (goto-char (point-max))
        (when (and (search-backward str nil t)
                   (progn
                     (forward-char magic-block-size)
                     (looking-at "\n")))
          (delete-char 1))))))

【讨论】：

+1 知道问题出在哪里是一个巨大的帮助，谢谢。
@BrentNewey 多年来我一直想自己追踪它，希望我提出的解决方案是一个很好的开始。
我尝试将 readmax 中的值更改为 65536 并重新编译 Emacs，但我仍然观察到相同的结果。不确定是否有其他地方对此限制进行了硬编码，或者这种方法是否可行。
看看我的回答。我想知道您的配置中是否有与我从 Emacs Wiki 获得的相同代码。

【解决方案2】：

我已经找到了解决这个问题的方法。我在我的配置文件中放入了以下来自http://www.emacswiki.org/emacs/SqlMode的代码

  (defun sql-add-newline-first (output)
    "Add newline to beginning of OUTPUT for `comint-preoutput-filter-functions'"
    (concat "\n" output))

  (defun sqli-add-hooks ()
    "Add hooks to `sql-interactive-mode-hook'."
    (add-hook 'comint-preoutput-filter-functions
              'sql-add-newline-first))

  (add-hook 'sql-interactive-mode-hook 'sqli-add-hooks)

删除代码后（因为它设置了comint-preoutput-filter-functions，也会影响shell-mode），我不再遇到这些问题。

我建议替换此代码以获得我想要的行为（到目前为止对我有效）：

(defun sql-add-newline-first (output)
  "Add newline to beginning of OUTPUT for `comint-preoutput-filter-functions'"
  (remove-hook 'comint-preoutput-filter-functions
           'sql-add-newline-first)
  (concat "\n" output))

(defun sql-send-region-better (start end)
  "Send a region to the SQL process."
  (interactive "r")
  (if (buffer-live-p sql-buffer)
      (save-excursion
    (add-hook 'comint-preoutput-filter-functions
          'sql-add-newline-first)
    (comint-send-region sql-buffer start end)
    (if (string-match "\n$" (buffer-substring start end))
        ()
      (comint-send-string sql-buffer "\n"))
    (message "Sent string to buffer %s." (buffer-name sql-buffer))
    (if sql-pop-to-buffer-after-send-region
        (pop-to-buffer sql-buffer)
      (display-buffer sql-buffer)))
    (message "No SQL process started.")))

(defvar sql-mode-map
  (let ((map (make-sparse-keymap)))
    (define-key map (kbd "C-c C-c") 'sql-send-paragraph)
    (define-key map (kbd "C-c C-r") 'sql-send-region-better)
    (define-key map (kbd "C-c C-s") 'sql-send-string)
    (define-key map (kbd "C-c C-b") 'sql-send-buffer)
    map)
  "Mode map used for `sql-mode'.")

基本上，我在我的 sql-send-region-better 代码开始发送输出之前添加了钩子，然后 在钩子内我再次删除了钩子，保证它只插入一个我想要的新行。

【讨论】：

啊！你为什么不把它放在维基页面？我以更糟糕的方式重新实现了它。
哦，实际上，我的实现适用于所有 sql-send-* 命令，所以我将在此处发布并更新 wiki。

【解决方案3】：

这是我对每个输入仅添加一次“\n”的实现：

(defvar sql-last-prompt-pos 1
  "position of last prompt when added recording started")
(make-variable-buffer-local 'sql-last-prompt-pos)
(put 'sql-last-prompt-pos 'permanent-local t)



(defun sql-add-newline-first (output)
  "Add newline to beginning of OUTPUT for
  `comint-preoutput-filter-functions'

  This fixes up the display of queries sent to the inferior
  buffer programatically.  But also adds extra new-line for
  interactive commands.

  "
  (let ((begin-of-prompt
         (or (and comint-last-prompt-overlay
                  ;; sometimes this overlay is not on prompt
                  (save-excursion
                    (goto-char (overlay-start comint-last-prompt-overlay))
                    (looking-at-p comint-prompt-regexp)
                    (point)))
             1)))
    (if (> begin-of-prompt sql-last-prompt-pos)
        (progn
          (setq sql-last-prompt-pos begin-of-prompt)
          (concat "\n" output))
      output)))

(defun le-sqli-setup ()
  "Add hooks to `sql-interactive-mode-hook'."
  (add-hook 'comint-preoutput-filter-functions
            'sql-add-newline-first t t))
(add-hook 'sql-interactive-mode-hook 'le-sqli-setup)

【讨论】：

【解决方案4】：

我的解决方案 - 添加一个换行符（但然后删除钩子以防止多个字符破坏文本）。然后在每次输入提示时重新添加钩子。

(defun sql-add-newline-first (output)
    "Add newline to beginning of sql OUTPUT, but remove the hook so
     that it doesn't output a newline everytime the output cache is
     filled."
    (remove-hook 'comint-preoutput-filter-functions 'sql-add-newline-first)
    (concat "\n" output))

(defun sql-readd-newline-first (ignore)
  "Readd the newline putting hook"
  (add-hook 'comint-preoutput-filter-functions 'sql-add-newline-first))

(defun sqli-add-hooks ()
  "Add the 'suicidal' newline printing hook, and another hook to
   respawn it at every input prompt."
  (add-hook 'comint-preoutput-filter-functions 'sql-add-newline-first)
  (add-hook 'comint-input-filter-functions 'sql-readd-newline-first))

(add-hook 'sql-interactive-mode-hook 'sqli-add-hooks)

另外，就我而言，我使用的是 postgresql。它有一个讨厌的习惯，即在多行查询（如 database-# database-# database-# | col | col | ）之后放置额外的提示，这会将列名推开。为了解决，我最终这样做了：

(defun sql-remove-continuing-prompts (output)
  (concat "\n" (replace-regexp-in-string "warren_hero[^=()]# " "" output)))

(defun sqli-add-hooks ()
  (add-hook 'comint-preoutput-filter-functions 'sql-remove-continuing-prompts))

(add-hook 'sql-interactive-mode-hook 'sqli-add-hooks)

【讨论】：