inets httpd cgi 脚本：如何检索 json 数据？答案

【问题标题】：inets httpd cgi script: How do you retrieve json data?inets httpd cgi 脚本：如何检索 json 数据？
【发布时间】：2018-08-16 17:12:02
【问题描述】：

我尝试过的 cgi 脚本无法从我的 inets httpd 服务器检索 json 数据。

为了在 cgi 脚本中检索 json 数据，您需要能够读取请求的主体，其中将包含以下内容：

{"a": 1, "b": 2}

使用 perl cgi 脚本，我可以像这样读取请求的正文：

my $cgi = CGI->new;
my $req_body = $cgi->param('POSTDATA');

我认为这是一种间接的方式来读取服务器管道到脚本标准输入的内容，因为在 python cgi 脚本中我必须编写：

req_body = sys.stdin.read()

当我从apache 服务器请求 cgi 脚本时，我的 perl 和 python cgi 脚本可以成功地从 apache 获取 json 数据。但是当我从我的inets httpd 服务器请求相同的 cgi 脚本时，我的 perl cgi 脚本对请求正文没有读取任何内容，并且我的 python cgi 脚本挂起然后服务器超时。我的 cgi 脚本能够从 inets httpd 服务器检索格式为 "a=1&b=2" 的数据——在这种情况下，perl 和 python 中的 cgi 工具会自动为我解析数据，所以不要尝试要阅读请求的正文，我只需访问 cgi 创建的结构。

这是我的 httpd 服务器配置 (server.conf)：

[
  {modules, [
    mod_alias,
    mod_actions,
    mod_esi,
    mod_cgi,
    mod_get,
    mod_log
  ]},
  {bind_address, "localhost"}, 
  {port,0},
  {server_name,"httpd_test"},
  {server_root,"/Users/7stud/erlang_programs/inets_proj"},
  {document_root,"./htdocs"},
  {script_alias, {"/cgi-bin/", "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"} },
  {erl_script_alias, {"/erl", [mymod]} },
  {erl_script_nocache, true},
  {error_log, "./errors.log"},
  {transfer_log, "./requests.log"}
].

我用这个程序启动我的 httpd 服务器 (s.erl)：

-module(s).
-compile(export_all).

%Need to look up port with httpd:info(Server)

ensure_inets_start() ->
    case inets:start() of
        ok -> ok;
        {error,{already_started,inets}} -> ok
    end.

start() ->
    ok = ensure_inets_start(),

    {ok, Server} = inets:start(httpd, 
        [{proplist_file, "./server.conf"}]
    ),
    Server.

stop(Server) ->
    ok = inets:stop(httpd, Server).

我的 cgi 脚本 (1.py)：

#!/usr/bin/env python3

import json
import cgi
import cgitb
cgitb.enable()  #errors to browser
import sys

sys.stdout.write("Content-Type: text/html")
sys.stdout.write("\r\n\r\n")

#print("<div>hello</div>")

req_body = sys.stdin.read()
my_dict = json.loads(req_body)

if my_dict:
    a = my_dict.get("a", "Not found")
    b = my_dict.get("b", "Not found")
    total = a + b
    print("<div>Got json: {}</div>".format(my_dict) )
    print("<div>a={}, b={}, total={}</div>".format(a, b, total))
else:
    print("<div>Couldn't read json data.</div>")

我的 cgi 脚本 (1.pl)：

#!/usr/bin/env perl

use strict;
use warnings;
use 5.020;
use autodie;
use Data::Dumper;
use CGI;
use CGI::Carp qw(fatalsToBrowser);
use JSON;

my $q = CGI->new;

print $q->header,
      $q->start_html("Test Page"),
      $q->h1("Results:"),
      $q->div("json=$json"),
      $q->end_html;

在终端窗口中启动服务器：

~/erlang_programs/inets_proj$ erl
Erlang/OTP 20 [erts-9.2] [source] [64-bit] [smp:4:4] [ds:4:4:10] [async-threads:10] [hipe] [kernel-poll:false]
Eshell V9.2  (abort with ^G)

1> c(s).              
s.erl:2: Warning: export_all flag enabled - all functions will be exported
{ok,s}

2> Server = s:start().
<0.86.0>

3> httpd:info(Server).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
 {server_name,"httpd_test"},
 {erl_script_nocache,true},
 {script_alias,{"/cgi-bin/",
                "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"}},
 {bind_address,{127,0,0,1}},
 {modules,[mod_alias,mod_actions,mod_esi,mod_cgi,mod_get,
           mod_log]},
 {server_root,"/Users/7stud/erlang_programs/inets_proj"},
 {erl_script_alias,{"/erl",[mymod]}},
 {port,51301},
 {transfer_log,<0.93.0>},
 {error_log,<0.92.0>},
 {document_root,"./htdocs"}]
4>

卷曲请求：

$ curl -v \
> -H 'Content-Type: application/json' \
> --data '{"a": 1, "b": 2}' \
> http://localhost:51301/cgi-bin/1.py

*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 51301 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 51301 (#0)
> POST /cgi-bin/1.py HTTP/1.1
> Host: localhost:51301
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 16
> 
* upload completely sent off: 16 out of 16 bytes

===== hangs for about 5 seconds ====

< HTTP/1.1 504 Gateway Time-out
< Date: Thu, 08 Mar 2018 11:02:27 GMT
< Content-Type: text/html
< Server: inets/6.4.5
* no chunk, no close, no size. Assume close to signal end
< 
* Closing connection 0
$

我的目录结构：

~/erlang_programs$ tree inets_proj/
inets_proj/
├── apache_cl.erl
├── cgi-bin
│   ├── 1.pl
│   └── 1.py
├── cl.beam
├── cl.erl
├── errors.log
├── htdocs
│   └── file1.txt
├── mylog.log
├── mymod.beam
├── mymod.erl
├── old_server.conf
├── old_server3.conf
├── old_server4.conf
├── requests.log
├── s.beam
├── s.erl
├── server.conf
└── urlencoded_post_cl.erl

【问题讨论】：

标签： apache erlang cgi inets

【解决方案1】：

我挖出了 cgi 规范的 RFC，上面写着：

RFC 3875                    CGI Version 1.1                 October 2004


4.2.  Request Message-Body

   Request data is accessed by the script in a system-defined method;
   unless defined otherwise, this will be by reading the 'standard
   input' file descriptor or file handle.

      Request-Data   = [ request-body ] [ extension-data ]
      request-body   = <CONTENT_LENGTH>OCTET
      extension-data = *OCTET

   A request-body is supplied with the request if the CONTENT_LENGTH is
   not NULL.  The server MUST make at least that many bytes available
   for the script to read.  The server MAY signal an end-of-file
   condition after CONTENT_LENGTH bytes have been read or it MAY supply
   extension data.  Therefore, the script MUST NOT attempt to read more
   than CONTENT_LENGTH bytes, even if more data is available.  However,
   it is not obliged to read any of the data.

我不明白扩展数据是什么，但关键是：

[cgi] 脚本不得尝试读取超过 CONTENT_LENGTH 字节，即使有更多数据可用。

如果我更改我的 python 脚本以读取内容长度而不是尝试读取整个 stdin 文件——它在收到 eof 信号之前不会停止读取——那么我的 python cgi 脚本成功检索 json来自我的 inets httpd 服务器的数据。

#!/usr/bin/env python3

import json
import sys
import os

content_len = int(os.environ["CONTENT_LENGTH"])

req_body = sys.stdin.read(content_len)
my_dict = json.loads(req_body)

sys.stdout.write("Content-Type: text/html")
sys.stdout.write("\r\n\r\n")

if my_dict:
    a = my_dict.get("a", "Not found")
    b = my_dict.get("b", "Not found")
    total = a + b
    print("<div>Content-Length={}</div".format(content_len))
    print("<div>Got json: {}</div>".format(my_dict) )
    print("<div>a={}, b={}, total={}</div>".format(a, b, total))
else:
    print("<div>Couldn't read json data.</div>")

'''
form = cgi.FieldStorage()

if "a" not in form:
    print("<H1>Error:</H1>")
    print("<div>'a' not in form</div>")
else:
    print("<p>a:{}</p>".format( form["a"].value) )


if "b" not in form:
    print("<H1>Error:</H1>")
    print("<div>'b' not in form</div>")
else:
    print("<p>b:{}</p>".format(form["b"].value) )
'''

服务器信息：

4> httpd:info(Server).
[{mime_types,[{"htm","text/html"},{"html","text/html"}]},
 {server_name,"httpd_test"},
 {erl_script_nocache,true},
 {script_alias,{"/cgi-bin/",
                "/Users/7stud/erlang_programs/inets_proj/cgi-bin/"}},
 {bind_address,{127,0,0,1}},
 {modules,[mod_alias,mod_actions,mod_esi,mod_cgi,mod_get,
           mod_log]},
 {server_root,"/Users/7stud/erlang_programs/inets_proj"},
 {erl_script_alias,{"/erl",[mymod]}},
 {port,65451},
 {transfer_log,<0.93.0>},
 {error_log,<0.92.0>},
 {document_root,"./htdocs"}]
5>

curl 请求（注意 curl 会自动计算内容长度并将其放入 Content-Length 标头中）：

~$ curl -v \
> -H 'Content-Type: application/json' \
> --data '{"a": 1, "b": 2}' \
> http://localhost:65451/cgi-bin/1.py

*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 65451 failed: Connection refused
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 65451 (#0)
> POST /cgi-bin/1.py HTTP/1.1
> Host: localhost:65451
> User-Agent: curl/7.58.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 16
> 
* upload completely sent off: 16 out of 16 bytes
< HTTP/1.1 200 OK
< Date: Fri, 09 Mar 2018 04:36:42 GMT
< Server: inets/6.4.5
< Transfer-Encoding: chunked
< Content-Type: text/html
< 
<div>Content-Length=16</div
<div>Got json: {'a': 1, 'b': 2}</div>
<div>a=1, b=2, total=3</div>
* Connection #0 to host localhost left intact
~$

这是我使用 inets httpd (1.pl) 得到的 perl 脚本：

#!/usr/bin/env perl

use strict;
use warnings;
use 5.020;
use autodie;
use Data::Dumper;
use JSON;

if (my $content_len = $ENV{CONTENT_LENGTH}) {

    read(STDIN, my $json, $content_len);
    my $href = decode_json($json);
    my $a = $href->{a};
    my $b = $href->{b};

    print 'Content-type: text/html';
    print "\r\n\r\n";
    print "<div>a=$a</div>";
    print "<div>b=$b</div>";

    #my $q = CGI->new; #Doesn't work with inets httpd server
    #my $q = CGI->new(''); #Doesn't try to read from stdin, do does work.

    #  print $q->header,
    #    $q->start_html("Test Page"),
    #    $q->div("json=$json"),
    #    $q->div("a=$a"),
    #    $q->div("b=$b"),
    #    $q->div("total=$total"),
    #    $q->end_html;
} 
else {
    my $error = "Could not read json: No Content-Length header in request.";
    print 'Content-type: text/html';
    print "\r\n\r\n";
    print "<div>$error</div>";


    #   my $q = CGI->new;
    #   print $q->header,
    #         $q->start_html("Test Page"),
    #         $q->h1($error),
    #         $q->end_html;
}

我无法让 perl 的 CGI 模块与从 STDIN 读取一起工作。 编辑： perlmonks 的一位好心人帮我解决了这个问题：

my $q = CGI->new('');

空白字符串告诉 CGI->new 不要从标准输入读取并解析数据。

【讨论】：