【问题标题】:Boost Property Tree fails to retrieve simple JSON in multi-threaded contextBoost Property Tree 无法在多线程上下文中检索简单 JSON
【发布时间】:2021-04-23 20:23:32
【问题描述】:

我正在尝试在我的 C/C++ 应用程序中使用 Boost.PropertyTree 解析一个简单的 JSON 字符串。

{"header":{"version":42,"source":1,"destination":2},"coffee":"colombian"}

这是我在 C/C++ 多线程应用程序中的设置方式(手动定义 JSON 字符串来演示问题)。

ParseJson.cpp

#ifdef __cplusplus
extern "C"
{
#endif

#include "ParseJson.hpp"

#ifdef __cplusplus
}
#endif

#include <iostream>
#include <sstream>
#include <string>

#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>

using boost::property_tree::ptree;
using boost::property_tree::read_json;
using boost::property_tree::write_json;

extern "C" MyStruct * const parseJsonMessage(char * jsonMessage, unsigned int const messageLength) {
    MyStruct * myStruct = new MyStruct();
    // Create empty property tree object.
    ptree tree;

    if (myStruct != nullptr) {
        try {
            // Create an istringstream from the JSON message.
            std::string jsonMessageString("{\"header\":{\"version\":42,\"source\":1,\"destination\":2},\"coffee\":\"colombian\"}");   // doesn't work
            std::istringstream isStreamJson(jsonMessageString);

            // Parse the JSON into the property tree.
            std::cout << "Reading JSON ..." << jsonMessageString << "...";
            read_json(isStreamJson, tree);
            std::cout << " Done!" << std::endl;

            // Get the values from the property tree.
            printf("version: %d\n", tree.get<int>("header.version"));
            printf("source: %d\n", tree.get<int>("header.source"));
            printf("coffee: %s\n", tree.get<std::string>("coffee").c_str());
        }
        catch (boost::property_tree::ptree_bad_path badPathException) {
            std::cout << "Exception caught for bad path: " << badPathException.what() << std::endl;
            return nullptr;
        }
        catch (boost::property_tree::ptree_bad_data badDataException) {
            std::cout << "Exception caught for bad data: " << badDataException.what() << std::endl;
            return nullptr;
        }
        catch (std::exception exception) {
            std::cout << "Exception caught when parsing message into Boost.Property tree: " << exception.what() << std::endl;
            return nullptr;
        }
    }
    return myStruct;
}

read_json() 调用似乎已完成,但从属性树中检索解析数据的 get() 调用失败:

Reading JSON ...{"header":{"version":42,"source":1,"destination":2},"coffee":"colombian"}... Done!
Exception caught for bad path: No such node (header.version)

我在 RHEL 7 上使用 Boost 1.53(编译器是 gcc/g++ 版本 4.8.5),并且我已经尝试了 post 中提到的与 Boost.PropertyTree 和多线程相关的两个建议。我已经为项目全局定义了BOOST_SPIRIT_THREADSAFE 编译定义。我还尝试了为该帖子建议的原子交换解决方案。这些都对症状没有任何影响。

奇怪的是,我可以使用另一个 public methods 来手动获取 Boost.Property 树的值:

std::cout << "front.key: " << tree.front().first << std::endl;
std::cout << "front.front.key: " << tree.front().second.front().first << std::endl;
std::cout << "front.front.value: " << tree.front().second.front().second.get_value_optional<std::string>() << std::endl;

这表明 JSON 已被实际解析:

front.key: header
front.front.key: version
front.front.value:  42

注意,我必须使用 std::string 来获取 header.version 值,因为尝试使用 get_value_optional&lt;int&gt;() 也会崩溃。

但是,这种手动方法不可扩展;我的应用程序需要接受几个更复杂的 JSON 结构。

当我尝试更复杂的 JSON 字符串时,这些字符串也被成功解析,但使用 get() 方法访问值同样失败,这一次导致程序崩溃。这是我从崩溃中提取的 GDB 回溯之一,但我对 Boost 不够熟悉,无法从中获得任何有用的信息:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffebfff700 (LWP 7176)]
0x00007ffff5aa8200 in std::locale::locale(std::locale const&) () from /lib64/libstdc++.so.6
Missing separate debuginfos, use: debuginfo-install boost-system-1.53.0-28.el7.x86_64 boost-thread-1.53.0-28.el7.x86_64 bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.176-5.el7.x86_64 elfutils-libs-0.176-5.el7.x86_64 glibc-2.17-292.el7.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-37.el7_7.2.x86_64 libattr-2.4.46-13.el7.x86_64 libcap-2.22-10.el7.x86_64 libcom_err-1.42.9-16.el7.x86_64 libgcc-4.8.5-39.el7.x86_64 libselinux-2.5-14.1.el7.x86_64 libstdc++-4.8.5-39.el7.x86_64 openssl-libs-1.0.2k-19.el7.x86_64 pcre-8.32-17.el7.x86_64 systemd-libs-219-67.el7_7.2.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-18.el7.x86_64
(gdb) bt
#0  0x00007ffff5aa8200 in std::locale::locale(std::locale const&) () from /lib64/libstdc++.so.6
#1  0x00007ffff5ab6051 in std::basic_ios<char, std::char_traits<char> >::imbue(std::locale const&) () from /lib64/libstdc++.so.6
#2  0x000000000041e322 in boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int>::get_value(std::string const&) ()
#3  0x000000000041c5b2 in boost::optional<int> boost::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value_optional<int, boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int> >(boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int>) const ()
#4  0x000000000041aa61 in boost::enable_if<boost::property_tree::detail::is_translator<boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int> >, int>::type boost::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value<int, boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int> >(boost::property_tree::stream_translator<char, std::char_traits<char>, std::allocator<char>, int>) const ()
#5  0x000000000041985d in int boost::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get_value<int>() const ()
#6  0x0000000000418673 in int boost::property_tree::basic_ptree<std::string, std::string, std::less<std::string> >::get<int>(boost::property_tree::string_path<std::string, boost::property_tree::id_translator<std::string> > const&) const ()
#7  0x0000000000414f4a in parseJsonMessage ()
#8  0x000000000040d8cd in ProcessThread () at ../../src/Processing.c:906
#9  0x00007ffff7bc6ea5 in start_thread () from /lib64/libpthread.so.0
#10 0x00007ffff55538cd in clone () from /lib64/libc.so.6

FWIW,我尝试将此代码放入一个简单的(单线程)main.cpp:

#include <iostream>
#include <sstream>
#include <string>

#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>

using boost::property_tree::ptree;
using boost::property_tree::read_json;
using boost::property_tree::write_json;

int main(int numArgs, char * const * const args) {
    
    ptree tree;

    try {
        // Create an istringstream from the JSON message.
        std::string jsonMessageString("{\"header\":{\"version\":42,\"source\":1,\"destination\":2},\"coffee\":\"colombian\"}");
        std::istringstream isStreamJson(jsonMessageString);

        // Parse the JSON into the property tree.
        std::cout << "Reading JSON..." << jsonMessageString << "...";
        read_json(isStreamJson, tree);
        std::cout << " Done!" << std::endl;
        // Print what we parsed.
        std::cout << "version: " << tree.get<int>("header.version") << std::endl;
        std::cout << "source: " << tree.get<int>("header.source") << std::endl;
        std::cout << "coffee: " << tree.get<std::string>("coffee") << std::endl;
    }
    catch (boost::property_tree::ptree_bad_path badPathException) {
        std::cout << "Exception caught for bad path: " << badPathException.what() << std::endl;
        return -1;
    }
    catch (boost::property_tree::ptree_bad_data badDataException) {
        std::cout << "Exception caught for bad data: " << badDataException.what() << std::endl;
        return -1;
    }
    catch (std::exception exception) {
        std::cout << "Exception caught when parsing message into Boost.Property tree: " << exception.what() << std::endl;
        return -1;
    }
    std::cout << "Program completed!" << std::endl;
    return 0;
}

这段代码运行良好:

bash-4.2$ g++ -std=c++11 main.cpp -o main.exe
bash-4.2$ ./main.exe 
Reading JSON...{"header":{"version":42,"source":1,"destination":2},"coffee":"colombian"}... Done!
version: 42
source: 1
coffee: colombian
Program completed!

那么,为什么 Boost.PropertyTree get() 方法不适用于多线程应用程序? 多线程应用程序是 C 和 C++ 代码混合的事实是否可以引起问题?我看到我的特定编译器版本(GCC 4.8.5)没有explicitly verified这个Boost库......这可能是编译器问题吗?还是 Boost 1.53 版本有问题?


基于提供的答案的更新:

诚然,我的 parseJsonMessage 方法的原始代码很混乱(数十次调试迭代的产物,并删除了与问题无关的代码)。下面是一个更精简的版本,没有分心(以及可能的红鲱鱼):

#ifdef __cplusplus
extern "C"
{
#endif

#include "DirectIpRev3.hpp"

#ifdef __cplusplus
}
#endif

#include <iostream>
#include <sstream>
#include <string>

#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>

using boost::property_tree::ptree;
using boost::property_tree::read_json;
using boost::property_tree::write_json;

extern "C" void parseJsonMessage2() {
    // Create empty property tree object.
    ptree tree;
    std::string jsonMessageString("{\"header\":{\"version\":42,\"source\":1,\"destination\":2},\"coffee\":\"colombian\"}");   //doesn't work
    std::istringstream isStreamJson(jsonMessageString);
    try {
        read_json(isStreamJson, tree);
        std::cout << tree.get<int>("header.version") << std::endl;
        std::cout << tree.get<int>("header.source") << std::endl;
        std::cout << tree.get<std::string>("coffee") << std::endl;
    }
    catch (boost::property_tree::ptree_bad_path const & badPathException) {
        std::cerr << "Exception caught for bad path: " << badPathException.what() << std::endl;
    }
    catch (boost::property_tree::ptree_bad_data const & badDataException) {
        std::cerr << "Exception caught for bad data: " << badDataException.what() << std::endl;
    }
    catch (std::exception const & exception) {
        std::cerr << "Exception caught when parsing message into Boost.Property tree: " << exception.what() << std::endl;
    }
}

在我的多线程程序中运行这个压缩函数会产生异常:

Exception caught when parsing message into Boost.Property tree: <unspecified file>(1): expected object or array

没有异常处理,它会打印更多信息:

terminate called after throwing an instance of 'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::property_tree::json_parser::json_parser_error> >'
  what():  <unspecified file>(1): expected object or array

我仍然不太确定是什么导致了这里的失败,但我倾向于按照建议使用 nlohmann

【问题讨论】:

  • 不错的更新。如果可以的话,我会再次 +1。

标签: c++ multithreading boost boost-propertytree


【解决方案1】:

请不要使用属性树来“解析”“JSON”。见nlohmannBoost.JSON

进一步

  • 您显然没有充分理由使用原始 newdelete
  • 您有未使用的参数
  • 您正在按值捕获多态异常
  • 出现任何异常时都会泄漏内存并在出现分配错误时返回空指针

结合这些,我 99% 确定您的崩溃是由其他原因引起的:Undefined Behaviour 倾向于在内存损坏后出现在其他地方(例如堆栈抖动或删除后使用、越界)等等)。

使用我的水晶球

  1. 猜测:你没有显示,但结构可能看起来像

    typedef struct MyStructT {
        int version;
        int source;
        char const* coffee;
    } MyStruct;
    

    一个幼稚的错误是分配coffee,就像你打印它一样:

    myStruct->coffee = tree.get<std::string>("coffee").c_str();
    

    这里的“明显”(?) 问题是c_str() 指向值节点拥有的内存,并由ptree 传递。当函数返回时,该指针已过时。哎呀。 UB

  2. 您正在使用new 分配结构(尽管由于extern "C" 它很可能是POD,所以它给您一种错误的安全感,因为所有成员都有不确定的值)。

    另一个天真的错误是传递使用::free 取消分配的C 代码(就像它对malloc-ed 所做的一切一样,对)。这是 UB 的另一个潜在来源。

  3. 如果您“确定”了第一个想法,例如使用strdup,您可能会遇到更多内存泄漏的问题。即使你正确使用delete myStruct(或者开始使用malloc),你也必须记住::free分配给strdup的字符串。

  4. 您的 API 是典型的 C 风格(可能是故意的),但为传递错误的 messageLength 导致越界读取敞开了大门。由于观察到您甚至没有在上面自己的示例代码中使用参数,因此增加了发生这种情况的可能性。

多线程应力测试

这是在 Coliru 上实时进行的多线程压力测试。它在 25 个线程上进行 1000 次迭代。

Live On Coliru

#ifdef __cplusplus
extern "C"
{
#endif

typedef struct MyStructT {
    int version;
    int source;
    char* coffee;
} MyStruct;

//#include "ParseJson.hpp"

#ifdef __cplusplus
}
#endif

#include <iostream>
#include <sstream>
#include <string>

#define BOOST_BIND_GLOBAL_PLACEHOLDERS
#include <boost/property_tree/ptree.hpp>
#include <boost/property_tree/json_parser.hpp>

using boost::property_tree::ptree;
using boost::property_tree::read_json;
using boost::property_tree::write_json;

extern "C" MyStruct* parseJsonMessage(char const* jsonMessage, unsigned int const messageLength) {
    auto myStruct = std::make_unique<MyStruct>(); // make it exception safe
    // Create empty property tree object.
    ptree tree;

    if (myStruct != nullptr) {
        try {
            // Create an istringstream from the JSON message.
            std::istringstream isStreamJson(std::string(jsonMessage, messageLength));

            // Parse the JSON into the property tree.
            //std::cout << "Reading JSON ..." << isStreamJson.str() << "...";
            read_json(isStreamJson, tree);
            //std::cout << " Done!" << std::endl;

            // Get the values from the property tree.
            myStruct->version = tree.get<int>("header.version");
            myStruct->source = tree.get<int>("header.source");
            myStruct->coffee = ::strdup(tree.get<std::string>("coffee").c_str());
            return myStruct.release();
        }
        catch (boost::property_tree::ptree_bad_path const& badPathException) {
            std::cerr << "Exception caught for bad path: " << badPathException.what() << std::endl;
        }
        catch (boost::property_tree::ptree_bad_data const& badDataException) {
            std::cerr << "Exception caught for bad data: " << badDataException.what() << std::endl;
        }
        catch (std::exception const& exception) {
            std::cerr << "Exception caught when parsing message into Boost.Property tree: " << exception.what() << std::endl;
        }
    }
    return nullptr;
}

#include <cstdlib>
#include <string>
#include <thread>
#include <list>

int main() {
    static std::string_view msg = R"({"header":{"version":42,"source":1,"destination":2},"coffee":"colombian"})";

    auto task = [] {
        for (auto i = 1000; --i;) {
            auto s = parseJsonMessage(msg.data(), msg.size());

            ::printf("version: %d\n", s->version);
            ::printf("source: %d\n", s->source);
            ::printf("coffee: %s\n", s->coffee);

            ::free(s->coffee);
            delete s; // not ::free!
        }
    };

    std::list<std::thread> pool;

    for (int i = 0; i < 25; ++i)
        pool.emplace_back(task);

    for (auto& t : pool)
        t.join();
}

输出(排序和唯一):

  24975 coffee: colombian
  24975 source: 1
  24975 version: 42

【讨论】:

  • 非常感谢您的详细回复!我很抱歉,因为我最初的 parseJsonMessage 函数没有尽可能小,还有很多需要假设的地方。你提出了几个好的观点,这当然可能导致了奇怪的行为。我在上面发布了一个更精简的版本,我相信它解决了/消除了您发现的问题点,但仍然存在问题。看到您的示例运行良好当然令人鼓舞!
  • 我还是有点怀疑,这可能是我正在使用的编译器/Boost 版本的问题。我在Coliru 上尝试了您的示例,但找不到调整编译器/Boost 版本的方法,因此我尝试了Wandbox。此示例使用 Wandbox 上的最新默认设置运行良好,但是当我调整编译器和 Boost 版本以匹配我的(分别为 4.8.5 和 1.53)时,它没有运行。正如您所建议的,我可能会改用 nlohmann 的 JSON 解决方案,但我的好奇心仍然很强烈。
  • 1.53 已过时。喜欢:February 4th, 2013 old。我回答了一个类似的问题9 years ago。我还记得我记得一些解析器在某个 boost 版本中被完全重写,很可能在 1.53 之后。
  • 确实,1.59 引入了 A new JSON parser with full Unicode support. 。 Wandbox 确认问题with 1.58,成功with 1.59
猜你喜欢
  • 2016-07-16
  • 1970-01-01
  • 2020-03-31
  • 2013-01-24
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2016-11-17
相关资源
最近更新 更多