【问题标题】:Loop through axios data and perform another axios request and concat the two results into one JSON循环 axios 数据并执行另一个 axios 请求并将两个结果连接成一个 JSON
【发布时间】:2021-12-26 08:42:29
【问题描述】:

所以我有一个快速 API 的 axios 请求,我的函数看起来像这样......

//Initialize the lookup API that utalizes rapidAPI to get breach data
app.get("/lookup/:email/:function", (req, res) => {
  var options = {
    method: "GET",
    url: "https://breachdirectory.p.rapidapi.com/",
    params: { func: `${req.params.function}`, term: `${req.params.email}` },
    headers: {
      "x-rapidapi-host": "breachdirectory.p.rapidapi.com",
      "x-rapidapi-key": `${config.RAPID_API_KEY}`,
    },
  };
    axios
      .request(options)
      .then(function (response) {
        res.json(response.data);
      })
      .catch(function (error) {
        console.error(error);
      });
  } 
});

res.json(response.data); 将在页面上显示如下结果:

{
  "disclaimer": "This data is aggregated from BreachDirectory, HaveIBeenPwned, and Vigilante.pw.",
  "info": "For full source info, request e.g. https://breachdirectory.tk/api/source?name=Animoto",
  "sources": [
    "123RF",
    "500px",
    "Adobe",
    "AntiPublic",
    "Apollo",
    "Bitly",
    "Dave",
    "Disqus",
    "Dropbox",
    "ExploitIn",
    "ShareThis",
    "Straffic",
    "Ticketfly",
    "Tumblr",
    "VerificationsIO"
  ]
}

我想遍历“sources”数组中的所有内容,并调用以下内容:

https://haveibeenpwned.com/api/v3/breach/[ITEM]

所以,第一个会调用https://haveibeenpwned.com/api/v3/breach/123RF

So each result from that call will look like this:

{
  "Name": "123RF",
  "Title": "123RF",
  "Domain": "123rf.com",
  "BreachDate": "2020-03-22",
  "AddedDate": "2020-11-15T00:59:50Z",
  "ModifiedDate": "2020-11-15T01:07:10Z",
  "PwnCount": 8661578,
  "Description": "In March 2020, the stock photo site <a href=\"https://www.bleepingcomputer.com/news/security/popular-stock-photo-service-hit-by-data-breach-83m-records-for-sale/\" target=\"_blank\" rel=\"noopener\">123RF suffered a data breach</a> which impacted over 8 million subscribers and was subsequently sold online. The breach included email, IP and physical addresses, names, phone numbers and passwords stored as MD5 hashes. The data was provided to HIBP by <a href=\"https://dehashed.com/\" target=\"_blank\" rel=\"noopener\">dehashed.com</a>.",
  "LogoPath": "https://haveibeenpwned.com/Content/Images/PwnedLogos/123RF.png",
  "DataClasses": [
    "Email addresses",
    "IP addresses",
    "Names",
    "Passwords",
    "Phone numbers",
    "Physical addresses",
    "Usernames"
  ],
  "IsVerified": true,
  "IsFabricated": false,
  "IsSensitive": false,
  "IsRetired": false,
  "IsSpamList": false
}

我想让我的 res.json 发送一个 JSON 字符串,该字符串将保留所有源,以及它为每个调用提取的 API 调用中的“标题”、“描述”和“徽标路径”的来源。所以我将有一个 JSON 字符串,其中包含来源以及每个来源的标题、每个来源的描述以及每个来源的 LogoPath。

【问题讨论】:

    标签: node.js json express axios


    【解决方案1】:

    你有两个选择:

    1. 创建一个promise数组并使用Promise.all运行
    app.get('/lookup/:email/:function', async (req, res) => {
      var options = {
        method: 'GET',
        url: 'https://breachdirectory.p.rapidapi.com/',
        params: { func: `${req.params.function}`, term: `${req.params.email}` },
        headers: {
          'x-rapidapi-host': 'breachdirectory.p.rapidapi.com',
          'x-rapidapi-key': `${config.RAPID_API_KEY}`,
        },
      };
    
      axios.request(options)
        .then((response) => {
          const requestTasks = [];
    
          for (let item of response.data.sources) {
            const itemOption = {
              method: 'GET',
              url: `https://haveibeenpwned.com/api/v3/breach/${item}`,
              headers: {
                'content-type': 'application/json; charset=utf-8'
              }
            };
    
            requestTasks.push(axios.request(itemOption));
          }
    
          return Promise.all(requestTasks);
        })
        .then((responseList) => {
          for (let response of responseList) {
            console.log(response.data);
          }
        })
        .catch((error) => {
          console.error(error);
        });
    });
    
    1. 使用async/await (promise)for await从for循环中获取数据
    app.get('/lookup/:email/:function', async (req, res) => {
      try {
        var options = {
          method: 'GET',
          url: 'https://breachdirectory.p.rapidapi.com/',
          params: { func: `${req.params.function}`, term: `${req.params.email}` },
          headers: {
            'x-rapidapi-host': 'breachdirectory.p.rapidapi.com',
            'x-rapidapi-key': `${config.RAPID_API_KEY}`,
          },
        };
    
        const response = await axios.request(options);
        for await (let item of response.data.sources) {
          const itemOption = {
            method: 'GET',
            url: `https://haveibeenpwned.com/api/v3/breach/${item}`,
            headers: {
              'content-type': 'application/json; charset=utf-8'
            }
          };
          const itemResponse = await axios.request(itemOption);
          console.log(itemResponse.data);
        }
      } catch (error) {
        console.error(error);
      }
    });
    

    【讨论】:

    • Uncaught ReferenceError: lookupData is not defined
    • 我没有在这段代码中使用lookupData变量,你确定这段代码有问题吗?
    • 啊..黄一。好的,所以我更改了 console.log(itemResponse.data);到 res.send(itemResponse.data);我得到他的错误 [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client 错误。但它确实打印出第一个值
    • 如果我希望它输出包含所有数据的 JSON 字符串,最好的方法是什么?因为即使是 console.log 也没有返回正确的 json
    • 更新:解决方案 1 和 2 都返回 HTTP 标头发送错误。
    【解决方案2】:

    这就是我设法使它工作的方法。

    首先:我没有任何 APi 密钥(也不想注册获得)所以我使用了一个虚拟 Api。 虽然逻辑和我测试的结果一样。

    其次,我将您所有的初始网址都保留在我使用的网址旁边。因此您可以轻松切换回原始网址。

    最后我对任何关键部分进行了评论,并将变量命名为 他们几乎描述了他们所做的事情。 所以你可以复制过去的测试来理解我的逻辑,然后根据你的用例进行调整。

    这里是代码

    // make sure to replace /lookup by  /lookup/:email/:function after testing my logic
                
            app.get('/lookup', async (req, res) => {     
              
              try {
              // in this options no change just switch back to your url
              var options = {
                method: 'GET',
                url: 'https://jsonplaceholder.typicode.com/albums',
                //  url: "https://breachdirectory.p.rapidapi.com/",
                // params: { func: `${req.params.function}`, term: `${req.params.email}` },
                // headers: {
                //   'x-rapidapi-host': 'breachdirectory.p.rapidapi.com',
                //   'x-rapidapi-key': `${config.RAPID_API_KEY}`,
                // },
              };
            // here you get all your sources list (in my case it an array of object check picture 1 bellow)
              const allSources = await axios.request(options)
              console.log(allSources.data);
            
            // because my dummy api response is a huge array i slice to limited number
              const reduceAllsource = allSources.data.slice(0,5);
              console.log(reduceAllsource);
            
              // note here you need to replace reduceAllsource.map by allSources.data.map 
              // because you don't need a sliced array
              
              const allSourcesWithDetails = reduceAllsource.map(async (_1sourceEachtime)=>{
                // here you can switch back to your original url 
                // make sure to replace [ITEM] by ${_1sourceEachtime}
                const itemOption = await axios({
                  method: 'GET',
                  url: `https://jsonplaceholder.typicode.com/albums/${_1sourceEachtime.id}/photos`,
                  // url:`https://haveibeenpwned.com/api/v3/breach/[ITEM]`
                  headers: {
                    'content-type': 'application/json; charset=utf-8'
                  }
                });
                // this the place you can mix the 2 result.
    
                const mixRes1AndRes2 ={
                  sources:_1sourceEachtime.title,
                  details:itemOption.data.slice(0,1)
                }
                
                return mixRes1AndRes2;
              })
            // final result look like the picture 2 below
            
            finalRes= await Promise.all(allSourcesWithDetails);
            return res.status(200).json({response: finalRes});   
            
              }
              catch (error) {
                console.log(error);
              }      
            });
    

    图片1

    图2

    【讨论】:

    • 必须为我的 API 调用定制它,但它有效!
    • 不错!这就是为什么过度评论它。您可以将其标记为答案。快乐编码
    • @JoshHolly 如果它解决了您的问题,您可以将其标记为答案
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2021-01-10
    • 2021-08-24
    • 2020-10-19
    • 2019-02-06
    • 1970-01-01
    • 2017-06-06
    相关资源
    最近更新 更多