我会尽力解释您遇到的错误是怎么回事。
您正在将一个列表传递给 get_urls({_, urls}) 函数，该函数与列表进行模式匹配并失败。现在，即使您正确传递了一个列表并枚举了 URL，当响应命中 |> Map.get(:body) 时您仍然会收到错误，因为您想获取 body 但您将获得一个正文列表，因此您仍然需要对其进行枚举，等等……
我会做这样的事情
def fetch_pages(url) do
url
|> HTTPoison.get()
|> response()
end
def process_pages(urls) when is_list(urls) do
resp =
urls
|> Task.async_stream(fn url -> fetch_pages(url) end)
|> Enum.map(fn {_, resp} -> resp end)
Enum.map(resp, fn r ->
r
|> Floki.parse_document!()
|> Floki.find(".review-entry")
|> Map.new(fn entry ->
[{"div", _, [date]}] = Floki.find(entry, "div.italic")
[{"p", _, [content]}] = Floki.find(entry, "p.review-content")
{date, content}
end)
end)
end
def response({:ok, %{body: {:ok, %{"error" => error}}}}) do
{:error, error}
end
def response({:ok, %{body: body}}), do: body
def response({:error, error}), do: {:error, error}
使用fetch_pages(URL)，您将能够测试一个 URL 并查看响应是什么，这也可以在以后重复用于其他页面，我在 process_pages(URLs) 中使用它。
process_pages(urls) 将处理您尝试使用 Floki 解析的 URL 列表。我在这里使用Task 模块同时获取这些页面。结果将是
[
%{
"June 17, 2021" => "Joe was great and took extra time to help make sure I got the car that I wanted not pushing me into a car I didn’t want. He even made sure my car was made ready in his day off. Great Job Thank You. \r\nMelisa S",
"June 20, 2021" => "Awesome service, Adrian was great to work with I told him what I wanted and he showed me the best car Thank you so much!",
"June 21, 2021" => "I heard a radio ad for McKaig Chevrolet and thought I would check it out; so I called and spoke with Brooke. She was incredibly helpful over the phone. I told my husband about it and we went. We met Joe Wynne and he was awesome. There was no pressure from him at all. He showed us what inventory they had, and even offered to find us a car if they didn't have it on the lot. He helped us find a great car, and I couldn't be happier with what we bought. Brandon walked us through the buying process and the different insurance plans. I will be back in the future when I get ready to buy again. It is worth the drive to Gladewater.",
"June 23, 2021" => "Dennis Made a very stressful process so simple for me. He had a very charismatic attitude and helped me in any I needed to make sure I left with something i love.",
"June 26, 2021" => "Joe was awesome and made car buying fast and easy!! He was very friendly and knowledgeable!! ",
"June 28, 2021" => "Overall a great experience, came in looking for a specific vehicle and got just that. The service was great Mrs. Kristina did a great job finding what I needed and I got to drive off in my new vehicle that same day! ",
"June 29, 2021" => "Excellent service great people and nothing shady about these people, willing to work with you and find the best deal for you. "
},
%{
"June 01, 2021" => "Excellent customer service. Got a 2017 Honda hatchback and I absolutely love it. Kristina helped me and she was awesome! Thank you guys so much! ",
"June 07, 2021" => "Loved our experience! Great listener! Understood what we needed as a geowing family and got us into the perfect fit. Always listened to my life story. ",
"June 09, 2021" => "Buying our first new car was so easy with the help of them! Kristina was so nice and made it so very smooth for us! ",
"June 10, 2021" => "Awesome experience. I hate pushy salespeople and Adrian was not! He was wonderful to work with. I appreciate the professional, polite young man. McKaig is lucky to have him as an employee. \r\n",
"June 11, 2021" => "Adrian Cortes was the best car salesman I have ever had! I have purchased a lot of cars. He was not pushy and very knowledgeable. ",
"June 12, 2021" => "Patrick and Mariela were so kind. Mariela remembered me from another visit I had with them last year and Patrick was consistent with keeping me up to date on the status of my vehicle. The job was completed sooner than expected and the problems I was having are gone! Super nice people that know what they are doing and I trust them to do it correctly. I drove from Longview to glade water and they shuttled me to Longview after drop off.",
"June 14, 2021" => "Amazing service, knowledge & paid great attention to find me exactly what I needed for my family & situation!!!! Never felt pushed or forced & I knew exactly every detail of every step!! Most definitely worth the drive from Longview to Gladewater!!!! I HOPE Adrian still works for McKaig when the time comes for me to purchase another vehicle!!!",
"May 31, 2021" => "I came in to this dealership to get a new car, and each person I interacted with was knowledgeable, easy to talk to, and accommodating. Dennis continually went out of his way to make the process easy, and he was even able to come pick us up and drive to the dealership because we didn't have a car prior. Taylor and Brandon were also extremely helpful. Great service overall and very satisfied with the car."
},
%{
"April 16, 2021" => "I really appreciate the help I got from Adrian he was very helpful and did everything he could to get me in a new car !! Definitely will be coming back when i need to make another purchase!! ",
"April 17, 2021" => "This was my first vehicle to pick and buy on my own. Adrian was amazing in helping me. I drove from Big Sandy and the moment I walked in Adrian was on it. I love my new Buick. Thanks again Adrian!!",
"April 21, 2021" => "Raymond Prazak was very patient & helpful! He worked hard to find the best vehicle to fit our budget! No pressure to buy the most expensive one! ?",
"April 29, 2021" => "a minute agoNEW\r\nI bought my 3rd vehicle from Mckaig. Dennis was my dealer this time and he was great to work with. I went in with an idea of what I want and he never tried to get me into anything outside of that. Went and picked my plates up today and he knew who I was instantly. Asked about how the family was enjoying it. I will definitely be back next time I'm in the market",
"May 06, 2021" => "My wife and I were in search of a newer car and had visited a dealership prior to coming toMckaig. The experience was not much of one as no one even acknowledged us. We got in our car and heard a radio ad for Mckaig in Gladewater so we decided to give it a shot. Literally seconds after walking in Joe said hello and made us feel welcome. He started the process and before we knew it we were test driving what is now our new family vehicle. He's only been with the dealership for a few months but has won our loyalty. He answered all of our questions and I feel like he will be an asset to their dealership. I'm writing this to say thanks for making us feel at home. This dealership really is for the people. ",
"May 12, 2021" => "Patient and works to met customers needs. They stayed with in my budget and was concerned with needs on the auto. \r\nWonderful people at your service ",
"May 17, 2021" => "I came in on the May 14th to have my oil changed on my truck , Mariela Hernandez is always a team player striving to help in any way she can . Austin was the Technician that did the service on my truck. He did a great job getting the oil changed and rotating the tires in a timely fashion also helped stopped the rattle from my exhaust and finding a problem with the vacuum pump. Great job to both of you guys. ",
"May 26, 2021" => "Great guy! Listened to every detail that I wanted on my new truck and got exactly what I wanted! No pressure no hassle. I will recommend him to everyone! "
},
%{
"April 05, 2021" => "Adrian was a great sales person. I drove in from Venus to get my oil changed and left with a brand new car. Thanks McKaig",
"April 09, 2021" => "Shonna was awesome she worked with us all afternoon and even stayed past closing to get us finished so we could leave with our car",
"April 10, 2021" => "In less than 3 hours I traded in my vehicle for a brand new one. Awesome service and no pressure to buy. Everyone is friendly and helpful. Today I dealt with Dennis, Adrian, Brandon, and Freddie. I highly recommend McKaig. ",
"April 13, 2021" => "I went to McKaig to possibly trade in my truck & left with a wonderful 2021 Buick Encore! Dennis Smith was a wonderful Sales person. He listened to exactly what I was wanting in a new vehicle & he helped me find it. He was respectful, helpful & personable. Brandon & Freddie were very helpful in the finance department. They helped me sign & seal the deal. I would recommend McKaig to anyone in the market for a new or used vehicle. ",
"April 14, 2021" => "I could not have picked a better place to do business with today! I have had previous car buying experiences at other dealerships where I’d been taken advantage of, but the people at McKaig made me feel so welcome from the start and walked me through every little detail. Adrian, Brandon, and Frankie made the process so quick and easy. Incredibly thankful for all of their help today!",
"April 16, 2021" => "I will purchase all of my future vehicles from McKaig Chevrolet Buick. My most recent purchase was a 2021 Chevrolet Silverado. Everyone who was involved was extremely helpful. Freddie Tomlinson was instrumental in suggesting which vehicle would suit my needs. Dennis Smith was extremely helpful with the sales process and Taylor was very detailed in explaining the Financing part of the transaction. The service here was awesome and the transaction was extremely smooth and easy. ",
"March 31, 2021" => "Lots of Knowledgeable, Friendly People. The person I talked to, Dennis Smith, was very Knowledgeable and Taylor also. Very pleasurable experience!!\r\nI would recommend anyone to check them out!!"
},
%{
"March 23, 2021" => "I have been buying vehicles at McKaig for decades, primarily for the service and the way they treat you. I feel like a part of the family when I walk in the door. ",
"March 24, 2021" => "Very nice people fairly easy process. Very nice used vehicle. Very friendly staff. Overall experience was great. Glad I came to your business. Will be sending customers your way",
"March 25, 2021" => "Had a great experience working with Adrian, he was very helpful, not pushy and very informative about everything and anything i had to ask. Definitely would recommend friends to come here for their shopping needs and to ask for adrian to be their salesman",
"March 26, 2021" => "We drove from New London and met Adrian at the door. From then till we signed the paperwork on a beautiful 2020 Chevy Equinox it was a real enjoyable experience. Everyone there were very friendly, just like a family.",
"March 28, 2021" => "I loved working with the staff at McKaig because everyone was so nice and friendly. They made me feel right at home! They were able to get me in a nice vehicle and I am completely satisfied. They really are a dealer FOR the people. ",
"March 30, 2021" => "Dennis was able to get me everything I was asking for in a vehicle. Not only that, but him as well as others, stayed late into the evening just to make sure I had something to drive away in. Couldn’t ask for better people to help me on my car-buying journey. "
}
]
response 是处理 HTTPoison 响应的辅助函数。编码愉快。