Python Web Scraping - 尝试从表格标签中获取数字

Question

难以从网页中提取一些我需要的数据。

这是我正在尝试的类型

from bs4 import BeautifulSoup
import urllib



url = 'http://www.myfxbook.com/community/outlook'

htmlfile = urllib.urlopen(url)

soup = BeautifulSoup(htmlfile)

for tag in soup.findAll('td'):
    print tag

（这会输出数千行代码）

This is the website

This is where the data that I need is held

我不知道如何使其足够具体

我需要在 'td' 标签内的 '5169lots' ，同样的东西对于 '2879lots' 再往下一点。

因此，我主要关注的是显示欧元兑美元多头和空头手数的两个数字。

任何帮助将不胜感激。

我正在使用 Python 2.7

Answer 1

from sys import exit
import lxml.html
import requests

url = 'http://www.myfxbook.com/community/outlook'
doc = requests.get(url)
root = lxml.html.fromstring(doc.text)

all_id_inputs = root.xpath("//input[starts-with(@id,'outlookTip')]/@id")

for ids in all_id_inputs:
    xpath_value = '//*[@id="{}"]/@value'.format(ids)
    table_value = root.xpath(xpath_value)
    root_table  = lxml.html.fromstring(table_value[0])

    # print table_value[0]
    # print 
    firt_value      =  root_table.xpath("//table/tr[2]/td[3]/text()")
    second_value    =  root_table.xpath("//table/tr[3]/td[3]/text()")
    third_value     =   root_table.xpath("//div/text()")[1]

    print firt_value
    print second_value
    print third_value
    print 

输出：

['4227.84 Lots']
['7968.21 Lots']
41% of traders are currently trading EURUSD.

['1330.00 Lots']
['3910.07 Lots']
17% of traders are currently trading GBPUSD.

['772.90 Lots']
['1339.60 Lots']
12% of traders are currently trading USDJPY.

['204.63 Lots']
['402.83 Lots']
6% of traders are currently trading GBPJPY.

['633.87 Lots']
['883.68 Lots']
8% of traders are currently trading USDCAD.

['172.53 Lots']
['555.24 Lots']
5% of traders are currently trading EURAUD.

['328.70 Lots']
['981.80 Lots']
8% of traders are currently trading EURJPY.

['747.65 Lots']
['361.27 Lots']
4% of traders are currently trading AUDCAD.

['197.44 Lots']
['139.89 Lots']
3% of traders are currently trading AUDJPY.

['285.17 Lots']
['329.60 Lots']
2% of traders are currently trading AUDNZD.

['1186.19 Lots']
['566.72 Lots']
9% of traders are currently trading AUDUSD.

['112.03 Lots']
['156.68 Lots']
1% of traders are currently trading CADJPY.

['146.43 Lots']
['236.74 Lots']
3% of traders are currently trading EURCAD.

['205.32 Lots']
['190.43 Lots']
1% of traders are currently trading EURCHF.

['0.00 Lots']
['0.00 Lots']
0% of traders are currently trading EURCZK.

['826.11 Lots']
['313.21 Lots']
7% of traders are currently trading EURGBP.

['3.67 Lots']
['3.01 Lots']
0% of traders are currently trading EURNOK.

['36.12 Lots']
['150.40 Lots']
2% of traders are currently trading EURNZD.

['5.74 Lots']
['1.92 Lots']
0% of traders are currently trading EURPLN.

['3.13 Lots']
['0.76 Lots']
0% of traders are currently trading EURSEK.

['18.04 Lots']
['4.85 Lots']
0% of traders are currently trading EURTRY.

['39.89 Lots']
['434.25 Lots']
3% of traders are currently trading GBPCAD.

['38.58 Lots']
['124.30 Lots']
1% of traders are currently trading GBPCHF.

['207.33 Lots']
['146.33 Lots']
1% of traders are currently trading NZDCAD.

['86.32 Lots']
['61.68 Lots']
1% of traders are currently trading NZDJPY.

['580.56 Lots']
['266.71 Lots']
5% of traders are currently trading NZDUSD.

['802.76 Lots']
['243.32 Lots']
4% of traders are currently trading USDCHF.

['0.00 Lots']
['0.22 Lots']
0% of traders are currently trading USDCZK.

['0.20 Lots']
['5.23 Lots']
0% of traders are currently trading USDHKD.

['9.18 Lots']
['7.99 Lots']
0% of traders are currently trading USDMXN.

['1.75 Lots']
['1.92 Lots']
0% of traders are currently trading USDNOK.

['2.32 Lots']
['0.41 Lots']
0% of traders are currently trading USDPLN.

['1.66 Lots']
['3.91 Lots']
0% of traders are currently trading USDSEK.

['31.71 Lots']
['11.61 Lots']
0% of traders are currently trading USDSGD.

['0.00 Lots']
['0.70 Lots']
0% of traders are currently trading USDTHB.

['13.18 Lots']
['3.78 Lots']
0% of traders are currently trading USDTRY.

['21.88 Lots']
['24.87 Lots']
0% of traders are currently trading USDZAR.

['28.33 Lots']
['76.17 Lots']
0% of traders are currently trading CHFJPY.

['160.82 Lots']
['35.39 Lots']
1% of traders are currently trading AUDCHF.

['156.37 Lots']
['337.38 Lots']
2% of traders are currently trading GBPNZD.

['70.59 Lots']
['9.52 Lots']
0% of traders are currently trading NZDCHF.

['7.56 Lots']
['44.56 Lots']
0% of traders are currently trading XAGUSD.

['347.22 Lots']
['372.50 Lots']
3% of traders are currently trading XAUUSD.

['85.77 Lots']
['48.52 Lots']
0% of traders are currently trading CADCHF.

['96.59 Lots']
['1021.96 Lots']
5% of traders are currently trading GBPAUD.

['2.00 Lots']
['0.00 Lots']
0% of traders are currently trading SPA35.

['10.22 Lots']
['14.55 Lots']
0% of traders are currently trading SGDJPY.

['0.42 Lots']
['0.00 Lots']
0% of traders are currently trading GBPNOK.

['29.20 Lots']
['24.30 Lots']
0% of traders are currently trading US30.

['4.15 Lots']
['3.30 Lots']
0% of traders are currently trading EURZAR.

['0.37 Lots']
['2.32 Lots']
0% of traders are currently trading AUDSGD.

['0.39 Lots']
['0.00 Lots']
0% of traders are currently trading GBPSEK.

['0.02 Lots']
['0.01 Lots']
0% of traders are currently trading CHFSGD.

['0.65 Lots']
['32.99 Lots']
0% of traders are currently trading EURSGD.

['0.24 Lots']
['0.25 Lots']
0% of traders are currently trading GBPSGD.

['173.06 Lots']
['14.01 Lots']
0% of traders are currently trading US500.

['10.80 Lots']
['9.10 Lots']
0% of traders are currently trading UK100.

['0.40 Lots']
['10.62 Lots']
0% of traders are currently trading NOKJPY.

['2.63 Lots']
['8.80 Lots']
0% of traders are currently trading ZARJPY.

['0.16 Lots']
['0.00 Lots']
0% of traders are currently trading GBPTRY.

['16.20 Lots']
['1.98 Lots']
0% of traders are currently trading USDRUB.

['0.20 Lots']
['0.22 Lots']
0% of traders are currently trading SEKJPY.

['0.02 Lots']
['0.05 Lots']
0% of traders are currently trading XAUAUD.

['0.00 Lots']
['0.48 Lots']
0% of traders are currently trading XAUEUR.

['0.01 Lots']
['3.40 Lots']
0% of traders are currently trading NAS100.

['0.30 Lots']
['0.02 Lots']
0% of traders are currently trading US2000.

['36.00 Lots']
['88.00 Lots']
0% of traders are currently trading FRA40.

['132.73 Lots']
['79.14 Lots']
0% of traders are currently trading GER30.

['1.00 Lots']
['0.50 Lots']
0% of traders are currently trading AUS200.

['0.00 Lots']
['0.00 Lots']
0% of traders are currently trading JPN225.

['0.00 Lots']
['0.06 Lots']
0% of traders are currently trading NOKSEK.

['0.06 Lots']
['0.28 Lots']
0% of traders are currently trading XAGEUR.

['0.00 Lots']
['0.03 Lots']
0% of traders are currently trading XPTUSD.

['0.00 Lots']
['0.01 Lots']
0% of traders are currently trading XPDUSD.

['0.89 Lots']
['4.13 Lots']
0% of traders are currently trading USDCNH.

['0.00 Lots']
['0.00 Lots']
0% of traders are currently trading IT40.

['1.10 Lots']
['3.42 Lots']
0% of traders are currently trading XTIUSD.

['0.55 Lots']
['0.82 Lots']
0% of traders are currently trading XBRUSD.

['18.00 Lots']
['18.00 Lots']
0% of traders are currently trading HK50.

['0.61 Lots']
['0.00 Lots']
0% of traders are currently trading XNGUSD.