如何使用 readability-lxml 从 html 中提取图像和文章？答案

【问题标题】：How can i extract images and articles from an html using readability-lxml?如何使用 readability-lxml 从 html 中提取图像和文章？
【发布时间】：2016-05-03 06:28:12
【问题描述】：

url = 'http://edition.cnn.com/'
    req = urllib.request.Request(url, data=None,
            headers={
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.47 Safari/537.36'
    }
    )
    f = urllib.request.urlopen(req)
    html = f.read()

    readable_article = Document(html).summary()
    readable_title = Document(html).short_title()

    print(readable_title)
    print(html)

    print(Document(html))

如何获取所有图像和文章？是否有任何内置功能？如果没有，那么如何？

【问题讨论】：

标签： python django readability

【解决方案1】：

我会推荐你正在寻找的报纸模块。

以下是我从他们的网站上获取的一些示例。您可以从这里下载并安装模块https://pypi.org/project/newspaper3k/

        >>> from newspaper import Article

        >>> url = u'http://fox13now.com/2013/12/30/new-year-new-laws-obamacare-pot-guns-and-drones/'
        >>> article = Article(url)
        >>> article.download()

        >>> article.html
        u'<!DOCTYPE HTML><html itemscope itemtype="http://...'
        >>> article.parse()

        >>> article.authors
        [u'Leigh Ann Caldwell', u'John Honway']

        >>> article.publish_date
        datetime.datetime(2013, 12, 30, 0, 0)

        >>> article.text
        u'Washington (CNN) -- Not everyone subscribes to a New Year's resolution...'

        >>> article.top_image
        u'http://someCDN.com/blah/blah/blah/file.png'

        >>> article.movies
        [u'http://youtube.com/path/to/link.com' ...]
        >>> article.nlp()

        >>> article.keywords
        [u'New Years', u'resolution', ...]

        >>> article.summary
        u'The study shows that 93% of people ...'
        >>> import newspaper

        >>> cnn_paper = newspaper.build(u'http://cnn.com')

        >>> for article in cnn_paper.articles:
        >>>     print(article.url)
        http://www.cnn.com/2013/11/27/justice/tucson-arizona-captive-girls/
        http://www.cnn.com/2013/12/11/us/texas-teen-dwi-wreck/index.html
        ...

        >>> for category in cnn_paper.category_urls():
        >>>     print(category)

        http://lifestyle.cnn.com
        http://cnn.com/world
        http://tech.cnn.com
        ...

        >>> cnn_article = cnn_paper.articles[0]
        >>> cnn_article.download()
        >>> cnn_article.parse()
        >>> cnn_article.nlp()
        ...
        >>> from newspaper import fulltext

        >>> html = requests.get(...).text
        >>> text = fulltext(html)

【讨论】：

看起来报纸包在安装方面有问题..不知道python3.4的可压缩性是否可用..无法安装..任何cmets
对于 Python 3，您应该使用报纸 3k (pypi.python.org/pypi/newspaper3k)