如何使用jsoup重定向到另一个页面并继续在ListView中打印内容？

Question

一般来说，我的网站主要内容是带有文本的帖子列表。所以我解析了这段 HTML 代码中的每一篇文章。

<div class="col-xs-12" style="margin:0.5em 0;line-height:1.785em">Some text</div>

为此，我创建了这个 AsyncTask。

class NewPostsAsyncTask extends AsyncTask<String, Void, String> {

    @Override
    protected void onPreExecute() {
        super.onPreExecute();

        progressDialog = new ProgressDialog(MainActivity.this);
        progressDialog.setTitle("Новые");
        progressDialog.setMessage("Загрузка...");
        progressDialog.setIndeterminate(false);
        progressDialog.show();
    }

    @Override
    protected String doInBackground(String... params) {
        Document doc;

        try {
            doc = Jsoup.connect(URL).get(); 

            content = doc.select("[style=margin:0.5em 0;line-height:1.785em]");
            titleList.clear();

            for (Element contents : content) {
                if (!contents.text().contains("18+")) {
                    titleList.add(contents.text());
                }
            }

        } catch (IOException e) {
            e.printStackTrace(); 
        }

        return null;
    }

    @Override
    protected void onPostExecute(String s) {
        super.onPostExecute(s);
        listView.setAdapter(adapter);
        progressDialog.dismiss();
    }
}

但是我有一些问题。所有帖子都不会存储在一个网页上。您必须在所有帖子的末尾单击链接才能重定向到包含帖子的另一个页面。

这个块有这个HTML代码。

    <div class="row"><div class="col-xs-12">
        <div class="paginator">

                <span class="pagina">1683</span> " | " 

                <span class="pagina"><a href="/page/1682">1682</a></span> " | " 

                <span class="pagina"><a href="/page/1681">1681</a></span> " | " 

                <span class="pagina"><a href="/page/1680">1680</a></span> " | " 

                <span class="pagina"><a href="/page/1679">1679</a></span> " | " 

                <span class="pagina"><a href="/page/3">3</a></span> " | "

                <span class="pagina"><a href="/page/2">2</a></span> " | " 

                <span class="pagina"><a href="/page/1">1</a></span>

        </div>
    </div>
</div>

如何在之前的帖子之后转到另一个页面，解析其他帖子并在 ListView 中打印它们？因此，我希望将本网站的所有帖子都放在一个 ListView 中。你能告诉我应该怎么做吗？

Answer 1

我会这样做：

示例代码

@Override
protected String doInBackground(String... params) {
    Document doc;

    // I supposed URL variable is initialized like this: URL="killpls.me";
    try {
        do {
            doc = Jsoup.connect(URL).get();

            content = doc.select("[style=margin:0.5em 0;line-height:1.785em]");
            titleList.clear();

            for (Element contents : content) {
                if (!contents.text().contains("18+")) {
                    titleList.add(contents.text());
                }
            }

            Element anchor = doc.select( //
                "#stories > div:nth-child(3) > div:nth-child(1) > div:nth-child(1) > span.pagina:not(:has(a)) + span > a" //
            ).first();
            if (anchor==null) {
                break;
            } else {
                doc = null;
                URL = anchor.absUrl("href"); 
            }
        } while(canContinue());
    } catch (IOException e) {
        e.printStackTrace(); 
    }

    return null;
}

private boolean canContinue() {
     // Implement custom logic here ...
     // Return true if additionnal posts should be downloaded false otherwise.
     return true;
}

一些细节

该方法的核心在于以下行：

Element anchor = doc.select( //
   "#stories > div:nth-child(3) > div:nth-child(1) > div:nth-child(1) > span.pagina:not(:has(a)) + span > a" //
).first();

只要存在下一页，first() 方法就会返回非 null 引用。到达第一页时，first() 返回 null 并且没有更多页面要获取。

#stories           /* Find an element with id `stories`*/
> div:nth-child(3) /* Select its third div child */
> div:nth-child(1) /* Select first div child of the previous div */
> div:nth-child(1) /* Select first div (DIV-a) child of the previous div */
> span.pagina:not(:has(a)) /* Select a span with class `pagina` without any anchor as child */
+ span /* Select closest span next to previous span and child of `DIV-a` */
> a    /* Here is the next page to fetch */