【问题标题】:WebRequests in C# are very CPU Intensive. Need something betterC# 中的 WebRequest 非常占用 CPU。需要更好的东西
【发布时间】:2021-09-09 22:20:47
【问题描述】:

我有一个程序需要扫描网络上运行我的程序的其他设备。我想出的解决方案是调用每个 ipAddress 来查看我的程序是否正在运行。

下面的代码完全阻塞了cpu:-

using Newtonsoft.Json;
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading;
using System.Threading.Tasks;

namespace FileWire
{
    class SearchNearby
    {

        private bool pc_search_cancelled = false;
        private SynchronizedCollection<Thread> PCSearchThreadList;
        private ConcurrentDictionary<String, String> NearbyPCList;

        public void NewPcFound(string s, string s1)
        {
            Console.WriteLine(string.Format("PC Found at: {0}    PC Name: {1}", s, s1));
        }

        public SearchNearby()
        {
            startPCScan();

            while (true)
            {
                bool isAnyAlive = false;
                foreach(Thread t in PCSearchThreadList)
                {
                    isAnyAlive |= t.IsAlive;
                }
                if (!isAnyAlive)
                {
                    Console.WriteLine("Search Complete");
                    foreach (var a in NearbyPCList)
                    {
                        Console.WriteLine(a.Key + " ;; " + a.Value);
                    }
                    startPCScan();
                }
                Thread.Sleep(100);
            }
        }


        private void startPCScan()
        {
            PCSearchThreadList = new SynchronizedCollection<Thread>();
            NearbyPCList = new ConcurrentDictionary<String, String>();
            pc_search_cancelled = false;
            String add = "";
            System.Net.IPAddress[] ad = System.Net.Dns.GetHostByName(System.Net.Dns.GetHostName()).AddressList;
            foreach (System.Net.IPAddress ip in ad)
            {
                add += ip.ToString() + "\n";
            }
            bool connected;
            if (add.Trim(' ').Length == 0)
            {
                connected = false;
            }
            else
            {
                connected = true;
            }
            if (connected)
            {
                try
                {
                    String[] addresses = add.Split('\n');
                    foreach (String address in addresses)
                    {
                        int myIP = int.Parse(address.Substring(address.LastIndexOf(".") + 1));
                        for (int def = 0; def <= 10; def++)
                        {
                            int finalDef = def;

                            for (int j = 0; j < 10; j++)
                            {
                                string finalJ = j.ToString();
                                Thread thread = new Thread(new ThreadStart(() =>
                                {

                                    if (!pc_search_cancelled)
                                    {
                                        for (int i = (finalDef * 25); i < (finalDef * 25) + 25 && i <= 255; i++)
                                        {
                                            if (!pc_search_cancelled)
                                            {
                                                if (i != myIP)
                                                {
                                                    String callToAddress = "http://" + address.Substring(0, address.LastIndexOf(".")) + "." + i + ":" + (1234 + int.Parse(finalJ)).ToString();
                                                    String name = canGetNameAndAvatar(callToAddress);
                                                    if (name != null)
                                                    {
                                                        NearbyPCList[callToAddress] = name;
                                                        NewPcFound(callToAddress, name);
                                                    }
                                                }
                                            }
                                        }
                                    }

                                }));
                                PCSearchThreadList.Add(thread);
                                thread.Start();
                            }
                        }
                    }
                } catch (Exception e) {
            }
        }
    }
        private String canGetNameAndAvatar(String connection)
        {
            String link = connection + "/getAvatarAndName";
            link = link.Replace(" ", "%20");
            try
            {

                var client = new HttpClient();
                client.Timeout = TimeSpan.FromMilliseconds(500);
                var a = new Task<HttpResponseMessage>[1];
                a[0] = client.GetAsync(link);
                Task.WaitAll(a);
                var b = a[0].Result.Content.ReadAsStringAsync();
                Task.WaitAll(b);
                
                Console.WriteLine(b.Result);
                
                string result = b.Result;
                result = result.Substring(result.IndexOf("<body>") + 6, result.IndexOf("</body>") - (result.IndexOf("<body>") + 6));
                AvtarAndName json = JsonConvert.DeserializeObject<AvtarAndName>(result);
                if (json != null)
                {
                    return json.name;
                }
            }
            catch
            {
                return null;
            }
            return null;
        }
    }

}

这是我在 Java 中使用的 Java 代码的确切 C# 版本:-

import com.sun.istack.internal.Nullable;
import org.apache.http.*;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.params.CoreConnectionPNames;
import org.apache.http.params.HttpParams;
import org.json.JSONObject;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.InetAddress;
import java.net.NetworkInterface;
import java.net.SocketException;
import java.net.URI;
import java.util.*;
import java.util.concurrent.CopyOnWriteArrayList;

public class PCScan {

    private static boolean pc_search_cancelled = false;
    private static List<Thread> PCSearchThreadList;
    private static HashMapWithListener<String, String> NearbyPCList;
    public static void main(String[] args) {

        start();

        while (true) {
            int numCompleted = 0;
            for (Thread t : PCSearchThreadList) {
                if (!t.isAlive()) {
                    numCompleted++;
                }
            }
            if (numCompleted == PCSearchThreadList.size()) {
                start();
            }
        }
    }

    private static void start() {

        try {
            startPCScan();
        } catch (SocketException e) {
            e.printStackTrace();
        }

        NearbyPCList.setPutListener(new HashMapWithListener.putListener() {
            @Override
            public void onPut(Object key, Object value) {
                System.out.println(key.toString() + ";;" + value.toString());
            }
        });
    }

    private static void startPCScan() throws SocketException {
        pc_search_cancelled = false;
        PCSearchThreadList = new CopyOnWriteArrayList<>();
        NearbyPCList = new HashMapWithListener<>();
        Enumeration<NetworkInterface> enumeration = NetworkInterface.getNetworkInterfaces();
        boolean connected;
        String add = "";
        while (enumeration.hasMoreElements()) {
            NetworkInterface interfacea = enumeration.nextElement();
            if (!interfacea.isLoopback()) {
                Enumeration<InetAddress> enumeration1 = interfacea.getInetAddresses();

                while (enumeration1.hasMoreElements()) {
                    String address = enumeration1.nextElement().getHostAddress();
                    if (address.split("\\.").length == 4) {
                        add += address + "\n";
                    }
                }
            }

        }

        System.out.println(add);
        connected = true;
        if (connected) {
            try {
                String[] addresses = add.split("\n");
                addresses = new HashSet<String>(Arrays.asList(addresses)).toArray(new String[0]);
                for (String address : addresses) {
                    int myIP = Integer.parseInt(address.substring(address.lastIndexOf(".") + 1));
                    for (int def = 0; def <= 10; def++) {
                        int finalDef = def;

                        for (int j = 0; j < 10; j++) {
                            int finalJ = j;
                            Thread thread = new Thread(new Runnable() {
                                @Override
                                public void run() {
                                    if (!pc_search_cancelled) {
                                        for (int i = (finalDef * 25); i < (finalDef * 25) + 25 && i <= 255; i++) {
                                            if (!pc_search_cancelled) {
                                                if (i != myIP) {
                                                    String callToAddress = "http://" + address.substring(0, address.lastIndexOf(".")) + "." + i + ":" + String.valueOf(Integer.parseInt("1234") + finalJ);
                                                    String name = canGetNameAndAvatar(callToAddress);
                                                    if (name != null) {
                                                        NearbyPCList.put(callToAddress, name);
                                                    }
                                                }
                                            }
                                        }
                                    }
                                }
                            });
                            PCSearchThreadList.add(thread);
                            thread.start();
                        }
                    }
//                        }
//                    }).start();
                }
            } catch (Exception e) {
            }
        }
    }
    private static String canGetNameAndAvatar(String connection) {
        String link = connection + "/getAvatarAndName";
        link = link.replaceAll(" ", "%20");
        try {
            HttpClient client = new DefaultHttpClient();
            HttpParams httpParams = client.getParams();
            httpParams.setParameter(
                    CoreConnectionPNames.CONNECTION_TIMEOUT, 500);
            HttpGet request = new HttpGet();
            request.setURI(new URI(link));
            HttpResponse response = client.execute(request);
            BufferedReader in = new BufferedReader(new
                    InputStreamReader(response.getEntity().getContent()));

            StringBuffer sb = new StringBuffer("");
            String line="";

            while ((line = in.readLine()) != null) {
                sb.append(line);
                break;
            }
            in.close();
            String result = sb.toString();
            result = result.substring(result.indexOf("<body>") + 6, result.indexOf("</body>"));
            JSONObject json = new JSONObject(result);
            if (json != null) {
                return json.getString("name");
            }
        }
        catch (Exception ignored){
            return null;
        }
        return null;
    }
    static class HashMapWithListener<K, V> extends HashMap<K, V> {

        private putListener PutListener;
        public void setPutListener(putListener PutListener) {
            this.PutListener = PutListener;
        }

        @Nullable
        @Override
        public V put(K key, V value) {
            PutListener.onPut(key, value);
            return super.put(key, value);
        }

        interface putListener {
            public void onPut(Object key, Object value);
        }

    }

}

java 代码运行得非常好,只使用了大约 20% 的 cpu,而 c# 代码完全锁定了 PC。我试过 Webclient、webrequest、httpClient。都具有几乎相同的性能。

我需要将代码放在 c# 中,因为我无法在我的程序中包含整个 JRE,因为它太大了。我的程序和 GUI 的其余部分是 WPF 格式。

另外,我需要代码在扫描端口 1234-1243 时最多花费 50 秒。即使在中端安卓手机上,这段代码也能正常工作。所以,我不知道是什么问题。

【问题讨论】:

  • 阅读You're using HttpClient wrong and it is destabilizing your software 和后续You're (probably still) using HttpClient wrong and it is destabilizing your software。此外,在您的程序中完全使用 async/await,这样您就不会只是在运行等待 Web 请求完成的线程。
  • 感谢您的链接。但主要问题是它确实可以与 webclient 或 webrequest 一起使用。它们在打开多个连接时是否也在内部效率不高?
  • “但主要关注点” - 所以你不关心用尽机器上所有可用的套接字,从而导致失败?好的...无论如何,正如我已经说过的:您应该完全使用 async/await。 Async/await 实际上会挂起线程并在 IO 工作发生时挂起工作(你知道,就像 Web 请求一样),然后在它回来时恢复。相反,您正在使用支持异步/等待的HttpClient,但放弃了此功能,而是为每个请求创建一个线程,这意味着您让 CPU 旋转等待请求完成。
  • 它们在内部是高效的,但您编写的代码效率低下。
  • 对于您的用例,多播广播可能更合适。搜索它们。

标签: c# performance httprequest


【解决方案1】:

你使用线程和多线程完全错误。

因为我无法真正理解您要做什么,因为一切都被限制在一个功能中,所以我无法为您提供更详细的解决方案。

但是让我建议如下:我明白了,你想在后台执行一些操作,连接到其他计算机,试试这样的操作

var taskList = new List<Task>();
foreach (var pc in computers)
{
  var currentPcTask = Task.Run(() => DoYourWorkForSomePcHere(pc));
  taskList.Add(currentPcTask);
}
Task.WaitAll(taskList.ToArray());

这将非常节省 CPU。

【讨论】:

  • Task.Run(() =&gt; WebRequest()) 绝对不是正确的方法。在您等待请求完成时,它会占用您的线程池。
  • @Llama 异步等待如何占用线程池?证明我错了。
  • 我没有说async 正在吃掉线程池。
  • 那么你的答案和我的唯一区别是,你使用带有返回值的异步方法,而我假设他使用没有返回值的同步代码。你给我反对票吗?
  • 不,这不是“唯一的区别”。 OP 正在使用的Task.WaitAll 将阻塞它运行的线程,直到任务完成。这不是异步方式。通过使用Task.Run(尽管是间接地)执行此代码,您将强制它在线程池(docs)上的线程中运行。因此,这样写,我们阻塞了一个线程池线程,直到请求完成。
【解决方案2】:

我会建议这样的事情(为了举例,我已经简化了):

private static HttpClient _client = new HttpClient() { Timeout = TimeSpan.FromMilliseconds(500) };

private async Task<Something> GetSomething(string url)
{
    using (HttpResponseMessage response = await _client.GetAsync(url))
    {
        string json = await response.ReadAsStringAsync();
        return JsonConvert.DeserializeObject<Something>(json);
    }
}

private async Task<Something[]> GetSomethings(string[] urls)
{
    IEnumerable<Task<Something>> requestTasks = urls.Select(u => GetSomething(u));
    Something[] results = await Task.WhenAll<Something>(requestTasks);
    return results;
}

您还应该使调用 GetSomethings 的方法异步,并等待它,并在调用链的整个过程中执行相同的操作。

async/await 使用线程池来执行,在请求的 IO 部分发生的同时,线程实际上是挂起的,这意味着在此期间没有使用 CPU 时间。当 IO 部分完成后,它会在 await 处恢复代码。

相关信息:

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2019-06-19
    • 1970-01-01
    • 2016-07-22
    • 1970-01-01
    • 2012-03-29
    • 2021-06-29
    • 1970-01-01
    • 2015-08-10
    相关资源
    最近更新 更多