Question

.NET平台下几种SOCKET模型的简要性能供参考

Answer 1

1.Socket + Threads/ThreadPool大概性能：小于1500个连接 实现：Accept一个Socket，就交给一个线程去管理，比较笨，但也比较有效，因为是同步方式，控制起来很方便。高级点的，就是交给一个线程池去管理，线程池由系统自动托管，省去了开销线程的时间。一般小型项目，用这个完全足够，开发也简单。但要注意，如果若干Socket长时间占用线程池中的线程，同时其它连接数又比较多，很容易出现提示说你没有足够的线程供使用。呵呵，让Socket少做点事，少占用时间，换一个快点的CPU是不错的方式。另外，如果有一些比较好的第三方线程池组件，也可以选择使用，比如SmartThreadPool。2.Socket + Select大概性能：大于1500个连接后性能下降 实现：Select是很常用的一种模型。是在阻塞功能中轮询一个或多个Socket，将要处理的Socket放到一个IList中，当Select轮询结束后，然后我们再自己处理这个IList中的Socket。具体的用法可以看一下MSDN。Select的效率并不能说是高的，因为当队列中待处理的Socket比较多的时候，处理最后几个Socket相当于要遍历所有前面的Socket，非常不划算的.3.Socket + Asynchronous大概性能：约7500个客户端连接 实现：BeginXXXX，EndXXXX，再熟悉不过了吧。异步Socket归根到底，还是用的线程池技术，用线程池来处理异步IO。这就又引出个问题，.NET的线程池又是用的什么实现方式，以前看过有人说，.NET的线程池是用的完成端口来实现的，我不知道这样的说法是不是正确，从查到的资料中也没有办法确认（希望这点有朋友可以告诉我）。异步Socket对于程序的处理流程来说比同步复杂了许多，异步回调函数的控制不如同步方式那样直观。但有一点我想应该是要注意的，就是回调函数应该轻装上阵，不应该处理过多的事务，对传递数据的处理，应该交给其它线程进行处理。 4.IOCP（完成端口）大概性能：约20000~50000个客户端连接 实现：现在.NET下有一些伪IOCP，大家可以去搜索一下，还没有见过开放出来的用这些伪IOCP来实现的SOCKET例子。我说的20000~50000个客户端连接，是指在C++下开发的情况，这样的情况下，需要用到的基本技术还包括内存池、查询算法等。伪IOCP能实现多少最大连接，没有资料可以查，如果有朋友知道，可以讨论一下。另外上 面提到的许多数据，是从一些资料上摘抄下来的，我没有自己试过，仅仅是拿出来和大家讨论一下。1. Introduction - Native Win32 IOCPI/O Completion Ports (IOCP) supported on Microsoft Windows platforms has two facets. It first allows I/O handles like file handles, socket handles, etc., to be associated with a completion port. Any async I/O completion event related to the I/O handle associated with the IOCP will get queued onto this completion port. This allows threads to wait on the IOCP for any completion events. The second facet is that we can create a I/O completion port that is not associated with any I/O handle. In this case, the IOCP is purely used as a mechanism for efficiently providing a thread-safe waitable queue technique. This technique is interesting and efficient. Using this technique, a pool of a few threads can achieve good scalability and performance for an application. Here is a small example. For instance, if you are implementing a HTTP server application, then you need to do the following mundane tasks apart from the protocol implementation:Create a client connection listen socket. Once we get the client connection, use the client socket to communicate with the client to and fro. You can implement it by creating one dedicated thread per client connection that can continuously communicate with the client to and fro. But this technique quickly becomes a tremendous overhead on the system, and will reduce the performance of the system as the number of simultaneous active client connections increase. This is because, threads are costly resources, and thread switching is the major performance bottle neck especially when there are more number of threads.The best way to solve this is to use an IOCP with a pool of threads that can work with multiple client connections simultaneously. This can be achieved using some simple steps...Create a client connection listen socket. Once we get the client connection, post an IOCP read message on the socket to an IOCP. One of the threads waiting for completion events on this IOCP will receive the first read message for the client. It immediately posts another read onto the same IOCP and continues processing the read message it got. Once processing the read message is completed, it again waits on the IOCP for another event. This technique will allow a small pool of threads to efficiently handle communication with hundreds of client connections simultaneously. Moreover, this is a proven technique for developing scalable server side applications on Windows platforms.The above is a simplified description of using IOCP in multithreaded systems. There are some good in-depth articles on this topic in CodeProject and the Internet. Do a bit of Googling on words like IO Completion Ports, IOCP, etc., and you will be able to find good articles.2. Introduction - Managed IOCPManaged IOCP is a small .NET class library that provides the second facet of Native Win32 IOCP. This class library can be used both by C# and VB.NET applications. I chose the name Managed IOCP to keep the readers more close to the techniques they are used to with native Win32 IOCP. As the name highlights, Managed IOCP is implemented using pure .NET managed classes and pure .NET synchronization primitives. At its core, it provides a thread-safe object queuing and waitable object receive mechanism. Apart from that, it provides a lot more features. Here is what it does:Multiple Managed IOCP instances per process. Registration of multiple threads per Managed IOCP instance. Dispatching System.Object types to a threadsafe queue maintained by each Managed IOCP instance. Waitable multi-thread safe retrieval of objects from the Managed IOCP instance queue by all the threads registered for that particular Managed IOCP instance. Ability to restrict the number of concurrent active threads processing the queued objects related to a particular Managed IOCP instance. Policy based replaceable/customizable approach for choosing a registered thread to process the next available queued object. Ability to pause the Managed IOCP processing. Internally, pauses processing of queued objects by registered threads. Also, by default, disallows enqueuing new objects (can be changed). Run the Managed IOCP instance. Internally re-starts the processing of queued objects by registered threads. Also allows enqueuing new objects (if it is disallowed previously). Modify the max. allowed concurrent threads at runtime. Provides easy accessibility to Managed IOCP instance runtime properties like... Number of active concurrent threads. Number of objects left in queue. Number of allowed concurrent threads. Running status. Safe and controlled closing of a Managed IOCP instance. 2.1. Managed IOCP in Job/Task Oriented Business ProcessesManaged IOCP can be used in other scenarios apart from the sample that I mentioned in the introduction to native Win32 IOCP. It can be used in process oriented server side business applications. For instance, if you have a business process ( _not_ a Win32 process) with a sequence of tasks that will be executed by several clients, you will have to execute several instances of the business process, one for each client in parallel. As mentioned in my introduction to native Win32 IOCP, you can achieve this by spawning one dedicated thread per business process instance. But the system will quickly run out of resources, and the system/application performance will come down as more instances are created. Using Managed IOCP, you can achieve the same sequential execution of multiple business process instances, but with fewer threads. This can be done by dispatching each task in a business process instance as an object to Managed IOCP. It will be picked up by one of the waiting threads and will be executed. After completing the execution, the thread will dispatch the next task in the business process instance to the same Managed IOCP, which will be picked up by another waiting thread. This is a continuous cycle. The advantage is that you will be able to achieve the sequential execution goal of a business process, as only one waiting thread can receive a dispatched object, and at the same time keep the system resource utilization to required levels. Also, the system and business process execution performance will increase as there are few threads executing multiple parallel business processes.3. Using Managed IOCP in .NET applicationsMultithreaded systems are complex in the context that most problems will show up in real time production scenarios. To limit the possibility of such surprises while using Managed IOCP, I created a test application using which several aspects of the Managed IOCP library can be tested. Nevertheless, I look forward for any suggestions/corrections/inputs to improve this library and its demo application.Before getting into the demo application, below is the sequence of steps that an application would typically perform while using the Managed IOCP library:Create an instance of the ManagedIOCP class: 2. using Sonic.Net;ManagedIOCP mIOCP = new ManagedIOCP();The ManagedIOCP constructor takes one argument, concurrentThreads. This is an integer that specifies how many maximum concurrent active threads are allowed to process objects queued onto this instance of ManagedIOCP. I used a no argument constructor, which defaults to a maximum of one concurrent active thread.From a thread that needs to wait on objects queued onto the ManagedIOCP instance, call the Register() method on the ManagedIOCP instance. This will return an instance of the IOCPHandle class. This is like native Win32 IOCP handle, using which the registered thread can wait on the arrival of objects onto the ManagedIOCP instance. This thread can use the Wait() method on the IOCPHandle object. The Wait() will indefinitely wait until it grabs an object queued onto the ManagedIOCP instance to which the calling thread is registered. It either comes out with an object, or an exception in case the ManagedIOCP instance is stopped (we will cover this later). 4. IOCPHandle hIOCP = mIOCP.Register();5. while(true)6. {7. try8. {9. object obj = hIOCP.Wait();10. // Process the object11. 12. }13. catch(ManagedIOCPException e)14. {15. break;16. }17. catch(Exception e)18. {19. break;20. }}Any thread (one that is registered with the ManagedIOCP instance and any non-registered thread) that has access to the ManagedIOCP instance can dispatch (Enqueue) objects to it. These objects are picked up by waiting threads that are registered with the ManagedIOCP instance onto which objects are being dispatched. 22. string str = "Test string";mIOCP.Dispatch(str);When a thread decides not to wait for objects any more, it should un-register with the ManagedIOCP instance. mIOCP.UnRegister();Once the application is done with an instance of ManagedIOCP, it should call the Close() method on it. This will release any threads waiting on this instance of ManagedIOCP, clears internal resources, and resets the internal data members, thus providing a controlled and safe closure of a ManagedIOCP instance. mIOCP.Close();There are certain useful statistics that are exposed as properties in the ManagedIOCP class. You can use them for fine tuning the application during runtime.// Current number of threads that are // concurrently processing the objects queued // onto this instance of Managed IOCP // (This is readonly property) int activeThreads = mIOCP.ActiveThreads;// Max number of concurrent threads // allowed to process objects queued onto this // instance of Managed IOCP (This is a read/write property) int concurThreads = mIOCP.ConcurrentThreads;// Current count of objects queued onto this Managed IOCP instance. // NOTE: This value may change very quickly // as multiple concurrent threads might // be processing objects from this instance of Managed IOCP queue. // So _do not_ depend on this value // for logical operations. Use this only for // monitoring purpose (Status reporting, etc.) // and during cleanup processes // (like not exiting main thread untill the queued object becomes 0, // i.e. no more objects to be processed, etc) // (This is readonly property) int qCount = mIOCP.QueuedObjectCount;// Number of threads that are registered with this instance of Managed IOCP // (This is readonly property) int regThreadCount = mIOCP.RegisteredThreads;3.1. Advanced usageFollowing are the advanced features of Managed IOCP that need to be used carefully.Managed IOCP execution can be paused at runtime. When a Managed IOCP instance is paused, all the threads registered with this instance of Managed IOCP will stop processing the queued objects. Also, if the 'EnqueueOnPause' property of the ManagedIOCP instance is false (by default, it is false), then no thread will be able to dispatch new objects onto the Managed IOCP instance queue. Calling Dispatch on the ManagedIOCP instance will throw an exception in the Pause state. If the 'EnqueueOnPause' property is set to true, then threads can dispatch objects onto the queue, but you need to be careful while setting this property to true, as this will increase the number of pending objects in the queue, thus occupying more memory. Also, when the Managed IOCP instance is re-started, all the registered threads will suddenly start processing a huge number of objects thus creating greater hikes in the system resource utilization.mIOCP.Pause();Once paused, the ManagedIOCP instance can be re-started using the Run method.mIOCP.Run();The running status of the Managed IOCP instance can be obtained using the IsRunning property:bool bIsRunning = mIOCP.IsRunning;You can retrieve the System.Threading.Thread object of the thread associated with the IOCPHandle instance, from its property named 'OwningThread'.3.2. Demo ApplicationI provided two demo applications with similar logic. The first is implemented using Managed IOCP, the other using native Win32 IOCP. These two demo applications perform the following steps:Create a global static ManagedIOCP instance or native Win32 IOCP. Create five threads. Each thread will dispatch one integer value at a time to the ManagedIOCP instance or native Win32 IOCP until the specified number of objects are completed. Start (creates a new set of five threads) and stop (closes the running threads) the object processing. The Sonic.Net (ManagedIOCP) demo application additionally demonstrates the following features of Managed IOCP that are unavailable in the Win32 IOCP:Pause and continue object processing during runtime. Change concurrent threads at runtime. Statistics like, Active Threads, Maximum Concurrent threads, Queued Objects Count and Running Status of Managed IOCP. Below is the image showing both the demo applications after their first cycle of object processing:Demo application resultsAs you can see in the above figure, Managed IOCP gives the same speed (slightly even better) as native Win32 IOCP. The goal of these two demo applications is _not_ to compare the speed or features of Win32 IOCP with that of Managed IOCP, but rather to highlight that Managed IOCP provides all the advantages of native Win32 IOCP (with additional features) but in a purely managed environment.I tested these two demo applications on a single processor CPU and a dual processor CPU. The results are almost similar, in the sense the Managed IOCP is performing as good as (sometimes performing better than) native Win32 IOCP.3.3. Source and demo application filesBelow are the details of the files included in the article's Zip file:Sonic.Net (folder) - I named this class library as Sonic.Net (Sonic stands for speed). The namespace is also specified as Sonic.Net. All the classes that I described in this article are defined within this namespace. The folder hierarchy is described below: 2. Sonic.Net3. |4. --> Assemblies5. |6. --> Solution Files7. |8. --> Sonic.Net9. |10. --> Sonic.Net Console Demo 11. |--> Sonic.Net Demo ApplicationThe Assemblies folder contains the Sonic.Net.dll (contains the ObjectPool, Queue, ManagedIOCP, IOCPHandle, and ThreadPool classes), Sonic.Net Demo Application.exe (demo application showing the usage of ManagedIOCP and IOCPHandle classes), and Sonic.Net Console Demo.exe (console demo application showing the usage of ThreadPool and ObjectPool classes).The Solution Files folder contains the VS.NET 2003 solution file for the Sonic.Net assembly project, Sonic.Net demo application WinForms project, and Sonic.Net console demo project.The Sonic.Net folder contains the Sonic.Net assembly source code.The Sonic.Net Console Demo folder contains the Sonic.Net console demo application source code. This demo shows the usage of the Managed IOCP ThreadPool, which is explained in my Managed I/O Completion Ports - Part 2 article. This demo uses a file that will be read by the ThreadPool threads. Please change the file path to a valid one on your system. The code below shows the portion in the code to change. This code is in the ManagedIOCPConsoleDemo.cs file.public static void ReadData(){ StreamReader sr = File.OpenText(@"C:\aditya\downloads\lgslides.pdf"); string st = sr.ReadToEnd(); st = null; sr.Close(); Thread.Sleep(100);}The Sonic.Net Demo Application folder contains the Sonic.Net demo application source code.Win32IOCPDemo (folder) - This folder contains the WinForms based demo application for demonstrating Win32 IOCP usage using PInvoke. When compiled, the Win32IOCPDemo.exe will be created in the Win32IOCPDemo\bin\debug or Win32IOCPDemo\bin\Release folder based on the current build configuration you selected. The default build configuration is set to Release mode. 4. Inside Managed IOCPThis section discusses the how and why part of the core logic that is used to implement Managed IOCP.4.1. Waiting and retrieving objects in Managed IOCPManaged IOCP provides a thread safe object dispatch and retrieval mechanism. This could have been achieved by a simple synchronized queue. But with synchronized queue, when a thread (thread-A) dispatches (enqueues) an object onto the queue, for another thread (thread-B) to retrieve that object, it has to continuously monitor the queue. This technique is inefficient as thread-B will be continuously monitoring the queue for arrival of objects, irrespective of whether the objects are present in the queue. This leads to heavy CPU utilization and thread switching in the application when multiple threads are monitoring the same queue, thus degrading the performance of the system.Managed IOCP deals with this situation by attaching an auto reset event to each thread that wants to monitor the queue for objects and retrieve them. This is why any thread that wants to wait on a Managed IOCP queue and retrieve objects from it has to register with the Managed IOCP instance using its 'Register' method. The registered threads wait for the object arrival and retrieve them using the 'Wait' method of the IOCPHandle instance. The IOCPHandle instance contains an AutResetEvent that will be set by the Managed IOCP instance when any thread dispatches an object onto its queue. There is an interesting problem in this technique. Let us say that there are three threads, thread-A dispatching the objects, and thread-B and thread-C waiting on object arrival and retrieving them. Now, say if thread-A dispatches 10 objects in its slice of CPU time. Managed IOCP will set the AutoResetEvent of thread-B and thread-C, thus informing them of the new object arrival. Since it is an event, it does not have an indication of how many times it has been set. So if thread-B and thread-C just wake up on the event set and retrieve one object each from the queue and again waits on the event, there would be 8 more objects left over in the queue unattended. Also, this mechanism would waste the CPU slice given to thread-B and thread-C as they are trying to go into waiting mode after processing a single object from the Managed IOCP queue.So in Managed IOCP, when thread-B and thread-C call the 'Wait' method on their respective IOCPHandle instances, the method first tries to retrieve an object from the Managed IOCP instance queue before waiting on its event. If it was able to successfully retrieve the object, it does