c# - .NET: How to efficiently check for uniqueness in a List<string> of 50,000 items? -

-

In some library codes, I have a list that can contain 50,000 items or more.

The callers of the library can call the methods, which resulted in being added to the wire list. How do I check efficiently for string specificity?

Currently, before adding a string, I am scanning the entire list and comparing each string in the in-linked string. It displays the scale problems above 10,000 objects.

I benchmark it, but I am interested in the insights.

  • If I replace the list with a list & lt;> then it will include () increases 10,000 items and beyond?
  • If I postpone the Specification Check after adding all the items, will it be faster? At that point I will need to examine every element against every element, even then an N ^^ 2 operation.
    • Edit

    A few basic benchmark results I have created an abstract class that exposes 2 ways: Fill and Fill the scan with just n items (I used 50,000). M Scans in the scan list (I use 5000) to see if a given value exists or not. Then I created the implementation of that class for the list, and for the second hashaseet.

    The used wire was evenly 11 characters in length, and randomly generated through a method in the abstract class.

    A very basic micro-benchmark 

      Hello from cheeses. The liststones are filling 50000 items ... 5000 scanning items ... filling time: 00: 00: 00.4428266 Scanning time: 00: 00: 13.0291180 Chiso hello 5000 scanning objects ... to fill time: 00: 00: 00.3797751 Scanning time: 00: 00: 00.4364431

    Therefore, that length for the stars is about 25x faster than the hashet's list, when scanning for specificity, besides the collection For this size, while adding items to the collection, there is a zero penalty on the list of hashetsets.

    The results are not interesting and valid, to get legitimate results, with random selection of implementation, I have to do warm-up interval, several tests. But I believe it will take only a few times.

    Thanks everyone

    EDIT2

    After adding randomization and multifunctional tests, Hasht constantly outlays the list in this case , Up to approximately 20x

    These results do not necessarily have to be variable lengths, more complex objects, or strings of different collection sizes.

You should use a class, which is specially designed for what you are doing.


Comments

Post a Comment

Popular posts from this blog

c# - How to capture HTTP packet with SharpPcap -

-
I would like to capture all incoming HTTP packets on my machine. To do this, I am using SharpPcap which is a WinPcap The cover is. SharpPcap works very well, but it captures TCP packets and what it wants to do for me is a very low level. Does anyone know how can I easily get full HTTP requests / reactions from all these TCP packets? Thanks SharpPcap is already able to capture the packet in the same way That's what the virus does (just in the code instead of the GUI). And you can either either parse them directly or you can put them in the drive in the common .pcap file format. The steps to parse the capture are: Open a connection at any time Loop a while Or start capturing using an event callback Parsing the raw packet for the type you want If you are reading .PPP dump files In addition to calling you an offline capture reader is almost the same, there is no need to select an interface, and you can set the majority There is no need to do the modes. All st
Read more

jquery - SimpleModal Confirm fails to submit form -

-
I am trying to confirm jQuery to show yes / no confirmation when I submit a form. I had modified (in the last list) to capture and block form submissions. However, it only works with jQuery 1.2.6, a recent upgrade stops 1.3.2, if yes is clicked, then by submitting the form. But I can not understand what is wrong / inconsistent in my code now. Demo (which works in both versions of a simple window.href implementation). $ (document) .ready (function () {$ ('input.delete'). Click (function (e) {e.preventDefault (); var target = $ (e .target); Confirm ("Really delete this event?", Function () {target.click ()});});}); How about something like this? The idea of ​​catching clicks (and fire!) On the submitted event is not click event. $ (document) .ready (function () {$ ('# myForm'). Submit (function () {var f = this; Confirm (" Remove? ", Function () {$ (f). Submit ();}) Return; Return the initial session;}}}}; Hope That this is som
Read more

php - Multiple Select with Explode: only returns the word "Array" -

-
By using Wordpress I have created a multiple selection box so that users can select categories to exclude. When the page is initially loaded, I see my predefined default option. However, when I select a new value and I save ... I only see the word "Array" and have not chosen anything else? & lt; Class = "amultiple" id = "& lt; php echo $ value ['id'] ;? gt;" name = "& gt; php echo $ value ['id'] ;; gt; []" Multiple = "multiple" size = "8" & gt; & Lt ;? Php global option; Foreign currency ($ value as value $) {if (get_settings ($ value ['id']) === FALSE {$$ value ['id'] = $ value ['std']; } And {$$ value ['id'] = get_settings ($ value ['id']); }} $ Ranges = & amp; Amp; Categories ('type = post & orderbiz = name & amp; hide_empty = 1'); If ($ ranges) {$ ex_cat = implode (',', $ tt_cat_exclude); Forex Currency ($ range a
Read more