I'm wondering if filling a list of objects in an object can be done quicker than what i see when i run it. I fill the CABCodes- & CABDetails-lists with a simple query using Dapper with .ToList();
So both the lists are in memory, yet the foreach-operation takes around 20 seconds.
CABCodes: ~10,000 objects
CABDetails: ~60,000 objects
List<CABCode> CABCodes = new List<CABCode>();
List<CABDetail> CABDetails = new List<CABDetail>();
public class CABCode {
public int Sequence { get; set; }
public string Code { get; set; }
public int Group { get; set; }
public List<CABDetail> Details { get; set; }
}
public class CABDetail {
public int CABSequence { get; set; }
public int Proptype { get; set; }
public string Propvalue { get; set; }
}
foreach (var c in this.CABCodes) {
c.Details = this.CABDetails.Where(x => x.CABSequence == c.Sequence).ToList();
}
Is there a far more efficient method of accomplishing this?
You have O(M*N) algorithmic time. Following code makes it O(M + N), which is fast:
var cabDetailsBySequence = CABDetails.ToLookup(d=>d.CABSequence);
foreach (var c in this.CABCodes) {
c.Details = cabDetailsBySequence[c.Sequence].ToList();
}
UPDATE: I've checked it works under 110ms, having 100 different Sequence codes.
Here is the test setup:
CABCodes = Enumerable.Range(0, 10000).Select(i=>new CABCode{Sequence = i%100}).ToList();
CABDetails = Enumerable.Range(0, 60000).Select(i=>new CABDetail{CABSequence = i%100}).ToList();
UPDATE 2: And you can make it even faster, (20 times aprox), if you don't mind having references to the same lists in different CABCode instances, and even faster if you do it in parallel. This way it runs under one millisecond on my 8 core system:
var cabDetailListsBySequence = cabDetailsBySequence.ToDictionary(i=>i.Key, i=>i.ToList());
// foreach (var c in this.CABCodes) {
// c.Details = cabDetailListsBySequence[c.Sequence];
// }
this.CABCodes.AsParallel().ForAll(c=>c.Details = cabDetailListsBySequence[c.Sequence]);
Linq is not always as fast as writing own loops with if/else statements. The second thing is, that ToList() create a new instance of an List, which also slow down your performance.
How is the performance with this code:
foreach (var c in this.CABCodes)
{
var detailList = new List<CABDetail>();
foreach(var d in CAPDetails)
{
if (d.CABSquence == c.Sequence)
{
detailList.Add(d);
}
}
c.Details = detailList;
}
maybe i've done some mistakes in the code, becuase i wrote in on-the-fly.