Filling List property of objects in a foreach-loop

Question

Filling List property of objects in a foreach-loop

c# dapper

Question

I'm wondering if filling a list of objects in an object can be done quicker than what i see when i run it. I fill the CABCodes- & CABDetails-lists with a simple query using Dapper with .ToList();

So both the lists are in memory, yet the foreach-operation takes around 20 seconds.

CABCodes: ~10,000 objects

CABDetails: ~60,000 objects

List<CABCode> CABCodes = new List<CABCode>();
List<CABDetail> CABDetails = new List<CABDetail>();

public class CABCode {
    public int Sequence { get; set; }
    public string Code { get; set; }
    public int Group { get; set; }
    public List<CABDetail> Details { get; set; }
}
public class CABDetail {
    public int CABSequence { get; set; }
    public int Proptype { get; set; }
    public string Propvalue { get; set; }
}

foreach (var c in this.CABCodes) {
    c.Details = this.CABDetails.Where(x => x.CABSequence == c.Sequence).ToList();
}

Is there a far more efficient method of accomplishing this?

Sigurd BÃ¸asÃ¦ter

Popular Answer

Linq is not always as fast as writing own loops with if/else statements. The second thing is, that ToList() create a new instance of an List, which also slow down your performance.

How is the performance with this code:

foreach (var c in this.CABCodes) 
{
  var detailList = new List<CABDetail>();
  foreach(var d in CAPDetails)
  {
    if (d.CABSquence == c.Sequence)
    {
      detailList.Add(d);
    }
  }
  c.Details = detailList;
}

maybe i've done some mistakes in the code, becuase i wrote in on-the-fly.

BendEg

View more on Stack Overflow

George Polevoy · Accepted Answer · 2/25/2016 11:48:02 AM

You have O(M*N) algorithmic time. Following code makes it O(M + N), which is fast:

var cabDetailsBySequence = CABDetails.ToLookup(d=>d.CABSequence);

foreach (var c in this.CABCodes) {
    c.Details = cabDetailsBySequence[c.Sequence].ToList();
}

UPDATE: I've checked it works under 110ms, having 100 different Sequence codes.

Here is the test setup:

CABCodes = Enumerable.Range(0, 10000).Select(i=>new CABCode{Sequence = i%100}).ToList();
CABDetails = Enumerable.Range(0, 60000).Select(i=>new CABDetail{CABSequence = i%100}).ToList();

UPDATE 2: And you can make it even faster, (20 times aprox), if you don't mind having references to the same lists in different CABCode instances, and even faster if you do it in parallel. This way it runs under one millisecond on my 8 core system:

var cabDetailListsBySequence = cabDetailsBySequence.ToDictionary(i=>i.Key, i=>i.ToList());

//  foreach (var c in this.CABCodes) {
//      c.Details = cabDetailListsBySequence[c.Sequence];
//  }   

this.CABCodes.AsParallel().ForAll(c=>c.Details = cabDetailListsBySequence[c.Sequence]);

Filling List property of objects in a foreach-loop

Question

Fastest Dapper Plus Extensions

Accepted Answer

Popular Answer

Performance

Expression Evaluator