T
T
trauus2019-03-07 17:08:09
Entity Framework
trauus, 2019-03-07 17:08:09

Why does EntityFramework gobble up memory on many simple queries?

I am mastering EntityFramework 6.2 in conjunction with SQLite in a desktop application on .NET 4.
At the first start, you need to download a list of 20,000 products from the server and write them to a local database, checking for uniqueness in one of the fields.
When debugging, the application began to crash with OutOfMemoryException, it turned out that processing every 700 elements eats up ~ 100 MB of memory. The application is 32-bit, so memory runs out before all elements are processed.
In the process of debugging, I came to a minimal piece of code that reproduces the situation.
Why is running several hundred queries db.Products.Where(pr => pr.SomeUniqueId == p.SomeUniqueId).FirstOrDefault(); even on an empty table so eats up memory?
And what to do with it?

[TestMethod]
public void MemoryUseTest()
{
  using (DbContext db = DbContextFactory.Create())
  {
    db.Configuration.AutoDetectChangesEnabled = false;
    db.Configuration.LazyLoadingEnabled = false;
    db.Configuration.ProxyCreationEnabled = false;
  
    var products = CreateRandomProducts(2000);		

    foreach (var p in products)
    {
      Product existing = db.Products.Where(pr => pr.SomeUniqueId == p.SomeUniqueId).FirstOrDefault();
    }
  }	
}

public class Product
{
  public int Id { get; set; }

  public String UniqueId { get; set; }
  public String FullName { get; set; }
  public decimal Capacity { get; set; }
}

private List<Product> CreateRandomProducts(int count)
{
  List<Product> prods = new List<Product>();

  for (int i = 0; i < count; i++)
  {
    Product p = new Product();

    p.Code = Guid.NewGuid().ToString();
    p.UniqueId = Guid.NewGuid().ToString();
    p.FullName = Guid.NewGuid().ToString();
    p.Capacity = 11.0m;

    prods.Add(p);
  }
  return prods;
}

Answer the question

In order to leave comments, you need to log in

1 answer(s)
C
CHolfield, 2019-03-07
@CHolfield

you the pervert, in a cycle on one to select records when the ready list is.
instead of

foreach (var p in products)
    {
      Product existing = db.Products.Where(pr => pr.SomeUniqueId == p.SomeUniqueId).FirstOrDefault();
    }

need to do
var listOfSomeUniqueId = products.Select(a => a.SomeUniqueId);
var listOfExisting = db.Products.Where(b => listOfSomeUniqueId.Contains(b.RoleId));

and then bypass the finished list of records existing in the database
foreach (var p in listOfExisting)
    {
      p.SomeProperty1 = "HuiPizdaDzhigurda";
      p.SomeProperty2 = 123;
    }

and in the end
compare the speed with your perversions.
by the way, I almost forgot to answer your questions:
1. because every time the entire db.Products table is taken into memory and one record is searched for in it, the garbage collector does not have time to clear the memory, and you yourself did not attend to this issue.
2. Think with your head.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question