LINQ or foreach?

Question

Programming

Good afternoon, fellow workers!
I write highly loaded services both for the corporate sector and for the mass consumer.
Technology stack: from pure Python-Perl-PHP to .NET and Java, not counting all sorts of 1C, SAP and other highly specialized software.
Recently, I began to ask myself the question: have we, programmers, gone far in terms of making our work easier?
I, as a fan of Entity Framework, Doctrine (and not only ORM in general), jQuery (XQuery and the like), within reasonable limits, of course, cannot get rid of the idea that making my work easier (templaters, frameworks with code generation, various helpers of all stripes) we load servers that are already a little expensive (I'm not even talking about the cost of hardware, but about the cloud infrastructure of AWS, Azure, Google, DigitalOcean, here Yandex pleased me with Cloud prices at a level almost like that of AWS)?
Abstraction from reality in the field of programming in the last 10 years has led to the fact that soon most programmers will simply forget about the stack, heap, optimization of SQL queries, because MS, Oracle, Zend and independent developers of software development tools simply go towards non-optimal code and purchasing servers on a space scale...
Here, for example, LINQ or its analogues in other languages... The speed of processing queries on LINQ is significantly (many times) lower than direct enumeration of collections, however, we, developers, implement all sorts of smart Parallel LINQ or start deal with thread splitting instead of not writing a LINQ query, but simply iterating over a collection ...
The same applies in some way to jQuery - after all, you can write everything in pure JS, using jQuery where it is needed.
It always seemed to me that using technology for the sake of code readability is absurd, look at my codes on Delphi 5-6-7 15 years ago, everything is clear and readable there, the code is supported, tests pass, but there is not even an ORM approach there , in general, half was done on stored proc, half on direct SQL, and I’m generally silent about enumeration of collections in Object Pascal of those years)
At the risk of incurring tons of hatred and rays of a brown rainbow, I will nevertheless draw a conclusion based solely on my practice: does the principle “it’s better to lose a day, then fly in 5 minutes” does not work in our industry? In most cases in my practice, when I received some code for revision or participated in a tender for a contract, my competitors gave out such astronomical amounts for the purchase of server capacities that I “made” them only on the cost of servers: my calculations for servers turned out to be 2-3-5 sometimes 10 times less than competitors' calculations precisely because of the archaic nature of my approaches to saving on the side of the source code and creating a balance in the solution, in order, first of all, to save the customer's money both now and at the project support stage.
By the way, if someone says that the code on LINQ and other “life-simplifiers” is easier to test, then I might agree, but after all, tests are written by programmers, even if they are called testers ...
A programmer, it seems to me, is primarily an engineer, and the engineering approach provides for the creation of an economically optimal solution.
This does not mean that I am against progress, but “the best is the enemy of the good” should also be taken into account and not rush to all the newfangled chips, especially if they are imposed by the server capacity manufacturer (for example, MS with its LINQ)
So, we will continue to follow the lead of geeks MS and Google, creating frameworks and other “convenient” solutions for rapid development, or will we learn to “keep the balance” ourselves?
By the way, for those who like to bring everything to the absolute, I’ll say right away: I don’t think that since I’m so smart, then everything should be written in C and ASM, but rather on punched cards, which is already there ...
I emphasize once again, the engineering approach! We save customer money.
Am I wrong?

Answer 1

1. This situation arises from the fact that a business usually needs to be ready yesterday, and it does not care what crutches will be in the code, in the worst case, it will not even think about the complexity of supporting all this.
2.

A programmer, it seems to me, is first of all an engineer, and the engineering approach provides for the creation of an economically optimal solution.
You envisage the creation of an optimal solution in terms of development, support and performance. From an economic point of view , a product made from crutches in a week is more often optimal for a business than a product of high quality in a month.
All these "fashionable" chips that you describe just exist to increase the speed of development. Yes, and money is needed, all of a sudden, not only for iron.
https://habr.com/company/badoo/blog/430722/ - a good article from badoo, with conclusions about the expediency of optimizing the code / adding servers

Answer 2

in the last 10 years, it has been going to the fact that soon most programmers will simply forget about the stack, heap, SQL query optimization

Most programmers have never been good at optimizing SQL queries.

“it is better to lose a day, then fly in 5 minutes” does not work in our industry?

Business looks where the profit is. If he spends two days without this scheme, then yes, it is better to lose and fly. If he spent 10 minutes without this scheme, and flew once a month, then the five-minute savings will pay off in 288 "flights" (in 24 years). Does he really need it?
80 years ago, almost every driver knew how to auto mechanic. And now, driving, you can increasingly meet people who cannot distinguish a washer reservoir from oil, for example.
This does not mean that no one in the world needs a highload and good code. It's just that it's not always beneficial for the mass market, it's the norm.

Answer 3

From the point of view of economics, what is better:
- look for a team of talented programmers for 12 months, who can not write a project in a year
- find a team of average and bad developers and rivet a solution on frameworks in 6 months
?
About the cost of iron / performance.
If your set of tools does not give a huge increase in development speed, or the set itself does not optimize the code, or the set does not force you to write optimal code, then you should change it
AND the name of the set does not matter

Answer 4

I don’t see a problem, loaded systems are written so often, but a simple application that should solve a specific problem ..
it’s cheaper to stick a server next to it, when the price for the server becomes much more expensive than the price for development, then they start optimizing

Answer 5

The speed of processing queries on LINQ is significantly (several times) lower than direct enumeration of collections, however, we, developers, are implementing all sorts of smart Parallel LINQ or starting to deal with thread splitting instead of not writing a LINQ query, but simply iterating over the collection

Why? If some solution is more profitable, then we use it. Parallel query processing is needed where exactly this is needed, and not "don't care how, but I want fast LINQ". Such logic can only mean a low, very low qualification of the individual.
Not from a better life. And even now, if you take it, the same Dapper is quite popular. Which also does not prevent combining EF with Dapper in one application. Perhaps you did not come across tasks where the game would be worth the candle.
In contrast, functional call chains in debugging are more complex than imperative code.
PS The developer must:
The question raised is purely technical. And all these: "Put another server, they're cheap!" - means a fakup of a business that initially saved on competent specialists, and now is forced to plug holes with money due to incorrect technical solutions. Such offices should be avoided, and colleagues who support the idea should be driven with rags and mocked.

Answer 6

we load the servers, which are already expensive

Developer time is infinitely more valuable and more important than paying a few extra dollars for a server.

Answer 7

The natural process, alas, cannot be avoided anywhere....
So not only in programming, but in everything. You don’t need to know the entire course of physics and electrical engineering to screw in a light bulb ... You don’t need to know how a carburetor / injector works to drive a car ...
And again, even among electricians there are those who are able to calculate the load on the network, which is then there’s still (I don’t understand anything about this), but there are those who can “choke” wires into the wall and put sockets with automatic machines. But if there weren’t such people, then calling an electrician "with education" would cost us a round penny.
Damn economics.... hateful word for a perfectionist :)
Don't worry like that! Good luck!

Answer 8

Well, as it were, linq ef also needs to be used wisely.
https://habr.com/post/269901/

Answer 9

we load the servers, which are already expensive

It's okay, they will buy more memory, and a more powerful processor will appear in a year. The servers last, the manufacturer makes a profit, the programmers get more automation, and everyone is happy.