Use Python or look for a better tool?

Question

Python

There is a heavy python script that, according to the schedule, automatically performs data manipulations and returns the result to the API. The problem is that the script devours a lot of resources, uses a bunch of third-party libraries.
In general, what is the subject - I want to optimize it. There are 2 ways - dig into the depths of python, optimize the code, think about how to properly manage dependencies, or rewrite in something more specialized like SCALA or GO.
We need community advice

Answer 1

Proceeding from the question and from the consideration of the abundance of libraries as a disadvantage, it is unlikely that the problem is in the programming language. :) Python is quite good for all sorts of scripts and data processing. I think Scala is the thing to think about when the system you want to make becomes so huge and complex that without ubiquitous internal DSLs, type systems, and category theory, you can't survive with it. So try to figure it out the right way.

Answer 2

the rock certainly can increase the efficiency of the script, BUT most likely the language itself will not solve the problem of consuming limited resources. When processing your data, you need to change the approach, it is possible to use streaming data processing, look towards Spark, or mapping engines (as mentioned above).