D
D
Denis2016-05-06 13:15:33
Python
Denis, 2016-05-06 13:15:33

The machine freezes during large calculations. How to solve the problem?

Hello.
I'm new to Python and I ran into a problem.
When calculating on large amounts of data, the whole machine freezes.
Specifically, downloaded https://www.continuum.io/downloads for PYTHON 3.5.
I'm running the whole thing in Far-e.
There are two m-tsy:

<60000x1766 sparse matrix of type '<class 'numpy.float64'>'
        with 120000 stored elements in Compressed Sparse Row format>

<60000x22861 sparse matrix of type '<class 'numpy.float64'>'
        with 8365759 stored elements in Compressed Sparse Row format>

from scipy.sparse import hstack
train=hstack([data1,data2]).toarray()

the computer hangs for 15 minutes.
How can I get around this? And it is even better to add, for example, the output of%, of the total number of calculations.
Thanks in advance.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
A
ADRian, 2016-05-06
@den_labs

If the problem is in the processor load, then you can reduce the priority of the thread so that the system does not hang.
But it looks like the whole problem is that you want to create a 11GB array with the .toarray() method

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question