P
P
PyTuch2020-03-27 13:42:52
Python
PyTuch, 2020-03-27 13:42:52

How to speed up np.dot()?

Colleagues, good day!
The following interests:
I from a DB receive df in the size of 10 million lines on n columns;
I select one column from this df further;
I make a copy of it consisting of inverse values;
I make the resulting copy as a string;
Then I multiply in a matrix way through np.dot () the column by the formed line.
-------------------------------------------------- -------------------------------------------------- --------------
Chunk - the first way to speed up, to multiply by parts - matrices chunk_size by 10 million
There are other options how it could be accelerated? Perhaps there is some optimal value for the chunk parameter?
chunk_size = 25

df_values ​​= DF.to_numpy()
v = df_values[:, criteria_number - 1:criteria_number]
length = v.shape[0]

kolvo_shagov = int(length / chunk_size) + int(
(length % chunk_size) / ((length % chunk_size - 1)))

b = 1 / v

if vector is None:
vector = np.zeros([v.shape[0], 1])
i = -1
for shag in range(kolvo_shagov):
i += 1
matrix = v[chunk_size * i:chunk_size * (i + 1), :].dot(bT)
gm = stats.gmean((matrix), axis=1)
gm = gm.reshape([len(gm), 1])
vector[chunk_size * i:chunk_size * (i + 1)] = gm

vector_sum = vector.sum()
vector = vector / vector_sum

Answer the question

In order to leave comments, you need to log in

1 answer(s)
U
U235U235, 2020-03-27
@U235U235

Excuse me, but if you multiply a vector by the transposed vector of reciprocals, wouldn't the result be N, the number of elements in the vector?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question