Answer the question
In order to leave comments, you need to log in
How to do parallel reading from a file, Python+Linux(GNU_parallel)?
Good day, dear ones!
There is a primitive bash script that reads a file: cat -v /dev/pts/2 | python3 'process.py'
Where is the python3 script:
import sys
for string in sys.stdin:
some_calculation(string)
cat -v /dev/pts/2 | parallel --pipe --recstart '>' python3 process.py
cat -v /dev/pts/2 | parallel --j 3 --pipe --block 12M python3 process.py
Answer the question
In order to leave comments, you need to log in
solution found, just play around with the --block option:
cat -v /dev/pts/2 | parallel --j 3 --pipe --block 12M python3 process.py
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question