The maximum number of lines in a file for normal reading?

Question

Django

Tell me please. I have a file generated from database data.
Each line looks like this:

2431387 ce/ce41395ee6b9cd31a89db1fb60646c.JPG ce/thumbs/ce41395ee6b9cd31a89db1fb60646c.JPG old-2015-1/1/7\n

(Number path1 path2 path3)
The file weighs ~ 900 mb, it contains ~5.9 million lines
I read only with the help of readline()
Should it be divided into parts, and if so, into how many parts should it be divided? (The script transfers images, will run on the server (~10-15 GB of RAM is free, so there are only restrictions on hdd + cpu))

Answer 1

All you need is:

with open("log.txt") as infile:
    for line in infile:
        do_something_with(line)

We read one line at a time without loading the entire file into memory.