Answer the question
In order to leave comments, you need to log in
How to increase the speed of reading files from disk?
Reading small (100KB) files from disk in a loop and adding their contents to a list for further processing does not raise hard disk usage above 3MB per second. Is this the limit or can I increase the reading speed?
CrystalDiskMark shows 40 MB, but this is on large files, and small ones, and even fragmented ones, can be read more slowly.
It makes no sense to use several processes, since this only loads the reading head of the hard drive, as I understand it.
But 3 MB? It's somehow implausible.
code like this
while(count<2000):
try:
data = open(path+str(num)+".html").read()
lst.append(data)
except:
pass
Answer the question
In order to leave comments, you need to log in
Oh yes) SSD would oh how it would speed up. I compare the IOPS of SSD and SATA-2 hard drives and salivate. But there is no such money. Yes, and I have 370 GB of files.
OS - WIN 7 x64. HDD - Hitachi SATA-2 2TB.
If there is a lot of RAM - RAMdisk . The 4GB version is free. All data stored on the Ram disk is loaded into RAM, but you can work with them like regular files on your hard drive.
If there are 3-4 thousand rubles. then you can take a 60-120 GB SSD.
Didn't find what you were looking for?
Ask your questionAsk a Question
731 491 924 answers to any question