How to read a large file in python?

Question

Python

There is a log weighing 2GB, you need to calculate the statistics for one of the fields, however, when reading it, an error occurs list index out of range, what should I do?

Answer 1

Here's some inspiration for you:

#!/usr/bin/python2.7
# -*- coding: utf-8 -*-

from collections import namedtuple

ntrow = namedtuple('ntrow', "f0, f1, mydata, f3")

with open('data.log', 'r') as f:
    for line in f:
        splitted = line.strip().split(', ')
        if len(splitted) == 4:
            entry = ntrow(*splitted)
            if entry.mydata.isdigit():
                print int(entry.mydata)

File content:
oweifoewi, oiud, 873, sdioju oweifoewi ,
oiud, 2