I
I
Igor2020-06-01 13:03:20
Python
Igor, 2020-06-01 13:03:20

How to write a parser?

The question is as follows.
There is a log, you need to make a parser.

1945827 xx.xxx.xxx.xxx TCP_TUNNEL/200 6811 CONNECT mail.ru:443 username HIER_DIRECT/xxx.xxx.xxx -

How to make a parser?
Found something like this on the internet
# -*- coding: utf-8 -*-
import datetime

f = open('/var/log/squid/access.log', 'r',encoding='UTF-8')

for line in f.readlines():
    try:
        dt = str( datetime.date.fromtimestamp( float( line.split( )[0][0:10] ) ) )
    except:
        dt = u'Error'
    
    try:
        ip = line.split( )[2]
    except:
        ip = u'Error'
    
    try:
        bytes = line.split( )[4]
    except:
        bytes = u'Error'
    
    try:
        miss = line.split( )[3]
        if miss == 'NONE/400':
            query = line.split( )[3]
            
        else:
            query = line.split( )[6]
    except:
        query = line.split( )[3]
    
    if dt != 'Error' and ip != 'Error' and bytes != 'Error' and miss != 'NONE/400':
        print ( dt + ' - ' + ip + ' - ' + bytes + ' - ' + query )
    else:
        continue
    

f.close()

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
ScriptKiddo, 2020-06-01
@ScriptKiddo

Use Grock. This is RegExp with goodies
https://pypi.org/project/pygrok/

from pygrok import Grok
text = 'gary is male, 25 years old and weighs 68.5 kilograms'
pattern = '%{WORD:name} is %{WORD:gender}, %{NUMBER:age} years old and weighs %{NUMBER:weight} kilograms'
grok = Grok(pattern)
print grok.match(text)

# {'gender': 'male', 'age': '25', 'name': 'gary', 'weight': '68.5'}

Grok for Squid is very easy to google. Or better yet, write your own

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question