S
S
stayHARD2015-10-27 19:19:36
Python
stayHARD, 2015-10-27 19:19:36

How to fix Segmentation fault(core dump)?

Good evening.
I catch an error - Segmentation fault(core dump)
Googling gives a lot of different options for the origin of this error. I don’t know what happens in this case, I couldn’t debug either (

from queue import Queue
from threading import Thread
from grab import Grab

def do_work(link):
    g = Grab()
    g.setup(timeout=30, connect_timeout=5)
    g.go(link)
    print(q.qsize())

def worker():
    while True:
        link = q.get()
        do_work(link)
        q.task_done()

q = Queue()
num_worker_threads = 50

for i in range(num_worker_threads):
    t = Thread(target=worker)
    t.daemon = True
    t.start()
list_urls = [
# big list with urls
]
for item in list_urls:
    q.put(item)

q.join()

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question