H
H
HexUserHex2020-05-05 19:51:06
Python
HexUserHex, 2020-05-05 19:51:06

Python dealing with threads?

Good afternoon,
I’m a little bit familiar with threads in python, since I need to add a lot of threading to my very small project .. To do this, I wrote a small example that pings hosts from the list and puts the result into the database, everything seems to work well, but I’m worried for sure I correctly understood the logic of working with threads .. didn’t forget / miss anything?

Two questions:
1. Am I closing streams correctly?
2. Will you have to open the database separately for each thread?
3. Tell me how you can simplify this algorithm.
4. What did I miss to close, check...? I'm looking for general advice...

#!/usr/bin/env python3
import threading, copy, sqlite3, subprocess
from threading import Thread

def connectCreateDb():
    try:
        conn = sqlite3.connect('./filedb.db')
        cursor = conn.cursor()
        cursor.execute("CREATE TABLE IF NOT EXISTS test (result TEXT)")
        conn.commit()
        return conn

    except Exception as e:
        showMessage(3, 'connectCreateDb()'+str(e))


def prescript(domain):
    try:
        #SQLite objects created in a thread can only be used in that same thread.
        conn = connectCreateDb()
        result = subprocess.check_output('ping -c 1 '+domain, shell=True)
        result_ping = str(result).split('\\n')[1]

        cursor = conn.cursor()
        cursor.execute("INSERT INTO test (result) VALUES(?)", (result_ping,) )
        conn.commit()

    except Exception as e:
        showMessage(3, 'prescript()'+str(e))



def main():
    max_threads=1
    list_threads=[]
    domains=['ya.ru', 'google.com', 'mail.ru', 'vk.com', 'gmail.com']

    #Создаю копию списка из которого буду удалять добавленные элементы
    domains_copy = copy.deepcopy(domains)

   #Пока есть более чем один активный поток ИЛИ наши списки ЕЩЕ равны при первом запуске
    while threading.activeCount() >= 1 or len(domains_copy) == len(domains):

        print('basic', len(list_threads) )

        for domain in domains:
            #Проверяю количество потоков
            print('for', len(list_threads), max_threads)

            if len(list_threads) < max_threads:
                print('Create thread for: ', domain)

                thread = Thread(target=prescript, args=(domain,))
                thread.start()

                #Удаляю из копии уже добавленный в поток домен
                domains_copy.remove(domain)
                list_threads.append(thread)


        #Цикл для проверки статуса каждого потока
        for d in list_threads:
            print('Checking: ', d)
            #Если поток отработал то удалить из списка
            if d.is_alive() is False:
                print('Remove thread: ', d)
                list_threads.remove(d)

        #Перезаписываем наш список исключая уже 'пропингованные' домены
        domains = domains_copy


        #Выход как только удалили последний отработавший поток И болше нет доменов для ping-a...
        if len(list_threads) == 0 and len(domains) == 0:
            break

    #Закрываю перед выходом все потоки
    for thread in list_threads:
        thread.join()


if __name__ == "__main__":
    main()

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question