Z
Z
zzz42016-12-12 20:07:23
MySQL
zzz4, 2016-12-12 20:07:23

How to parse mysql database using python3?

Please advise a newbie on the following question. Level of knowledge - the book "Simple Python. Modern style of programming" Lubanovich. And that's it :(
Python 3.5.2, PyMySQL (0.7.9),
Ubuntu
16.04 downloads files from the given links
ID:integer increment
URL: string
Name: string
Path: string
Downloaded: tinyint 1
NotFound: tinyint 1
Script task : Connects
to the storage in a loop using the URL field,
saves the file in the Path with the name Name and sets the Downloaded flag Download
errors:
NotFound - set NotFound flag
Exception - An unknown storage error. We do not set the flag, we try again when the script is restarted
Structure errors:
In order not to create a structure initially, it is easier to validate it and create it on the go.
That is, you need to check the Path for the presence, if not - create
Can someone at least give a vector, what needs to be done and in what sequence? At least step by step, what needs to be done, and I myself would google code examples. :)
I was able to connect to the database, but I don’t understand what to do next:

import pymysql

db = pymysql.connect(host='localhost', port=, user='root', passwd='', db='Pictures')

cursor = db.cursor()
cursor.execute("SELECT * FROM URL")

Answer the question

In order to leave comments, you need to log in

2 answer(s)
R
Roman Mindlin, 2016-12-13
@zzz4

1. Read this book https://dmkpress.com/catalog/computer/programming/...
Most of your questions will disappear.
2. In essence:

try:
    cursor.execute("SELECT * FROM index WHERE complete = FALSE")
except Exception as e:
    print(e)
    return None

Chose what hasn't been downloaded yet
import requests
session = requests.Session()

rows = cursor.fetchall()
for row in rows:
    print("open %s"%row[1])
    try:
        req = session.get(row[1])
#скачал что надо, обработал
    except AttributeError as e:
        print(e)
        return None

here download, pull out the necessary data
try:
    cursor.execute("INSERT INTO pages (fields) VALUES (%s)", ("что там напарсил",))
except Exception as e:
    print(e)
    return None

Parse and write to the database
try:
    cursor.execute("UPDATE index SET complete = TRUE WHERE id = %s",(row[0],))
except Exception as e:
    print(e)
    return None

We note that we managed to download . We complete the
transaction
. The train of thought is something like this. The code for postgres, for muscle may be slightly different.

R
Roman Mirilaczvili, 2016-12-13
@2ord

  1. Draw a block diagram of the algorithm at the functional level, without going into implementation details.
  2. When working with DBMS - SQL language to help.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question