A
A
Andrey Neumyvanny2019-05-30 21:23:19
Django
Andrey Neumyvanny, 2019-05-30 21:23:19

Parsing sites in Python, how to upload the data that you parsed to your site?

I'm not particularly strong in parsing, I'm just trying to master it. Let's say we have a website written in Django. I want to parse 1000 product items, pictures, headings, title from another site. In the Django admin panel, the fields for creating a new product are registered. The task is to insert all the fields that have been parsed into the admin fields. PHP and Drupal have such a thing. For each field you write a script and it parses and inserts into the field, then creates a page. As I understand it, when parsing in Python, data can only be uploaded to a Csv file. What do we do next? Database? Please send. How to parse there is material, but then I don’t understand something

Answer the question

In order to leave comments, you need to log in

2 answer(s)
K
kova1ev, 2019-05-30
@kova1ev

Parse directly into the database, why insert the parsed into the fields of the django admin panel, what a mess. A janga site will simply display information from this database.
upd: if your parser uploads data only in csv and you are too lazy to figure out how to connect to some database in python, do this
- parse, write everything to your csv
- run this database manager for your database, phpmyadmin or some other
- do sql query "COPY table_name FROM 'c:/parser/my_data.csv' (format csv);"
(Of course, the fields in the table must match the csv file)

F
FulTupFul, 2019-05-30
@FulTupFul

The easiest option is to write some string that will write your parsed data to the database:

def save_data():
    file = open("scratch.json", 'r')  # Предположим это json
    data = json.loads(file.read()) # Загружаем json
    file.close()
    for row in data:
        Scratch.objects.update_or_create(title=row['title'], image=row['image'], price=row['price']) # Название модели и полей надеюсь вы сами подставите

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question