L
L
LordOftheCode2021-09-17 16:41:51
Python
LordOftheCode, 2021-09-17 16:41:51

The script does not send complete information from the parsing?

I'm parsing a site with stopg games
. Everything is fine for me, all games and information are displayed in the console, etc.
and when I try to write this information to a .json file, the last entry is written, how can I make sure that all the parsed information is recorded?

def get_first_news():
    url = 'https://stopgame.ru/review/new'
    r = requests.get(url)

    soup = BeautifulSoup(r.text, 'html.parser')

    articles_cards = soup.find_all('div', {"class": "item article-summary"})


    news_dict = {}
    for article in articles_cards:
        article_url = "https://stopgame.ru" + article.find('a', {"class": "article-image image"}).get("href")
        article_date_time = article.find("span", {"class": "info-item timestamp"}).text
        article_title = article.find("div", {"class": "caption caption-bold"}).text.replace("\n", "")
        print(f'{article_title} | {article_url} | {article_date_time}')

        article_id = url.split('/')[-2] + '/' + url.split('/')[-1]

        news_dict[article_id] = {
            "article_url" : article_url,
            "article_date_time" : article_date_time,
            "article_title" : article_title
        }

    with open("news_dict.json", "w") as file:
        json.dump(news_dict, file, indent=4, ensure_ascii=False)

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
soremix, 2021-09-17
@LordOftheCode

Wrong received article_id, it is the same on each iteration of the loop, so the value in the dictionary is simply overwritten each time

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question