In BeautifulSoup, when parsing, instead of text, some parameters are displayed in {{}}. How to fix it?

Question

Python

I have a task: to parse special offers from the AVON website.
When executing this code:

import requests
from bs4 import BeautifulSoup as BS
url = "https://my.avon.ru/specialnye-predlozhenija/19/vse-spetspredlozheniya"
r = requests.get(url)
html = BS(r.content, "html.parser")
obj = html.find('a', attrs={"class": "SpecialOfferTitle"})
print(obj)

tag text displays {{::MaxText(specialOffer.Headline, (Lay...

Answer 1

We press F12 and carefully study the requests (especially xhr). After carefully studying, we make a script request to the address " my.avon.ru/api/specialoffersapi/getcategorypromotions?promoCategoryId=19 ", we get the finished json and work with it.

Answer 2

Most likely, these attributes are displayed after working out JS scripts, and requests cannot handle this. You can try using Selenium

Answer 3

Content is rendered via JS.
See the source code only through Ctrl+U.
To find where the information you need comes from, press F12 -> Network -> refresh the page. In the XHR tab, see what requests are sent and what responses they receive. Find the one you need, repeat it through requests
In your case, the URL will be

https://my.avon.ru/api/specialoffersapi/getcategorypromotions?promoCategoryId=19