A
A
Alexander Shilov2018-11-20 09:05:57
Python
Alexander Shilov, 2018-11-20 09:05:57

How to parse large text into a string?

Good day, toasters)
And I'm interested in such a question that I've been struggling with for 3 days and I can't overcome it.
There is the following code:

def get_html(url):
  html = requests.get(url)
  return html.text
def get_descriptions(html):
  description = ''
  soup = bs(html, 'lxml')
  description = soup.find('div', class_='item-description').text
  return description
file_desc = open('file.txt', 'a')
url = 'Какой-то сайт'
html = get_html(url)
result_descriptions = str(get_descriptions(html)).replace('\n', ' ').replace('@', '')
file_desc.writelines('{}\n'.format(result_descriptions))
file_desc.close()

The thing is that from some pages it takes a description and writes it down, and from some pages it gives the following error.
Picture with error
5bf3a400ba1be716426007.png

Tried like this:
file_desc.writelines('{}\n'.format(result_descriptions[0:450]))

But it helps in rare cases. I want everything to always work properly.
Help, please, who than can.

Answer the question

In order to leave comments, you need to log in

3 answer(s)
J
JRazor, 2018-11-20
@tabbols95

On Habré there is an article Unicode for Dummies
Must read for all pythonists and not only

A
alternativshik, 2018-11-20
@alternativshik

description .encode('utf-8')

N
Nikolay, 2018-11-21
@EarlAthos

file_desc = open('file.txt', 'a', encoding='utf-8')

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question