S
S
Sergey Bard2017-09-07 10:14:51
Python
Sergey Bard, 2017-09-07 10:14:51

Why can't I change the encoding?

Hello. Started learning python and scrapy. I'm parsing the sitemap like this

# -*- coding: utf-8 -*-
from scrapy.spiders import SitemapSpider
class MySpider(SitemapSpider):
  name = "jobs"
  sitemap_urls = ['http://example.com/sitemap.xml']

  def parse(self, response):
    # print(response.body)
    title = response.xpath('/html/body/div/div[1]/h1/text()').extract_first()
    # y = title.encode("utf-8","ignore").decode("cp1251","ignore")
    # y = title.encode("utf-8","ignore").decode("utf-8","ignore")
    yield {'Title': title}

but it gives me everything in this encoding "Крем РѕС‚ мозолей Рё РСϰтоптышей"
added the line FEED_EXPORT_ENCODING = 'utf-8' to settings.py
but it didn't change anything, I tried to do the same as indicated to the code, i.e. ( title.encode("utf-8","ignore").decode("cp1251","ignore") ), but that didn't help either.
Please tell me what I'm doing wrong.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
D
Derevyanko Alexander, 2017-09-07
@serg_small_developer

As a rule, all site components must support utf-8, both the web server and the DBMS, and the code is written in the same encoding, etc. I suggest you check everything one by one.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question