How to fix error on Heroku server?

Question

Python

Hello. Everything works fine locally. On the server it gives an error:

2022-02-06T21:22:11.274882+00:00 app[worker.1]: Traceback (most recent call last):
2022-02-06T21:22:11.274885+00:00 app[worker. 1]: File "/app/main.py", line 72, in
2022-02-06T21:22:11.275081+00:00 app[worker.1]: bot.polling(none_stop = True, interval = 0, timeout = 100) # start bot, pass time so bot doesn't crash
2022-02-06T21:22:11.275088+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.9/ site-packages/telebot/__init__.py", line 427, in polling
2022-02-06T21:22:11.275272+00:00 app[worker.1]: self.__threaded_polling(none_stop, interval, timeout)
2022-02-06T21:22:11.275280+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.9/site-packages/telebot/__init__.py", line 451, in __threaded_polling
2022-02-06T21:22:11.275475+00:00 app[worker.1]: self.worker_pool.raise_exceptions()
2022-02-06T21:22:11.275489+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.9/site-packages/telebot/util.py", line 111, in raise_exceptions
2022-02-06T21:22:11.275604+00:00 app[worker.1] : six.reraise(self.exc_info[0], self.exc_info[1], self.exc_info[2])
2022-02-06T21:22:11.275613+00:00 app[worker.1]: File "/app /.heroku/python/lib/python3.9/site-packages/six.py", line 719, in reraise
2022-02-06T21:22:11.275874+00:00 app[worker.1]: raise value
2022-02-06T21:22:11.275883+00:00 app[worker.1]: File "/app/.heroku/python/lib/python3.9/site-packages/telebot/util.py", line 62, in run
2022-02-06T21:22:11.275962+00:00 app[worker.1]: task(*args, **kwargs)
2022-02-06T21:22:11.275970+00:00 app[worker.1] : File "/app/main.py", line 61, in callback_Func
2022-02-06T21:22:11.276104+00:00 app[worker.1]: rez = func(int(call_id_page))
2022-02-06T21 :22:11.276118+00:00 app[worker.1]: File "/app/main.py", line 24, in func
2022-02-06T21:22:11.276289+00:00 app[worker.1]: rez_news = [f' {new[2]} | [{new[1]}]({new[-1]}) - {new[0]}' for new in pd.parse_subject("news", id_page)]
2022-02-06T21:22:11.276306+00 :00 app[worker.1]: TypeError: 'NoneType'

def func(id_page):
    rez_news = [f' {new[2]} | [{new[1]}]({new[-1]}) - {new[0]}' for new in pd.parse_subject("news", id_page)]

parsing:

# -------- отбор нужной страницы с новостями ---------
 ls_news = []
 xpath_for_new = [[f'//*[@id="newsList"]/div[{id}]/div/div[1]/p[2]', f'//*[@id="newsList"]/div[{id}]/div/div[1]/p[1]',
                    f'//*[@id="newsList"]/div[{id}]/div/div[2]/a'] for id in range(2, 11)] 
 if int(id_page) <= int(max_kol_pages): 
     for new in xpath_for_new: # распаковка списка списков
           info_news = [driver.find_element_by_xpath(info).text for info in new]    
            url_post = driver.find_element_by_xpath(new[-1]).get_attribute("href") # забираю ссылку из арибута href
            ls_news.append([max_kol_pages] + info_news + [url_post])    
            print("\n\n--- %s seconds (/news) ---" % (time.time() - start_time))
            return ls_news

Exception handling doesn't help, the program crashes...

Answer 1

I am 100% sure that the site you are trying to scrape is protected from this by blocking the addresses of free hosts, including Heroku.