How to catch a server error when parsing?

Question

Python

I wrote a python + selenium parser, everything works as it should, runs through links, collects all the information (with an open browser).
Sometimes, due to the fault of the server, the parser crashes with an error.
The server responds with 502 (Bad gateway).
The parser crashes , the xpath I need was not found (of course, because an unexpected page was loaded)
Question: how to rewrite the script so that when such an error occurs, the parser does not crash, but makes another attempt (for example, refreshing the page helps), is this possible?

Answer 1

1. You can open links in new tabs. Make several attempts to open the link in a new tab and check if there is an error. More or less like this:

main_window = driver.current_window_handle
#open the link in a new tab
clickable_element.send_keys(Keys.CONTROL + Keys.RETURN)
#switch to that tab
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + Keys.TAB)
driver.switch_to_window(main_window)
#close the tab
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + 'w')
driver.switch_to_window(main_window)

2. You can open a second browser window, I mean new instance, and work with them in parallel. Open all links in the second window.

Answer 2

try:
    #code
except:
    driver.refresh()