E
E
EVG822022-04-16 13:33:31
Python
EVG82, 2022-04-16 13:33:31

How to fix parsing error?

I wrote a familiar site parsing script and disappeared from the connection.
In general, during the test, such an error occurs

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page

document This error happens to many people.

Only now I did not understand how to apply the solution.
Somehow the code needs to be supplemented so that it ignores this error.

Here is the code

class Google:
    dir = os.path.dirname(__file__)
    driver = webdriver.Chrome(os.path.join(dir, driver_name))
    driver.maximize_window()
    time = 0.3
    was = []

    def level(self, text, element):
        if element.text.split('\n')[0] in self.was:
            return
        else:
            self.was.append(element.text.split('\n')[0])
        text.append(element.text.split('\n')[0])
        try:
            element.find_element(By.XPATH, './/ul')
        except NoSuchElementException:
            with open('text.txt', 'a') as f:
                print(element.tag_name, element.text, text)
                f.write('/'.join(text)+'\n')
        else:
            pluses = element.find_elements(By.XPATH, './/*[starts-with(@id,"plus_")]')
            for plus in pluses:
                try:
                    plus.click()
                    time.sleep(self.time)
                except ElementNotInteractableException:
                    continue
                except ElementClickInterceptedException:
                    plus.click()
                    time.sleep(self.time)
            elements = element.find_elements(By.XPATH, './/li')
            for el in elements:
                # print(el.tag_name, el.text, element.text)
                self.level(text, el)
        text.pop(-1)

    def parse(self, url):
        with open('text.txt', 'w') as f:
            pass
        self.driver.get(url)
        text = []
        pluses = self.driver.find_elements(By.XPATH, '//*[starts-with(@id,"plus_")]')
        for plus in pluses:
            plus.click()
            time.sleep(self.time)

        elements = self.driver.find_elements(By.XPATH, '//li')
        for element in elements[7:]:
            self.level(text, element)

        time.sleep(1)


Tell me where to add the code recommended here?
https://habr.com/ru/post/573144/

Answer the question

In order to leave comments, you need to log in

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question