M
M
Master Ruby2021-08-29 22:01:21
Python
Master Ruby, 2021-08-29 22:01:21

Why does Golang parsing take longer to do the job than Python parsing?

For parsing in Golang I use GoColly:

func main() {
  c := colly.NewCollector()
  start := time.Now()

  // Find and visit all links
  c.OnHTML("div.category-container", func(e *colly.HTMLElement) {
    // e.Request.Visit(e.Attr("href"))

    e.ForEach("div.category-label-block a.category-label-link", func(_ int, tournament *colly.HTMLElement) {
      tournament.Request.Visit(tournament.Attr("href"))

      e.ForEach("div.command div.member-name.nowrap a.member-link", func(i int, name *colly.HTMLElement) {
        name.Request.Visit(name.Attr("href"))
      })

    })

  })

  // c.OnRequest(func(r *colly.Request) {
  // 	fmt.Println("Visiting", r.URL)
  // })

  c.Visit("https://www.marathonbet.ru/su/popular/Tennis")
  finish := time.Since(start)
  fmt.Println(finish)  // 43 секунды
}


Python:

import time

import requests
from bs4 import BeautifulSoup

start = time.time()
url = 'https://www.marathonbet.ru/su/popular/Tennis?cpcids=402555'
# get body response
response = requests.get(url).text
soup = BeautifulSoup(response, 'lxml')
for i in soup.find_all('div', 'category-container'):
    print(i.find('a', 'category-label-link').get('href'))
    for k in soup.find_all('div','bg coupon-row'):
        print(k.find('a', 'member-link').get('href'))
finish = time.time()
print(finish - start)  # 2 сек.


After all, this cannot be, I compiled the code in Golang.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
K
kirillinyakin, 2021-08-29
@Dunaevlad

Judging by the golang code, you visit each page, and in python you just get the href attribute

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question