Why doesn't it find the page with requests, but finds it with cURL and Go?

Question

Python

I make a request to a page in python using requests - the server responds with a page with the title "page not found"
the same but cURL - I
tried everything fine GO - I also get the necessary data, even if I don't send headers and coockie
example url
in python:

import requests 
import coockie

main_url = coockie.host
def main():
     r = requests.get(main_url, headers=coockie.headers, cookies=coockie.cookies)
     return bs(r.text ,'lxml')

on Go:

package main

import (
  "fmt"
  "io/ioutil"
  "log"
  "net/http"
)

func main() {
  client := &http.Client{}
  req, err := http.NewRequest("GET", "https://ks-rus.com/catalog/zadvizhki_shibernye/", nil)
  if err != nil {
    log.Fatal(err)
  }
  req.Header.Set("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:86.0) Gecko/20100101 Firefox/86.0")
  req.Header.Set("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8")
  req.Header.Set("Accept-Language", "ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3")
  req.Header.Set("Connection", "keep-alive")
  req.Header.Set("Referer", "https://ks-rus.com/")
  req.Header.Set("Upgrade-Insecure-Requests", "1")
  req.Header.Set("TE", "Trailers")
  req.Header.Set("Cookie", "BITRIX_SM_SALE_UID=многабукав")
  resp, err := client.Do(req)
  if err != nil {
    log.Fatal(err)
  }
  bodyText, err := ioutil.ReadAll(resp.Body)
  if err != nil {
    log.Fatal(err)
  }
  fmt.Printf("%s\n", bodyText)
}

cURL

curl 'https://ks-rus.com/catalog/zadvizhki_shibernye/' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:86.0) Gecko/20100101 Firefox/86.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' -H 'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3' --compressed -H 'Connection: keep-alive' -H 'Referer: https://ks-rus.com/' -H 'Cookie: BITRIX_SM_SALE_UID=многабукав' -H 'Upgrade-Insecure-Requests: 1' -H 'TE: Trailers'

there is a suspicion that this is all due to the fact that requests https cannot

Answer 1

requests https can't

a?
Even without headers and cookies, it gets exactly the same code as in the browser.

spoiler

cookie, не coockie

Answer 2

Write user agent for python as for the rest. On the server most likely the rule forbidding requests for any bots.