How to get a list of reviews about an organization from Yandex Maps?

Question

Yandex maps

Good day!
It was necessary to implement the loading of reviews on the organization's website from Yandex maps. I did not find such an API method in the documentation and it was useful to see how the cards themselves receive them, I found such an AJAX request that is sent when loading the Yandex page with the selected organization:

https://yandex.ru/maps/api/business/fetchReviews?csrfToken=<csrf-токен>&ajax=1&id=<ID_организации>&page=1

But the problem is that this method is tied to a CSRF token, and if it is incorrect, then the result will be a 403 code and an actual token. If you take the received new token in the browser and substitute it in the link, then everything is fine, I get the list of reviews in JSON format. But if you do the same programmatically, then a new token is generated for each get request.
Example (top - received token with a deliberately erroneous request, bottom - the result of a request with this token):

I understand why and why it works this way - in the case of this api method. It is not clear how you can get a list of reviews programmatically (and is it possible in general to implement this without shamanism)? It is necessary to do this on the server side, because the reviews themselves will be pulled into the site database for moderation, etc.

Answer 1

Example implementation in php:

function get_request_result($request)
           {
                $ch = curl_init();
                 curl_setopt($ch, CURLOPT_URL, $request);

        
                /*Храним сеанс в сессии*/
                curl_setopt($ch, CURLOPT_COOKIEFILE, true);
                curl_setopt($ch, CURLOPT_COOKIEFILE, "cookiefile");
                curl_setopt($ch, CURLOPT_COOKIEJAR, "cookiefile");
                /**/

                curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
                $server_output = curl_exec($ch);

                curl_close($ch);

                return json_decode($server_output);
            }

$request_token = "https://yandex.ru/maps/api/business/fetchReviews";
$token = get_request_result($request)->csrfToken;
$request_reviews = "https://yandex.ru/maps/api/business/fetchReviews?ajax=1&businessId=id_организации&csrfToken={$token}&page=1&pageSize=5";
$reviews_obj =  get_request_result($request_reviews);

Then you can programmatically get a list of reviews.

Answer 2

The AJAX request stopped working, so I wrote a parser for the page
https://yandex.ru/maps/org/{organization_id}/reviews/
First, let's write the simplest function for a cURL request

function getRequestResult($request){
    $ch=curl_init();
    curl_setopt($ch,CURLOPT_URL,$request);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $server_output = curl_exec($ch);
    curl_close($ch);
    return $server_output;
}
$org_url='https://yandex.ru/maps/org/{&organization_id}/reviews/';
$html=getRequestResult($org_url);

Got a huge HTML page, let's trim it. We need the content of the div with the class reviews-view__reviews .

$pos=strpos($html,'<div class="reviews-view__reviews">');
$part=substr($html,$pos);
$pos2=strpos($part, '</section>');
$part=substr($fx,0,$pos2);

Further, using any HTML parser, we achieve the desired result. I used Simple HTML DOM

require_once("simple_html_dom.php");
$html=str_get_html($part);
if(count($html->find('.business-review-view'))){
    foreach($html->find('.business-review-view') as $div){
        //Значит, нашли отзыв
        $review=[];
        //Ищем автора
        if(count($div->find('.business-review-view__author a'))){
            $reviewAuthor=$div->find('.business-review-view__author a span')[0]->innertext;
            $review['author']=$reviewAuthor;
        }
        if(count($div->find('.business-review-view__author meta'))){
            $reviewImage=$div->find('.business-review-view__author meta')[0]->content;
            $review['image']=$reviewImage;
        }
        if(count($div->find('.business-review-view__date'))){
            $reviewDate=$div->find('.business-review-view__date meta')[0]->content;
            $review['date']=$reviewDate;
        }
        if(count($div->find('.business-review-view__body-text'))){
            $reviewText=$div->find('.business-review-view__body-text')[0]->innertext;
            $review['text']=$reviewText;
        }
        $reviews[]=$review;
    }
}

As a result, we get the following data in the reviews array : user avatar, name, date of publication of the review, and, in fact, the text of the review.
It is better not to use this code all the time, it is better to cache the data and, if necessary, pull it out of the cache
Yandex can always change the html code and then the method will not work

Answer 3

Use sessions.
How I won. Created a session. Then I made an initial POST request and received a token from Yandex. After that, I substituted it into the link of interest and made a second request. As a result, I received the information I needed.

from bs4 import BeautifulSoup
import requests
import json
from urllib.parse import urlparse

url = "https://yandex.ru/maps/api/business/fetchReviews?ajax=1&businessId=221702091001&csrfToken=97fd195e6cf78eb6f395d7c395d296ab5c11d1f1:1571717475&page=1&pageSize=5&ranking=by_time&reqId=1571655534207434-3187132131-sas1-1604&sessionId=1571655349968_626529"

url_post = "https://yandex.ru/maps/api/business/fetchReviews"

s = requests.session()
p = s.post(url_post)

data = json.loads(p.content)

parsed = urlparse(url)
querys = parsed.query.split("&")
querys[2]='csrfToken'+"="+data['csrfToken']
querys = '&'.join(querys)

url=url_post+"?"+querys

r = s.get(url)

soup = BeautifulSoup(r.content, 'html.parser')
print(soup)

Answer 4

why through Yandex maps, you can parse reviews like this

https://market.yandex.ru/shop/<ID_организации>/reviews

?

Answer 5

The method has stopped working (. Any more ideas?!