C
C
chernyshiov2020-07-06 23:59:33
Parsing
chernyshiov, 2020-07-06 23:59:33

Steam parsing, via google spreadsheet, problems with importxml?

I can't parse price from https://steamcommunity.com/
I want to create a page with auto-updating prices
https://steamcommunity.com/market/listings/730/Sti...
for example from this page, I need a minimum sale price, I tried it like and from the line, and from the table, writes N \
5f03905b345a4587451313.png
A I also tried it through Excel, the internal browser does not see the price at all, because of the lack of Java support, as I understand it.

Answer the question

In order to leave comments, you need to log in

1 answer(s)
G
Grigory Boev, 2020-07-07
@chernyshiov

Try to get the page code with this function. If the required data is not there, then they are loaded by scripts.

/**
* ОТкрывает URL и возращает код страницы
* Telegram - @ProgrammerForever
*
* @param {string} URL URL который нужно открыть
* @param {boolean} isCut Указывакт, нужно ли обрезать страницу до 50000 символов по длине, по умолчанию false
* @param {boolean} noScript Указывакт, нужно ли удалять скрипты из кода
* @return Исходный код страницы
* @customfunction
*/
function getHTML(URL,isCut,noScript) {
  if ((URL === undefined)||(URL == "")) { return "#ОШИБКА Пустой URL";};
  if (isCut === undefined) {var isCut=true;};
  if (noScript === undefined) {var noScript=true;};
  if (URL.map){     //Если задан диапазон
    return URL.map(getHTML);
  }else{
    try {
      var payload = {
        'rand':(new Date()).getTime()
      };
      var headers={
        'Connection': 'keep-alive',
        'Cache-Control': 'max-age=0',
        'Upgrade-Insecure-Requests': 1,
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3',
        'Accept-Encoding': 'gzip, deflate, br',
        'Accept-Language': 'ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7'
      };
      var options = {
        'method' : 'get',
        'headers' : headers,
        'payload': payload
      };
      
      var response = UrlFetchApp.fetch(URL,options);
      var charset=response.getAllHeaders["charset"];
      //var responseText=response.getContentText(charset?charset:"windows-1251");
      var responseText=response.getContentText(charset?charset:"UTF-8");
      if (noScript){ 
        responseText=responseText.replace(/<script[^>]*>(?:(?!<\/script>)[^])*<\/script>/gmi,"");
        responseText=responseText.replace(/<!--.*?-->/gmi,"");
        responseText=responseText.replace(/<link.*?\/>/gmi,"");
        responseText=responseText.replace(/<meta.*?\/>/gmi,"");
        responseText=responseText.replace(/[\n\r\t]/gmi,"");
        
      };
      if (isCut&&(responseText.length>50000)){return responseText.substring(0,50000);}else{return responseText;};
    } catch (err) {
      //return JSON.stringify(err);
      return "#ОШИБКА "+err.message;
    };
  };
};

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question