Is it possible to get a list of all possible toster.ru tags?

Question

Toaster

Is it possible to get a list of all possible toster.ru tags, and if so, how? To subscribe to all interested. Of course, you can browse all 62 (currently) pages tagged "For Questions" through a browser in search of interesting ones on the topic, but it's so long and dreary, in the form of some simple .txt file containing all possible tags en masse would be much better.

Answer 1

This possibility is not provided. If the range of your interests is limited to certain topics - you can use the search, if not - on the page of each tag, in the right column, there are tags close to it. There you can find tags related to your interests.

Answer 2

const puppeteer = require('puppeteer');
const fs = require('fs');
const endOfLine = require('os').EOL;

const pageStart = "https://toster.ru/tags/?page=";
const selector = ".card__head-title a";
const fileName = "tags.txt";
const pages = 62;

(async () => {
  const browser = await puppeteer.launch();

  const promises = [];
  for (let i = 1; i <= pages; i++) {
    promises.push(browser.newPage().then(async page => {
      const url = pageStart + i;
      console.log(url);
      await page.goto(url, {timeout: 100000});
      return await page.$$eval(selector, elements => {
        return Array.from(elements).map(element => element.innerText);
      });
    }));
  }

  const pagesData = await Promise.all(promises);

  const logger = fs.createWriteStream(fileName, {
    flags: 'a'
  });

  pagesDataByLines = pagesData.reduce((acc, el) => ([...acc, ...el]));

  pagesDataByLines.forEach((line, index) => {
    logger.write(`${index}. ${line} ${endOfLine}`);
  });

  await browser.close();
})();

first time I wrote scraper, so 99.99% that something here is a crutch, but it works =)