S
S
Skelman2019-09-30 19:20:21
Node.js
Skelman, 2019-09-30 19:20:21

Promise.all hangs. Doesn't give any error or result. What is the problem?

There is a code:

const scrapTitleAndRatings = async () => {
  const html = await request.get(
    "https://www.imdb.com/chart/moviemeter?ref_=nv_mv_mpm"
  );
  const $ = await cheerio.load(html);

  const movies = $("tr")
    .map((i, element) => {
      const title = $(element)
        .find("td.titleColumn > a")
        .text();
      const urlDescription = `https://www.imdb.com${$(element)
        .find("td.titleColumn > a")
        .attr("href")}`;
      const rating = $(element)
        .find("td.ratingColumn.imdbRating > strong")
        .text();
      return { title, rating, rank: i, urlDescription };
    })
    .get();
  return movies.filter(m => m.title);
};

const getPosterImageUrl = async movies => {
  for (let i = 0; i < movies.length; i++) {
    try {
      const posterImageUrel = await nightmare
        .goto(movies[i].posterUrl)
        .evaluate(() =>
          $(
            "#photo-container > div > div:nth-child(3) > div > div.pswp__scroll-wrap > div.pswp__container > div:nth-child(2) > div > img:nth-child(2)"
          ).attr("src")
        );
      movies[i].posterImageUrel = posterImageUrel;
      savePostersToDisk(movies[i]);
    } catch (error) {
      console.log(error);
    }
  }
  return movies;
};

const getPostUrl = async movie => {
  try {
    const html = await request.get(movie.urlDescription);
    const $ = await cheerio.load(html);
    movie.posterUrl = "https://www.imdb.com" + $("div.poster > a").attr("href");
    return movie;
  } catch (error) {
    console.log(error);
  }
};

const scrapPosterUrl = async movies => {
  try {
    const promisemovies = movies.map(async movie => getPostUrl(movie));
    const moviesPostersUrl = await Promise.all(promisemovies);
    return moviesPostersUrl;
  } catch (error) {
    console.log(error);
  }
};

async function main() {
  try {
    let movies = await scrapTitleAndRatings();
    movies = await scrapPosterUrl(movies);
    console.log(movies);
    // movies = await getPosterImageUrl(movies);
    // console.log(movies);
  } catch (error) {
    console.log(error);
  }
}

The problem is in the scrapPosterUrl function, which cannot return all promises. Moreover, if I run the code on the mobile Internet, then everything returns. Can someone suggest why this is happening? And how to fix this problem?

Answer the question

In order to leave comments, you need to log in

1 answer(s)
S
Sergey Melodin, 2019-09-30
@melodyn

I can't say exactly why, but they don't:

const promisemovies = movies.map(async movie => getPostUrl(movie));
const moviesPostersUrl = await Promise.all(promisemovies);

because map doesn't support async by default. It's
quite possible that the legs of the problem grow from there. Actually, you just need to return an array of requests:
In general, the code needs refactoring, because everything is mixed up - layout, logic, horses, people. The problem with finding the problem comes from here.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question