S
S
spoki2013-12-03 15:26:55
Java
spoki, 2013-12-03 15:26:55

Problem downloading images in multi-threaded mode?

Hello.
I have the following problem:
Through requests to ajax.googleapis.com/ajax/services/search/ I get json responses and from the received urls I create new streams with an image loader:

for(final GoogleResponseData responseData : googleResponse.getResults()) {
////
  new Thread() {
    @Override
    public void run() {
      try {
        new ImageDownloader().downloadImage(responseData.getUrl(), path, fileName);
      } catch (IOException e) {
        e.printStackTrace();
      }
    }
  }.start();
////
}

loader code:
private void downloadImage(String imageUrl, String path, fileName) throws IOException {
  URL u = new URL(imageUrl);
  
  String type = imageUrl.substring(imageUrl.lastIndexOf("."));
  
  URLConnection connection = u.openConnection();
  
  File imageFile = new File(path);
  imageFile.mkdirs();
  OutputStream os = new FileOutputStream(path  + fileName + type);
  try {
    IOUtils.copy(connection.getInputStream(), os);
  } catch(IOException e) {
    e.printStackTrace();
  } finally {
    os.close();
  }
}

As a result, most of my images are either not saved or not fully downloaded.
Although if this is done in one stream, all images are downloaded without problems.

Answer the question

In order to leave comments, you need to log in

2 answer(s)
S
student13, 2013-12-05
@spoki

Judging by the above code, everything seems to be normal, more code is needed. There is a guess that you are not correctly waiting for the completion of all threads and exit without waiting for the end of the download on all threads.

M
mardy_bum, 2013-12-03
@mardy_bum

It is possible that most of the downloaded images are on the same host, and when your streams crowd into the server, it drops them due to some of its limits.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question