B
B
bit_rainbow2018-08-20 18:58:32
Scala
bit_rainbow, 2018-08-20 18:58:32

How to make 1000 requests in Scala and add up the total result?

I'm in Skala for a couple of days. After the docks, there was a misunderstanding.
There is, for example, a url, user/info/:id, which produces homogeneous data.
I need to send 1000 requests like: user/info/1, user/info/2, user/info/1000
and merge the result together.
Consistently - too slow.
I did this:
====

val u1 = Future {
            ..... do request
    }

    val u1000 = Future {
            ..... do request
    }

    val req = Future.sequence(Seq(u1, u2, u1000))
    var result = Await.result(req, duration)

1. Future.sequence(Seq(u1, u2, u1000)) Does the Future run in parallel or sequentially? If in parallel, how many threads?
2. Do I understand correctly that Await.result(req, duration) does blocking until all Futures are completed.
3. How to speed up the code?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
A
Artem, 2018-08-22
@mrRontgen

Good afternoon)
In this form, any decision will be a crutch. It's best to use the Akka Streams paradigm.
i) Create a Source that will contain all possible requests
ii) Create a Flow that will send requests in parallel. You can use Map, you can specify the number of threads there.
iii) Create a Sink that will accumulate responses.
iv) At the output, get one Future with all the answers.
v) Profit) More details here .

M
Maxim Moseychuk, 2018-08-21
@fshp

1) sequence will not start future execution. Futures start executing at the moment of creation, i.e. even before sequence.
2) Correct. Only you also have a timeout. Doing so is not very good. Use onComplete/map
3) You'd better look at ParSeq for this task.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question