Y
Y
Yevhenii K2020-11-09 03:10:52
Laravel
Yevhenii K, 2020-11-09 03:10:52

Why is Job only updating half of the data?

There is a Job that should update the records in the database. The problem is that for each run it only updates half. And so with each launch, always only ~ 50%. Starts at 7500, 3800, 1900, etc.
There are no errors in the logs. Why is this happening?

[2020-11-08 23:50:26][58] Processing: App\Jobs\FixNonUniqueAddresses
[2020-11-08 23:50:40][58] Processed:  App\Jobs\FixNonUniqueAddresses

<?php

namespace App\Jobs;

use App\Models\Address;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;

class FixNonUniqueAddresses implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    protected $timeout = 1000;

    public function __construct()
    {
        //
    }

    public function handle()
    {
        Address::where('addressable_type', 'places')->whereNull('type')->chunkById(
            50,
            function ($addresses) {
                $addresses->each(function ($address) {
                    $newData = transformStreet(
                        $address->street,
                        $address->lang
                    );
                    $address->update($newData);
                });
            }
        );
    }
}

Answer the question

In order to leave comments, you need to log in

2 answer(s)
V
vism, 2020-11-09
@AFI19

because you are updating the condition on which you make the selection.
Well, look, you have 1000 entries, for example, ID from 1 to 1000.
You take the first 50 and update. there are 950 not updated
A chunk makes limit +50
And the next selection you start with an indent of 50 records, as a result, records with ID 51-100 are skipped, and 101-150 will be updated.
the limit is again +50
, another 50 are skipped and so on.

P
pLavrenov, 2020-11-09
@pLavrenov

Perhaps the runtime is interfering with this. It's best to remake this job so that it creates a lot of smaller jobs instead of processing all the records now. Besides such job to hammer a slot in queue for a long time.
Put horizon and you will see what happens to them.

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question