How to quickly load data into the database with checking each record for uniqueness?

Question

PHP

Hello.

On the site, you need to upload data from a csv file to the database. In this case, each line from the csv file must be checked in the database.

The data should not be in the database, only new data is loaded into the database.

Tell me how to implement it correctly?

So far I have implemented it like this and I think it's wrong, loading is very slow, and if there is a lot of data in the database, then the request ends with a 502 error.

$wordsFromFile = array_map(function($data) { return str_getcsv($data,";");}, file($request->file('text_file')) );

        // Все слова из базы
        $myWords = Word::all();
        $dbWords = [];

        foreach ($myWords as $myWord) {
            $dbWords[] = $myWord->word;
        }

        $wordsResult= [];
        foreach($wordsFromFile as $key => $data)
        {
            if($key == 0) {continue;}

            $word = isset($data[0]) ? mb_strtolower($data[0]) : "empty_field";
            $translation = isset($data[1]) ? $data[1] : "empty_field";
            $state = isset($data[2]) ? $data[2] : 0;

            //$wordInDatabase = $myWords->where('word', $word)->first();

            if(!in_array($word, $dbWords))
            {
                $wordsResult[] = [
                    'word' => mb_strtolower($word),
                    'translation' => mb_strtolower($translation),
                    'state' => $state,
                ];
            }
        }

        Word::insert($wordsResult);

Answer 1

When using INSERT IGNORE - record will be added only if it is unique.