K
K
khodos_dmitry2018-06-11 10:48:54
PHP
khodos_dmitry, 2018-06-11 10:48:54

Why is it so slow to save data in CSV?

I save using the function:

function put_ten() {
  global $link;
  $query = "SELECT `item`.`id`, `item`.`name`, `item`.`cat1`, `item`.`cat2`, `item`.`cat3`, `item`.`cat4`, `item`.`price`, `img`.`source` FROM `item` JOIN `img` ON `item`.`id` = `img`.`item_id` WHERE `item`.`inserted` = 0 LIMIT 1000";
  $result = mysqli_query($link, $query) or die(mysqli_error($link));
  for ($i = 1; $data = mysqli_fetch_assoc($result); $i++) {
    $query = "SELECT `name`, `value` FROM `param` WHERE `item_id` = {$data['id']}";
    $result_param = mysqli_query($link, $query) or die(mysqli_error($link));
    $params = '';
    while($data_param = mysqli_fetch_assoc($result_param)) {
      $params .= $data_param['name'].': '.$data_param['value'].';';
    }
    $csv .= $data['name'].';'.$data['cat1'].';'.$data['cat2'].';'.$data['cat3'].';'.$data['cat4'].';'.$data['price'].';'.'http://armatura-truba.ru/'.$data['source'].';'.$params."\r\n";
    file_put_contents('items.log', $i."\t".date("i:s")."\r\n", FILE_APPEND);
  }
  file_put_contents('items.csv', $csv, FILE_APPEND);
}

Sometimes it turns out that one hundred records per second will be saved, sometimes in one and a half, two minutes (most often). Why is that? And is there a way to save a selection from different tables faster?

Answer the question

In order to leave comments, you need to log in

2 answer(s)
K
khodos_dmitry, 2018-06-11
@khodos_dmitry

Put indexes on `item_id` and everything became pretty fast.

T
ThunderCat, 2018-06-11
@ThunderCat

function put_ten() {
  global $link;
  $query = "SELECT `item`.`id`, `item`.`name`, `item`.`cat1`, 
                `item`.`cat2`, `item`.`cat3`, `item`.`cat4`, 
                `item`.`price`, `img`.`source`, `param`.`name` pname, `param`.`value` pvalue
          FROM `item` 
          JOIN `img` 
          ON `item`.`id` = `img`.`item_id` 
          JOIN `param` 
          ON `item`.`id` = `param`.`item_id` 
          WHERE `item`.`inserted` = 0 
          LIMIT 1000";
// дальше 1 циклом нормально пройтись
}

UPD: Without indexes it was... WITHOUT INDEXES, CARL!!! How is that?

Didn't find what you were looking for?

Ask your question

Ask a Question

731 491 924 answers to any question