How to correctly write the getBlobReader function?

Question

go

The essence of one program is that it can store any binary file, breaking it into pieces of a fixed size plus the remainder. The chunks are saved to the blobs directory, where each is named after a hash of a cryptographic function from the content. The hashes are stored in the database and are associated with the name of the source file.

You then need to recreate the original file by gluing the pieces one by one.
It is not possible to correctly write the getBlobReader function. At the input keys (hash sums), at the output - one io.ReadCloser, which can read from all the pieces in turn, until they all run out.

I sketched out the code for an example:

package main

import (
  "io"
  "log"
  "os"
  "path/filepath"
)

func getBlobReader(keys []string) (io.ReadCloser, error) {
  var rc io.ReadCloser
  var err error
  for _, key := range keys {
    filePath := filepath.Join(".", "blobs", key)

    rc, err = os.Open(filePath)
    if err != nil {
      return nil, err
    }

  }
  return rc, nil
}

func main() {
  keys := []string{
    "2050-part1",
    "2050-part2",
    "2050-part3",
  }

  rc, err := getBlobReader(keys)
  if err != nil {
    log.Fatalln(err)
  }

  _, err = io.Copy(os.Stdout, rc)
  if err != nil {
    log.Fatalln(err)
  }

  log.Println("Done")
}

Answer 1

You need to implement your io.Reader, and in Read already read data in turn from parts of the file.
Here is a working example implementation

package main

import (
    "io"
    "log"
    "os"
    "path/filepath"
)

type BlobReader struct {
    keys       []string
    currentKey uint64
    reader     io.ReadCloser
}

func NewBlobReader(keys []string) *BlobReader {
    return &BlobReader{
        keys: keys,
    }
}

// Read реализация интерфейса io.Reader, чтобы можно было ваш reader использовать в io.Copy
func (br *BlobReader) Read(p []byte) (int, error) {
    var err error

    // открываем каждый файл по очереди
    if br.reader == nil {
        filePath := filepath.Join(".", "blobs", br.keys[br.currentKey])
        br.reader, err = os.Open(filePath)
        if err != nil {
            return 0, err
        }
    }

    // читаем данные из текущего открытого файла, пока данные не закончатся
    n, err := br.reader.Read(p)

    // если данные в файле закончились, закрываем его
    if err == io.EOF {
        br.currentKey++
        br.reader.Close()
        br.reader = nil

        // io.EOF в err должно вернуть только у последнего файла, чтобы io.Copy считал все файлы и не завис на последнем.
        if br.currentKey < uint64(len(br.keys)) {
            err = nil
        }
    }

    return n, err
}

func main() {
    blobReader := NewBlobReader([]string{
        "2050-part1",
        "2050-part2",
        "2050-part3",
    })

    _, err := io.Copy(os.Stdout, blobReader)
    if err != nil {
        log.Fatalln(err)
    }

    log.Println("Done")
}

You will also need to implement the Close method on your Reader so that there are no file descriptor leaks