How to take part of a file?

A

Alexey Firsov2012-02-14 07:22:38

PHP

Alexey Firsov, 2012-02-14 07:22:38

The task is the following. There is a file of about 50-70ts lines, lines of different lengths.
You need to take a random string from the file, but do not load the file into the stream. since its size is 150 meters. and one random line is needed.

There have been attempts to move the internal file pointer by a number of bytes.
But since the line is different, 2 lines can be messed with in a piece.

Even I was sad, nothing came into my head at all. Does anyone know or have experienced?

Reply

Answer the question

In order to leave comments, you need to log in

9 answer(s)

A

avalak, 2012-02-14
@avalak

If it is impossible to drive the data into the database, then I can offer a crutch bike. You can build an index (a file with offset: length, data align pairs) and use it to access rows.

A

Alex Bunin, 2012-02-14
@azxc

Are you talking about a program method? If yes, then you can take a random pointer somewhere inside the file and then find the nearest "\n" (or whatever end-of-line pointer is), and then take a piece from this to the next end-of-line pointer (or end-of-file if this is suddenly the last line ).

D

deadkrolik, 2012-02-14
@deadkrolik

filesize + fseek + fseek left N again until we find a newline + fread

A

Alexey Firsov, 2012-02-14
@lesha_firs

well, yes, software :) I took the pointer. but how to find "\n"? without uploading a file?

S

strib, 2012-02-14
@strib

en.wikipedia.org/wiki/Memory-mapped_file

E

EndUser, 2012-02-14
@EndUser

"Random string" is it random from the word random() or from the word "arbitrarily conceived with number N"?
If the second, then create a file index.

A

Arris, 2012-02-18
@Arris

And not an option to build an index on a file? Type "line number" - "offset". Simulate a DBMS, so to speak?

A

AHTOH, 2012-02-18
@AHTHOH

I would define the maximum possible length of the string, and in an arbitrary place (fseek) I would read double the length (fread). And then, in the resulting fragment, I would look for a string limited by two newlines or even sscanf.

D

DmZ, 2012-02-18
@DmZ

And why look for something if native PHP functions will find everything perfectly for you.
You fseek to a random location in the file. Do the first fgets() or stream_get_line() with a sufficient buffer size - they are guaranteed to find the end of the line themselves. Next, do fseek from the original location + the read length of the string - i.e. guaranteed to get to the beginning of the trail. line that you do fgets() and use.
(You need to add checks for EOF of course)
This way you will get your line at the cost of the memory occupied by the fgets/stream_get_line buffer and that's it.