How to read a large file?

[HARRY01]

I need to read a pretty large file ( > 32KB) into my program. The following code does't read at all:

local( fpin$,fl$,buf$,r$);

fseek(fpin$,-1,SEEK_END);
fl$=ftell(fpin$)+1;
sdim(buf$,TBYTE,fl$);
r$=fread(&buf$,fl$,1,fpin$);
if (r$<>fl$).....
In debugging mode I can see that buf$ is correctly dimensioned, but that zero bytes are read.
What is wrong?

Harry

[PointOfLight]

Could you post your fopen statement?

[kornalius]

You need to read in batches. You cannot read the whole thing all at once.

You can use LoadStr() to solve the problem very easily, else use the following:

SDIM(buf$, TBYTE, 4096);
f$ = fopen("c:\\ge.1", "r");
x$ = 0;
repeat
r$=fread(&buf$ + x$, 1, 10, f$);
x$ = x$ + r$;
until (r$ < 10);
fclose(f$);
ShowMessage(buf$);

[PointOfLight]

Is there any reason you can't read the whole thing in at once? I don't see anything in the fread documentation on MSDN that indicates this should be a problem. However, I did notice one thing, Harry. You're using fseek to move to the end of the file, then I don't see another fseek to move to the beginning of the file before you read. After your call to ftell, try

fseek(fpin$, 0, SEEK_SET);

If that still doesn't work and you have to go to Kornalius' method of reading in as chunks, I would recommend 4096 as the chunck size. That should be pretty decent for the file size you're talking about.

[HARRY01]

Eric,

The fOpen is OK, I guess, as I get the correct length with ftell.

I found already that the file pointer has to be reset to BOF. But it didn't bring the solution.

I still have to do some more testing, but I got the impression that the file pointer is not correctl;y maintained when I read in chunks.
I found strange value, far too high. When I read the file of 30K in chunks of 1000 after 16 or 17 chunks EOF is reached. After the first chunk, the position is 1007, then .......4340, 8492 etc..
Tomorrow I will look further into this problem.

Harry

[kornalius]

Read one byte at the time but by chunks, like in my example, I read 10 bytes of 1 byte. Each record holds 1 byte.

[PointOfLight]

I don't necessarily think it's a good idea to read a file in all at once, but it should have worked, especially with a file as small as you were talking about. So, I decided to play. Here's my code:


local(fpin$,fpout$,fl$,buf$,r$);

fpin$ = fopen(&quot;c:\\program files\\ppl\\runtime\\AS_Logo_sm.bmp&quot;, &quot;rb&quot;);
fpout$ = fopen(&quot;c:\\temp\\AS_Logo_sm.bmp&quot;, &quot;wb&quot;);

fseek(fpin$,0,SEEK_END);
fl$=ftell(fpin$);
sdim(buf$,TBYTE,fl$);
fseek(fpin$, 0, SEEK_SET);
r$=fread(&amp;buf$,fl$,1,fpin$);
if(r$ &gt; 0)
// ShowMessage(r$ + &quot; &quot; + fl$);
fwrite(buf$, fl$, 1, fpout$);
end;

fclose(fpout$);
fclose(fpin$);


I ran this only on the desktop, but it worked just fine. I was able to open up the resulting bitmap without any problems, which is around 41K in size.

[HARRY01]

Eric,

You were right in asking me for my fopen statement. I used 'r' as parameter, you used 'rb', whatever that means. With 'rb' it works perfectly. May be that the Help information on fopen should be improved. Thanks for your assistance.

Harry

[PointOfLight]

A lot of these functions are simply wrappers for the corresponding C functions. While this won't always work, if you don't find enough information in the help file, try going to msdn.microsoft.com and searching for the function there. In the case of fopen, MSDN lists all of the possabilities for the second parameter. I don't think there are too many that haven't already been covered, so I will look into adding them. At any rate, the &quot;b&quot; stands for binary. Without it, fopen must assume that you are trying to read a text file, and will probably break at the first character that it deems &quot;not text&quot;. This could be a carriage return, Ctrl-Z (which is perfectly valid at the beginning of a file, even though for DOS it means end of file), or many other things. Anyway, I know this was a long explanation, so to recap:

&quot;rb&quot; = read binary
&quot;wb&quot; = write binary

[HARRY01]

Eric,

Thanks for the explanation. Sometimes life is confusing:

fread: in the Help file as receiving item the variable is mentioned but in our discussion the address of (&amp;) the variable is used. Is this contradictory?

/ or //: I get the impression that the use of / or // is sometimes 'don't care', in other cases only one form can be used.

Harry

[HARRY01]

Eric,

Please read \\ i.s.o. //.

Harry

[PointOfLight]

In regards to fread, it appears that it works fine both with or without the &amp;. I'll have to defer to Kornalius on this one, because I was always under the impression that if you wanted a parameter to be modified, you had to use &amp;.

As for your other question, I'm not sure what you're asking about the / or //.

[kornalius]

You need to pass a pointer to a memory location. With or without the & is very fine.

You could add a pointer address to a number, like

&buf$ + x$