hamishavery wrote:
> Hello Paulo,
Hi, Hamish
> Many thanks for the advice. My application is a geophysical
> datalogger that produces a lot of data continuously. Because of this,
> in combination with the need for data integrity if there is a power
> failure, I expected 'sync' to be the best and most honest mount
> choice. My understanding was that 'async' mounts reported optimistic
> write speeds for burst data because the dirty data cache absorbed the
> slow write speed of the phyical media, but I need sustained write
> speeds over many days.
Well, if you need data integrity in the event of a power failure you'll
also need to use a journaling filesystem and a "good" brand SD card.
'Sync' alone won't guarantee that.
But just think about what 'sync' does: every write your program does
must be "on disk" before the write call returns. So, for every write
(even if it is smaller than the erase block size of the device), the SD
card must read an erase page, erase it, write the data you sent along
with the rest of the data read *and* do the same for metadata (file size
changed, dir atime, inode alocation, etc.).
This is just plain awful for performance.
If you just need data integrity (but not the exact last event before
power failure), then let the system buffer writes for you. Use ext3 (or
similar) and let it do the "commit every 5 seconds" thing that it
usually does.
Of course you could also get better performance by coalescing writes
yourself and just log to disk when you had something like 128 kBytes of
information, but this is just doing in the application what the OS is
already able to do for you.
Also, you can control the size of the dirty data. I haven't used a 2.4
kernel in a long while, but in a 2.6 kernel you have these settings:
/proc/sys/vm/dirty_ratio
/proc/sys/vm/dirty_background_ratio
IIRC in a 2.4 kernel there is a similar setting but with a different
name ("pdflush_ratio" or something(?)).
In a 2.6 kernel you can also control how fast the kernel should commit
dirty data to disk with:
/proc/sys/vm/dirty_expire_centisecs
/proc/sys/vm/dirty_writeback_centisecs
> Your point about the efficiency of writing blocks at a time with
> 'async' mounted media is very interesting and I am looking into that.
> However, TS have reported much higher write speeds in their tests
> even with the SD card mounted as 'sync'. My real concern though is
> that I have done something wrong that prevents me from achieving the
> speeds TS have reported to me.
They are probably writing large blocks at a time, so the "multiple small
writes" problem shouldn't be so severe.
> I have tried two brands of SD card, a generic 512Mb 60x card and a
> 256Mb Kingston card. The generic card is faster than the Kingston
> card but not startling so. I have tried using many small or one big
> partition, and block sizes of 1kb to 4kb with little difference.
Since you're using 'sync' it is the size of *your* writes that matter.
Block sizes should make little difference indeed.
> All
> have ext2 format. I have a 32Mb yaffs root fs. The SD card driver is
> mounted with DMA enabled (insmod sdcard.o dmaenable=1).
Again, since you're using 'sync', you'll have to wait for the write to
complete, so there is no performance advantage of using dma. Unless
you're running multiple processes and can schedule something else to run
in the meanwhile.
--
Paulo Marques
Software Development Department - Grupo PIE, S.A.
Phone: +351 252 290600, Fax: +351 252 290601
Web: www.grupopie.com
"The face of a child can say it all, especially the
mouth part of the face."
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/ts-7000/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/ts-7000/join
(Yahoo! ID required)
<*> To change settings via email:
<*> To unsubscribe from this group, send an email to:
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
|