[Arm-netbook] Must watch video of GK802

[Gordan Bobic]

 On Fri, 11 Jan 2013 13:52:22 +0000, jm <joem at martindale-electric.co.uk> 
 wrote:
> On Fri, 2013-01-11 at 13:03 +0000, Gordan Bobic wrote:
>> On Fri, 11 Jan 2013 12:31:38 +0000, jm 
>> <joem at martindale-electric.co.uk>
>>  wrote:
>> > On Fri, 2013-01-11 at 11:14 +0000, Gordan Bobic wrote:
>> >> On Fri, 11 Jan 2013 10:56:54 +0000, jm
>> >> <joem at martindale-electric.co.uk>
>> >>  wrote:
>> >>
>> >> >>  nilfs is fully log structured and always appends
>> >> >>  to the end, with a garbage collection thread that
>> >> >>  starts from the head of the log (block device), and writes
>> >> >>  the de-garbaged data back to the tail of the log.
>> >> >>
>> >> >>  Is f2fs more clever and less write-intensive (due to
>> >> >>  the nature of nilfs2's garbage collection)?
>> >> >
>> >> > f2fs - I think it gets all its speed because there
>> >> > is no defragging until it reaches back to the beginning
>> >> > at which point it is forced defrag.
>> >>
>> >>  That's not really that much better than nilfs2. That just
>> >>  doesn't bother garbage collecting until free space drops
>> >>  below a settable threshold. Garbage collector thread
>> >>  is also throttlable and nice-able (and ionice-able, but
>> >>  that only works on CFQ which is largely unfit for any
>> >>  purpose these days).
>> >>
>> >> >>  Either way, this is by no means a pre-requisite for
>> >> >>  using such devices. Many existing ARM machines
>> >> >>  have been happily running off eMMC and SD devices long
>> >> >>  term without any issues regarding media wear-out.
>> >> >
>> >> > Correct if its only reading. But general purpose
>> >> > Linux needs to read and write.
>> >>
>> >>  My SheevaPlugs do plenty of writing - they are used as
>> >>  general purpose servers. I use a Toshiba AC100 daily
>> >>  as a general purpose laptop. SD/eMMC wear really isn't
>> >>  as big a deal as you might think.
>> >>
>> >> >>  My SheevaPlugs run off an SD card, and have been for
>> >> >>  at least a couple of years without any issues,
>> >> >
>> >> > I have 2 but I run general purpose Linux and have
>> >> > ruined several flash drives despite being ext2 formatted.
>> >> > I would log into them and set it on download a large
>> >> > number of files. One file at a time not a problem, but I think
>> >> when
>> >> > running parallel downloads seems to be the time when it kills 
>> it.
>> >>
>> >>  My guess would be that you have been using particularly
>> >>  bad flash media. Pretec USB sticks are known to be
>> >>  particularly useless (killable in a few hours using a
>> >>  basic iozone test). But most have been fine (Kingston
>> >>  controller based USB sticks and SD cards,
>> >>  SanDisk MLC and Integral SLC SD cards, etc. I have
>> >>  a number of servers using them as primary storage and
>> >>  none have exhibited problems after a couple of years
>> >>  of use.
>> >
>> > One device I know that got burned was a kingston.
>> > But then again my detailed knowledge of SD card controllers
>> > is close to nil.
>> >
>> >> >>  as
>> >> >>  have my Toshiba AC100s. It's really not a problem.
>> >> >>  Performance on random-writes is a much bigger issue,
>> >> >>  which is one place where something like nilfs2 helps
>> >> >>  a great deal by making all writes always linear,
>> >> >>  this making your random write MB/s the same as your
>> >> >>  linear write MB/s. Reads are extremely quick on
>> >> >>  SD cards regardless of whether they are sequential
>> >> >>  or random
>> >> >
>> >> > Random is slower because the SDCard registers have to
>> >> > be set up to start reading at a new address.
>> >> > Sequential is fast because one read after another
>> >> > can be instigated without needing to send in
>> >> > an address for each read.
>> >>
>> >>  See my benchmarks here:
>> >>
>> >> 
>> http://www.altechnative.net/2012/01/25/flash-module-benchmark-collection-sd-cards-cf-cards-usb-sticks/
>> >
>> > Thank you for the link - I can see you spent time and money
>> > to know what is good.
>>
>>  I also accept contributions for that list. :)
>>
>> > Generally I aim for best price
>> > and avoid anything with bad reviews. That is not a robust
>> > method - because I treat them as throw away items
>> > not to be trusted for long term storage of data.
>>
>>  My findings are that the class ratings and price are in
>>  general no way correlated (positively or negatively) to
>>  the random-write performance. It is also not obviously
>>  brand-related, either, but that could be because the
>>  same brand will come with a range of different flash
>>  controllers. Kingston controllered devices seem to be
>>  less bad than the rest, on average, but I don't really
>>  have a big enough sample to categorically back that up.
>>
>> >>  Most SD cards manage at least 1000 IOPS (4KB blocks)
>> >>  on random reads, some over double that. If that isn't
>> >>  fast enough you must have the fastest ARM machine in
>> >>  the world.
>> >>
>> >> > Writes have similar problem - but doubly slow because writing
>> >> > is inherently a slower process.
>> >>
>> >>  See the benchmarks at the link above. Random-write
>> >>  performance on generic flash media (proper SSDs excluded)
>> >>  is appallingly slow. Random writes are a major
>> >>  performance issue for SD cards. Random reads are fine.
>> >
>> > I'm looking at the data from the link,
>> > but I could not see where the difference
>> > between reading contiguous data and random reads is published.
>>
>>  Last link in the article to get the performance tables.
>>  Then scroll down, it's in the 2nd table.
>
> Ah yes the old scroll down trick! :)
> But the numbers don't add up.

 How do you figure that?

> There is no direct table for sequential read v random read.

 No, the tables are separate, but the devices listed in them
 are the same. It makes it easier to see the details depending
 on whether you need the flash module for running the OS from
 or for a digital camera. For the former you want random IOPS,
 for the latter sequential MB/s speeds.

> Picking apart that first table for Kingston Elite Pro.
> The 4K block size means that the address is sent after each block
> is sent - potentially.
> With a 1M block, address is not sent until the entire block is sent
> potentially. So the performance goes from 12.5MB/s to 56.9 MB/s 
> because
> the 12.5MB/s is processed more like a random read while the 56.9 is
> processed more like a sequential read.
>
> In the second table you got IOPS of 1237 per 4k random block per 
> second
> which translates to 4096x1237 which is about 5MB/s.
>
> So the speed difference between a genuine random read and sequential
> read is 5MB/s to 56.9MB/s or about 10x speed drop between random read
> and sequential read. Between 5x to 10x speed drop is not uncommon.

 Yup, that's right.

>> >> >>  so potential de-linearization of them
>> >> >>  isn't problem.
>> >> >>
>> >> >> > The great thing is that you can now yank out the OS
>> >> >> > and put a new OS in seconds. No way to brick a device.
>> >> >>
>> >> >>  There are several other devices that work this way.
>> >> >
>> >> > My ignorance - I haven't yet come across any common tablets
>> >> > or devices that have uSDCard for its OS. Hence the excitement.
>> >>
>> >>  I seem to recall the Via APC does the same thing. The Genesi
>> >>  Efika MX Smartbook by default checks the uSD card slot for an
>> >>  OS first, IIRC, and only then tries to boot off internal CF
>> >>  (soldered on, non-removable, sadly) I'm sure there are others
>> >>  with similar features.
>> >
>> > Sorry that is slightly different to what I was saying.
>> > The OS for those devices is in the soldered down flash for those
>> > devices. There is usually extra uSD to boot alternative OSes.
>>
>>  Hmm... I thought the Via APC only booted from the SD card.
>>  Also, DreamPlug has no on-board flash, except for the uboot.
>>  You can only put the kernel and the OS on some removable media.
>>  It has an internal uSD slot, external SD slot, USB and SATA.
>>
>> > The GK802 has two uSD card slots in the video - one is internal 
>> and
>> > hidden that contains the OS that ships with the product
>> > as well the second accessible uSD card to boot your own OS.
>> > http://www.youtube.com/watch?v=dHtQvGyV6Mk
>>
>>  Sounds similar to what the DreamPlug does.
>
> Yes:
> http://blog.bertelsen.co/2011/07/backing-up-internal-sd-card.html

 I have a number of DreamPlugs as well as SheevaPlugs in the
 compile farm I use for RedSleeve. :)

 Gordan