From peter.barada@logicpd.com Thu Mar 22 14:54:55 2012 Received: from [174.46.170.154] (helo=edprlnx06.logicpd.com) by stoneboat.aleph1.co.uk with smtp (Exim 4.72) (envelope-from ) id 1SAjPa-0004Cx-D4 for yaffs@lists.aleph1.co.uk; Thu, 22 Mar 2012 14:54:55 +0000 Received: from [172.16.10.3] (unknown [172.16.10.3]) by edprlnx06.logicpd.com (Postfix) with ESMTP id AD64D600095; Thu, 22 Mar 2012 09:53:29 -0500 (CDT) Message-ID: <4F6B3D2E.9000009@logicpd.com> Date: Thu, 22 Mar 2012 10:54:38 -0400 From: Peter Barada User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.27) Gecko/20120216 Lightning/1.0b2 Thunderbird/3.1.19 MIME-Version: 1.0 To: yaffs@lists.aleph1.co.uk, manningc2@actrix.gen.nz References: <4F5B8A41.4090404@logicpd.com> <201203221714.28693.manningc2@actrix.gen.nz> In-Reply-To: <201203221714.28693.manningc2@actrix.gen.nz> Content-Type: multipart/mixed; boundary="------------090306060003070302020903" X-Host-Lookup-Failed: Reverse DNS lookup failed for 174.46.170.154 (deferred) X-SA-Exim-Connect-IP: 174.46.170.154 X-SA-Exim-Mail-From: peter.barada@logicpd.com X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on stoneboat.aleph1.co.uk X-Spam-Level: X-Spam-Status: No, score=-1.1 required=4.5 tests=AWL,BAYES_00,RDNS_NONE, SPF_PASS autolearn=no version=3.3.1 X-SA-Exim-Version: 4.2.1 (built Mon, 22 Mar 2010 06:51:10 +0000) X-SA-Exim-Scanned: Yes (on stoneboat.aleph1.co.uk) Subject: Re: [Yaffs] How to test in linux w/real NAND using userspace test harness X-BeenThere: yaffs@lists.aleph1.co.uk X-Mailman-Version: 2.1.13 Precedence: list List-Id: Discussion of YAFFS NAND flash filesystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2012 14:54:56 -0000 This is a multi-part message in MIME format. --------------090306060003070302020903 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 03/22/2012 12:14 AM, Charles Manning wrote: > On Sunday 11 March 2012 06:07:13 Peter Barada wrote: >> Charles e al, >> >> I'm working on adding read-distrub counting to YAFFS (to force a garbage >> collection when the number of reads in a block hit a certain limit (20K >> for MT29C4G48MAZAPAKQ5 OMAP PoP), and I'm looking for a straight-forward >> testing harness that will beat up YAFFS pretty hard. >> >> Googling around didn't come up with much obvious, so I'm asking what to >> use to test YAFFS out-of-kernel (i.e. using a userspace app that mounts >> a partition and thrash it) instead of nandsim and in-kernel testing... >> >> Thanks in advance! > Peter > > There are three ways I can think of to achieve this: > > Take the simulation code in yaffs direct and write a AND interface that calls > the userspace mtd functions. These are specced in mtd/mtd-user,h > > Another way to do this is to work the changes into a hacked version of u-boot > or such. > > Yet another way is to stick with Linux. Straight Linux will cache data in the > VFS which will not abuse the read path as you desire. You can ask Linux to > drop the caches by doing: > echo 3 > /proc/sys/vm/drop_caches > > Thus something like: > > while true; do cat /yaffs/dir/file > /dev/null; done > > in parallel with > > while true; do sleep 0.1; echo 3 > /proc/sys/vm/drop_caches; done > > should do some serious read pounding. I've written a script (attached) to do read pounding by: 1) Erase a MTD device and mount as /mnt/yaffs 2) Create 1MB of data from /dev/urandom as /mnt/yaffs/test-data 3) Create reference md5sum of that file 4) copy from /mnt/yaffs/test-data to /tmp/srcfile 5) compare md5sum of /tmp/srcfile to reference md5sum 6) copy /tmp/srcfile to /mnt/yaffs/test-dataN 7) sync the data 8) drop the page cache 9) cop /mnt/yaffs/test-data-N to /tmp/newfile 10) compare md5sum of /tmp/newfile to reference md5sum 11) If N > 1, delete /mnt/yaffs/test-data(N-1) 12) Dump selected YAFFS stats 13) Increment N and loop to step 4 if not to many failures or N hasn't hit a limit (100K) If the md5sums don't match, then its a failure, and since I have /tmp/srcfile and /tmp/newfile I then use a loop to wade through the files, 2K at a time looking for differences and dump the pages that differ. The thinking is that /mnt/yaffs/test-data will be read on each loop iteration to create the copy, so it should force read-disturb pretty quickly (I tweaked my read-disturb driver to throw -ESTALE from MTD after only 500 reads of a block instead of 20K), so reading 64MB from a block would trigger -ESTALE. After letting it run using a 32MB partition on two boards overnight (~9 hours of testing) I didn't see any failures: Pass: 18467 Failed: 0 Sat Jan 1 09:12:16 UTC 2000 Copy /mnt/yaffs/test-data to test-data18468 Force unwritten blocks to MTD Dropping page cache Calculate MD5 of fresh copy test-data18468 Removing test-data18467 n_ecc_stale.......... 784805 n_ecc_fixed.......... 0 n_ecc_unfixed........ 0 n_page_writes........ 10706425 n_page_reads......... 19993297 n_erased_blocks...... 6 n_erasures........... 167254 n_retried_writes..... 0 n_retired_blocks..... 0 n_stale_blocks....... 0 "n_ecc_stale" is the number of times yaffs_handle_chunk_error() was called with an -ESTALE state, "n_stale_blocks" tracks the number of current stale blocks, and in that testing only saw 1 block still stall in step 12, and then only at the end of three passes. But I want to beat it up even _further_, and was looking to see if there already existed a tool that people regularly use on real hardware to beat up YAFFS. I'm now looking at LTP (and its ltp-fsx.c test) to see if it will suit my needs. Under separate cover I'll send my read-disturb changes to for comment to see if my approach is sane... -- Peter Barada peter.barada@logicpd.com --------------090306060003070302020903 Content-Type: application/x-shellscript; name="nand-small-stress-test2.sh" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="nand-small-stress-test2.sh" IyEvYmluL3NoCgptbnRfcHQ9L21udC95YWZmcwojc2V0IC14CiMgVW5tb3VudCBOQU5EIGFu ZCBlcmFzZQp1bW91bnQgJG1udF9wdAoKTVREX1JBV19ERVZJQ0U9L2Rldi9tdGQzCmlmIFsg IiQxIiAhPSAiIiBdOyB0aGVuCiAgICBNVERfUkFXX0RFVklDRT0kMQpmaQoKZWNobyAiVXNp bmcgJE1URF9SQVdfREVWSUNFIgpNVERfQkxPQ0tfREVWSUNFPWBlY2hvICRNVERfUkFXX0RF VklDRSB8IHNlZCAtZSAncy9tdGQvbXRkYmxvY2svJ2AKZmxhc2hfZXJhc2VhbGwgJE1URF9S QVdfREVWSUNFCgojIFNsZWVwIGZvciBhIGZldyB0byBhbGxvdyBvdXRwdXQgZm9ybSBmbGFz aF9lcmFzZWFsbCB0byBzaG93IHVwIGJlZm9yZQojIGtlcm5lbCBtZXNzYWdlcyBmcm9tIG1v dW50CnNsZWVwIDUKCiMgbW91bnQgdGhlIGZsYXNoCm1rZGlyIC1wICRtbnRfcHQKaWYgWyAt biAiJDIiIF07IHRoZW4KICAgIG9wdGlvbnM9Ii1vICQyIgpmaQptb3VudCAtdCB5YWZmcyAk b3B0aW9ucyAkTVREX0JMT0NLX0RFVklDRSAkbW50X3B0CmNkICRtbnRfcHQKCiMgTmFtZSBv ZiBmaWxlIHRvIGNvcHkKZm5hbWU9dGVzdC1kYXRhCgojIFNvdXJjZSBvZiBmaWxlCnNyYz0k bW50X3B0Cgp0b3RhbHNpemU9YGV4cHIgMTAyNCBcKiAxMDI0YApwYWdlc2l6ZT0yMDQ4Cm51 bXBhZ2VzPWBleHByICR0b3RhbHNpemUgLyAkcGFnZXNpemVgCgpzcmNmaWxlPSR7c3JjfS8k e2ZuYW1lfQoKZWNobyAiVG90YWwgYnl0ZXMgJHRvdGFsc2l6ZSA9ICRudW1wYWdlcyBwYWdl cyIKZWNobyAiQ3JlYXRpbmcgJHNyY2ZpbGUgcmFuZG9tIHRlc3QgZGF0YSwgc2l6ZT0kdG90 YWxzaXplIgoKZGQgaWY9L2Rldi91cmFuZG9tIG9mPSRzcmNmaWxlIGJzPSRwYWdlc2l6ZSBj b3VudD0kbnVtcGFnZXMKCmZhaWxlZF9jbnQ9MApjbnQ9MQoKIyBDYWxjdWxhdGUgZ29vZCBt ZDVzdW0gb2YgdGhlIGNhcmQKbWQ1PWBjYXQgJHNyY2ZpbGUgfCBtZDVzdW1gCgpuZXdmaWxl PQojIE1ha2UgaW5pdGlhbCBjb3B5IGZyb20gdGhlIFNEIGNhcmQKIyBjcCAke3NyY30ke2Zu YW1lfSAke2ZuYW1lfSR7Y250fQoKIyBMb29wIHRocm91Z2ggMTAwMDAwIHBhc3Nlcwp3aGls ZSBbICRjbnQgLWx0IDEwMDAwMCAtYSAkZmFpbGVkX2NudCAtbHQgMjAgXTsgZG8KICAgICMg UmVwb3J0IHRoZSBwYXNzIHdlJ3JlIG9uCiAgICBlY2hvICJQYXNzOiAkY250IEZhaWxlZDog JGZhaWxlZF9jbnQiCiAgICBkYXRlCgogICAgIyBzYXZlIG5ldyBmaWxlIHNpbmNlIHdlJ2xs IHJlbW92ZSBpdCBhZnRlciB0aGUgY29weQogICAgbGFzdF9maWxlPSRuZXdmaWxlCgogICAg IyBDYWxjdWxhdGUgbmV4dCBmaWxlIGluZGV4CiAgICBjbnQyPWBleHByICRjbnQgKyAxYAog ICAgbmV3ZmlsZT0ke2ZuYW1lfSR7Y250Mn0KCiAgICAjIGNvcHkgdGhlIGZpbGUKICAgIGVj aG8gIkNvcHkgJHNyY2ZpbGUgdG8gJG5ld2ZpbGUiCiAgICBjcCAkc3JjZmlsZSAvdG1wL3Ny Y2ZpbGUKICAgIGlmIFsgJD8gLW5lIDAgXQogICAgdGhlbgoJZWNobyAiQ29weSBmcm9tICRz cmNmaWxlIC0+IC90bXAvc3JjZmlsZSBmYWlsZWQ7IGFib3J0aW5nIgoJZXhpdCAtMQogICAg ZmkKICAgIGNwIC90bXAvc3JjZmlsZSAkbmV3ZmlsZQogICAgaWYgWyAkPyAtbmUgMCBdCiAg ICB0aGVuCgllY2hvICJDb3B5IGZyb20gL3RtcC9zcmNmaWxlIC0+ICRuZXdmaWxlIGZhaWxl ZDsgYWJvcnRpbmciCglleGl0IC0xCiAgICBmaQoKICAgIGVjaG8gIkZvcmNlIHVud3JpdHRl biBibG9ja3MgdG8gTVREIgogICAgIyBGb3JjZSBhbnkgdW53cml0dGVuIGJ1ZmZlcnMgdG8g TkFORChmbHVzaCB0aGUgcGFnZSBjYWNoZSkKICAgIHN5bmMKCiAgICAjIERyb3AgdGhlIHBh Z2UgY2FoY2UgKGZvcmNlcyByZS1yZWFkIG9mIGRhdGEpCiAgICBpZiBbIC1lIC9wcm9jL3N5 cy92bS9kcm9wX2NhY2hlcyBdOyB0aGVuCgllY2hvICJEcm9wcGluZyBwYWdlIGNhY2hlIgoJ ZWNobyAzID4gL3Byb2Mvc3lzL3ZtL2Ryb3BfY2FjaGVzCiAgICBmaQoKICAgICMgQ2FsY3Vs YXRlIHRoZSBtZDVzdW0gb2YgdGhlIGZyZXNoIGNvcHkKICAgIGVjaG8gIkNhbGN1bGF0ZSBN RDUgb2YgZnJlc2ggY29weSAkbmV3ZmlsZSIKICAgIGNwICRuZXdmaWxlIC90bXAvbmV3Zmls ZQogICAgdGVzdF9tZDU9YGNhdCAvdG1wL25ld2ZpbGUgfCBtZDVzdW1gCgogICAgIyBJZiBp dHMgZGlmZmVyZW50LCB0aGVuIHB1bnQKICAgIGlmIFsgIiRtZDUiICE9ICIkdGVzdF9tZDUi IF07IHRoZW4KIwlzZXQgLXgKCWZhaWxlZF9jbnQ9YGV4cHIgJGZhaWxlZF9jbnQgKyAxYAoJ ZWNobyAibWQ1c3VtOiAkbmV3ZmlsZSAkdGVzdF9tZDUgZGlmZmVyZW50IHRoYW4gZXhwZWN0 ZWQgJG1kNSIKCSMgZmluZCB3aGVyZSB0aGV5IGRpZmZlcgoJcGFnZT0wCgl3aGlsZSBbICRw YWdlIC1sdCAkbnVtcGFnZXMgXTsgZG8KCSAgICBkZCBpZj0vdG1wL3NyY2ZpbGUgb2Y9L3Rt cC9zcmNwYWdlIGJzPSRwYWdlc2l6ZSBza2lwPSRwYWdlIGNvdW50PTEgMj4gL2Rldi9udWxs CgkgICAgZGQgaWY9L3RtcC9uZXdmaWxlIG9mPS90bXAvbmV3cGFnZSBicz0kcGFnZXNpemUg c2tpcD0kcGFnZSBjb3VudD0xIDI+IC9kZXYvbnVsbAoJICAgIGNtcCAtcyAvdG1wL3NyY3Bh Z2UgL3RtcC9uZXdwYWdlCgkgICAgaWYgWyAkPyAtbmUgMCBdOyB0aGVuCgkJZWNobyAiRmls ZXMgZGlmZmVyIGluIHBhZ2UgJHBhZ2U6IgoJCWhleGR1bXAgLUMgL3RtcC9zcmNwYWdlID4g L3RtcC9zcmNwYWdlLmR1bXAKCQloZXhkdW1wIC1DIC90bXAvbmV3cGFnZSA+IC90bXAvbmV3 cGFnZS5kdW1wCgkJZWNobyAiU291cmNlIHBhZ2U6IgoJCWNhdCAvdG1wL3NyY3BhZ2UuZHVt cAoJCWVjaG8gIk1pc21hdGNoIHBhZ2U6IgoJCWNhdCAvdG1wL25ld3BhZ2UuZHVtcAojCQli cmVhawoJICAgIGZpCgkgICAgcGFnZT1gZXhwciAkcGFnZSArIDFgCglkb25lCiMJc2V0ICt4 Cgljb250aW51ZQojCWV4aXQgLTEKICAgIGZpCgogICAgIyByZW1vdmUgdGhlIHByZXZpb3Vz IGNvcHkKICAgIGlmIFsgLW4gIiRsYXN0X2ZpbGUiIF07IHRoZW4KCWVjaG8gIlJlbW92aW5n ICRsYXN0X2ZpbGUiCglybSAkbGFzdF9maWxlCiAgICBmaQoKICAgICMgc3JjZmlsZT0kbmV3 ZmlsZQoKICAgICMgdXBkYXRlIHRoZSBjb3VudAogICAgY250PSRjbnQyCgogICAgIyBEdW1w IG91dCB0aGUgWUFGRlMgZWNjIGNvdW50cwogICAgY2F0IC9wcm9jL3lhZmZzID4gL3RtcC95 YWZmcy5zdGF0cwogICAgY2F0IC90bXAveWFmZnMuc3RhdHMgfCBncmVwICdebl9lY2MnCiAg ICBjYXQgL3RtcC95YWZmcy5zdGF0cyB8IGdyZXAgJ15uX3BhZ2UnCiAgICBjYXQgL3RtcC95 YWZmcy5zdGF0cyB8IGdyZXAgJ15uX2VyYXMnCiAgICBjYXQgL3RtcC95YWZmcy5zdGF0cyB8 IGdyZXAgJ15uX3JldCcKICAgIGNhdCAvdG1wL3lhZmZzLnN0YXRzIHwgZ3JlcCAnXm5fc3Rh bGUnCiAgICBjYXQgL3RtcC95YWZmcy5zdGF0cyB8IGdyZXAgJ3N0cmlrZScKICAgIGVjaG8K CmRvbmUKCmV4aXQgMAo= --------------090306060003070302020903--