Arduino: SD storage: troubleshooting guidline – public worldwide collaboration

[also posted in New fast data logging sketches]

I started a google document to give some overview on ways to troubleshoot SD card usage with Arduino.

“pls have a first look, put your comments in that and then we can decide to create a (sticky?) post in Arduino:Storage to promote this document and the ongoing work on it. As soon as we collaboratively decide, that it`s kind of ready for the wiki, we can copy or link it there. Maybe over time we can identify some named editors keeping the edit access and everyone else can only comment. Let´s see how this works, it´s also the first public worldwide community document editing for me smiley

And everyone please feel free to edit, modify, restructure whatever you want in this document. In the worst case we can revert to any previous version thanks to the cloud :o)

As this document is stored under my account @ google I hereby reserve all rights to remove any inappropriate, abusive or whatsoever unhelpful stuff that may come in; but of course the technical content remains owned by the community.”

https://docs.google.com/document/d/1lGFHpHPfvfGxSzjU8yo5DBLjO2pM0Bl7nwp85h4pftI/edit?usp=sharing

[gdocs id=1lGFHpHPfvfGxSzjU8yo5DBLjO2pM0Bl7nwp85h4pftI type=’document’]

Arduino writing to SD Card – Next test based on Fat16Libs fast data logging sketches

 

As all my previous attempts only lead to one sample per 5milliseconds on average, I started to try the library and example from fat16lib, you can find this library here: New fast data logging sketches

My first measurements showed a sampling rate of about 62,5k samples per second (one sample every 16µs). This is a factor 375 better than before, but so far I only wrote the voltage measurement and micros()%256 as timestamp. In my previous tests I wrote micros(), voltage, pulse width and current to the SD, which was around a factor 8 of data. First guess: with the new approach I could achieve an improvement by a factor of 46 – further ADC and digital PIN reading not yet considered.

Next I changed to full ADC 16Bit logging..errr…10 Bit on a 328p 😉

Before coming to the examples, let´s start with the reference, the voltage as recorded with the oszilloscope:

IMG_20130324_002451 IMG_20130324_002618IMG_20130324_002646

Here we see the voltage jump, which takes about 80µs according to the scope.

Measurements with the Arduino FastLogger

Example 1, Sample interval 25.00µs: ADC Volt is the ADC reading, ie 0-1024 (but as my calibration isn´t correct anyway…this is only for illustration)

Configuration:First measurements....
ADC clock MHz: 1.000
Sample interval usec: 25.00
Sample Rate: 40000.00
Record time ms: 2004
Sample count: 160268
Overruns: 0

 

Example 2, Sample interval 8µs:

Here you can see that the number of overruns dramatically increases and the number of samples is even lower than with the lower sample rate in example 1. We can also see large gaps over the whole duration of the measurement. In the detail of the first phase it shows only little more detail than the measurement in example 1.

Configuration:Measurement with 125000Khz Sample Rate
ADC clock MHz: 2.000
Sample interval usec: 8.00
Sample Rate: 125000.00
Record time ms: 2020
Sample count: 27210
Overruns: 164254

Measurement with 125k - Detail

 

 

 

 

Example 3, sample interval 12.5µs:

Still a lot of overruns….

Configuration:Fat16FastLibTest4
ADC clock MHz: 2.000
Sample interval usec: 12.50
Sample Rate: 80000.00
Record time ms: 2007Fat16FastLibTest5
Sample count: 91996
Overruns: 87106

 

 

 

 

 

 

Example 4, sample interval 16µs

Configuration:Fat16FastLibTest6
ADC clock MHz: 1.000
Sample interval usec: 16.00
Sample Rate: 62500.00
Record time ms: 2007
Sample count: 122444
Overruns: 47098

Fat16FastLibTest7

 

 

 

 

 

Example 4, sample interval 20µs

ADC clock MHz: 1.000
Sample interval usec: 20.00
Sample Rate: 50000.00
Record time ms: 2008
Sample count: 134646
Overruns: 33000

Conclusion

Depending on the SD card and hardware you are using, you need to experiment with the sample rate and need to decide, how many overruns you can accept. In my current test setup 40000 seems to be the best option; here I most often get 0 overruns. Ok, I could now fiddle somewhere betweein 40000 and 50000, but I doubt this will be worth the effort before I test with the real measurement of RPM, voltage and current….

Sample Rate Overruns Sample count Sample interval usec ADC clock MHz
 40000.00 0 160268  25.00 1.000
 50000.00 33000 134646  20.00 1.000
 62500.00 47098 122444  16.00 1.000
 80000.00 87106 91996  12.50 2.000
 125000.00 164254 27210  8.00 2.000

 

Way forward

Let´s see, when I want to spend time on this again. Next todo is to combine this with my previous approach, where the interrupt is triggered from the RPM measurement impulses and only in the interrupt voltage and current are measured. I´m wondering how this will work together with the buffering in the fastLogger lib – and if 328p memory will suffice for all together 🙂

 

More interesting videos and reading

After some hours of Sebring 12h livestream and a relaxing mindfulness workshop, I did some youtube-surfing as TV only had the usual nonsense.

So….below is the continued list. See Mal ein paar interessante Videos und Leseliste for the previous post and the Thinking category for all of it.

Dan Pink: The puzzle of motivation

The…ok…one of the mismatches between science knowledge and business doing. A lot around the candle model and applying incentive on 21st century creative workers.
Rewards leading to lower performance by limiting creativity. So it´s about doing less of the known and meanwhile useless procedures and getting rid of traditional management methods, which are great for compliance but bad for engagement. Results oriented work environment!

how to be fearlessly creative by Justine Musk

Successful people do not lead statistically sensible lives. — Srinivasan S Pillay” ….and about Pandora´s Box.
“Your brain is the arena.”

Tom Wujec: 3 ways the brain creates meaning

6 minutes absolutely worth it. How seeing works and how meaning evolves through Visual interrogation and mental model.

Self Control: Dan Ariely at TEDxDuke

Ever had more of anything you wish you had? “In the future we will be wonderful people” 🙂
The time dimension in controlling your life and the impacts of rewards.

David Anderson: Your brain is more than a bag of chemicals

Learning from fruit flies to find a cure for mental disorders.

Die Welt fragt, Gunter Dueck antwortet

German podcast on various current topics.

The Believing Brain – Presented by Dr Michael Shermer

How we are easily being made believe things, we better didn´t believe. For example how we intuitively make meaningful patterns our of information noise.

See also “Fast thinking, slow thinking” in my list of books.

How to Be Mindful in an ‘Unmanageable’ World by Tony Schwartz @ HBR.ORG

Great reference on Wisdom 2.0, a conference linking the (virtual) social era media to mindfulness.