Experiences with Arduino,SD Card and Memory consumption – Now It works !!!

Getting an SD card to work on my Arduino Uno (I changed boards with my father – Thanks! 🙂 ), was an interesting endeavour during the last days.

I was aware that getting an SD card to work with an Arduino can be a funny experiment. See for example: http://www.ladyada.net/products/microsd/, which is a great tutorial getting started and set up the wiring, but already details some issues, which generally arise around using SD cards with Arduino. As a side note: in parallel I watched the great Arduino video tutorials on http://www.jeremyblum.com/ . Gr8 stuff!

What I tried and didn´t help…..

Each of the following items may have helped, but I´m too lazy to try if taking these preconditions away would break the system:

  • Different SD cards, in particular exchanging SDHC card by a non-high speed SD card.
    Meanwhile I can report that also SDHC card (<= 2GB) seems to work
  • Providing external power supply instead of powering from USB only
  • Trying different libraries and ways to work with the SD card
    Only one of these libraries really worked, see below

What helped – Watch out for memory overflows

Some time during all the testing it occurred to me that all attempts and failures felt very arbitrary.

Short version: I had buffer / heap overflows, because the libraries I first used require lots of memory before even using them during runtime. In particular after initializing the SD card free heap memory partially went down to 200 Bytes. Add to that some function calls increasing the consumed stack size (used for function paramters and local variables) plus lots of nested calls: voila, you get arbitrary behaviour and sometimes the Arduino even resets, for example because the stack pointer (SP) wanders into the used heap.

I had set up a small console program for the arduino such that I could work from the Arduino PC Serial Monitor with commands like, create file, delete file, dir/ls (directory listing etc).

Using this arduino program I can analyze memory consumption and memory leaks induced by the different file operations. In particular ls/dir proved to be a pain in the …memory ;), because of the traversal of directories and files, which by some libs or examples is even done by a recursive algorithm.

Tools for Arduino memory analysis during runtime

The Fat16 library includes a function FreeRam() could maybe have helped, but I found this only after all the other fuzz described above.
The following two references and coding very very helpful to show the current values of SP (StackPointer) and free/allocated heap memory during runtime of my Arduino sketch.
http://playground.arduino.cc/Code/AvailableMemory  and http://andybrown.me.uk/wk/2011/01/01/debugging-avr-dynamic-memory-allocation/

Final Setup

Finally my arduino program now works using the Arduino Fat16 library on the root folder only of the SD card and this works stable at least in short time tests and according to my memory usage analysis. As my use case does not include long-term tracking and not lots of files are involved I should be fine with that, but will keep an eye on memory – which anyway is a good idea on a 2K RAM microcontroller 🙂 SerialMonitorConsole

Information and coding sources

Future Art ?

I started to read some things about Arduino boot loaders capable to load and start programs from SD card. In combination with approaches for Arduino development environments on Android, I could potentially create sketches on my tablet and directly download them to an arduino (so far no such option seems to exist….).

Loading a new Bootloader to my Diecimila/ATMega168

After somehow I couldn´t upload any new software to my Arduino I learned how to use the Popolu USB AVR Programmer I luckily already had purchased before.

Even worse than that I didn´t immediately recognize that my software updates weren´t loaded to the Arduino, because somehow I didn´t see the upload error messages when working with the Arduino VS plugin; only in the Arduino programming software I saw those messages.
Attention: If you use this description for any of your purposes you do it on your own risk. I shall not be kept liable in any manner and I make no warranties.

Symptoms

  • I was getting upload verification errors
  • Messages like ‘not in sync’

Root Cause

I still have no clue. My best guess is that I had some voltage glitches from the electronics attached to my Arduino….

Connecting Programmer with PC and Arduino

  1. Connect Polulu to USB (PC)
    The Green light must be constantly on; anything else indicates that the PC connection doesn´t work (e.g. wrong USB driver)
  2. Power up Target Device (in my case Arduino Duemilanove)
  3. Connect ISP cable from Polulu to Target Device
    The yellow led on Pololu must blink

Uploading the Bootloader to the Arduino

As the whole procedure didn´t work from the Arduino IDE, I learned to use AVRDude from the command line. As an additional hurdle, avrdude.conf was missing on my windows installation.
http://www.ladyada.net/learn/avr/avrdude.html was a great help 🙂

I then created the following batch file

Attention: be sure to use the right COMxx interface

” avrdude -p m328p -P COM2 -c avrispv2 -e -U flash:w:bootloader.hex
” avrdude options doku under http://www.nongnu.org/avrdude/user-

manual/avrdude_4.html#Option-“Descriptions
” my Pololu connects on COM15
” -e erases what´s been there before
” -U flash  says that the flash ROM shall be written
” -v verbose
rem http://www.geocities.jp/arduino_diecimila/bootloader/files/avrdude.conf
rem Geocities 🙂 LOL. Didn´t know it still exists 😉
rem COM16 !!! Argh. Cost mw 30min to find out I´d used the wrong COM 🙁
C:
cd “C:\Program Files\ArduinoSoftware\Entwicklungsumgebung\Windows\arduino-1.0.1\hardware\arduino\bootloaders\atmega\”
“C:\Program Files\ArduinoSoftware\Entwicklungsumgebung\Windows\arduino-1.0.1\hardware\tools\avr
\bin\avrdude.exe” -C “C:\Users\Robert\Desktop\Arduino\avrdude.conf” -p m168 -P com16 -c avrispv2 -e  -v -U flash:w:ATmegaBOOT_168_diecimila.hex
pause

avrdude.conf

As no avrdude.conf file was on my machine, I created the following avrdude.conf
Attention: This must of course fit your programmer and target device.

Update 08/19/14:

had errorswhen writing the bootloader to a 328p :  stk500v2_command() command failed rc=-1. When using the -F option of AVRDUDE, it said the device signature was invalid:

avrdude.exe: Device signature = 0x000000
avrdude.exe: Yikes! Invalid device signature.
avrdude.exe: Expected signature for ATMEGA328P is 1E 95 0F

I then added the -B and -i options to the AVRDUDE call:

“C:\Program Files (x86)\Arduino\hardware\tools\avr\bin\avrdude.exe” -C “C:\Program Files (x86)\Arduino\hardware\tools\avr\etc\avrdude.conf” -p m328p -P
com3 -c avrispv2 -e -v -U flash:w:ATmegaBOOT_168_atmega328.hex -i 90 -B 22 -F

 

Update 08/13/18:

My father has now also ‘repaired’ an Arduino Uno, it worked fine with  Atmel Studio 7.0 and these descriptions :

[HowTo] Workaround für Heimgebrauch: Dropbox->Synology Copy via Android Device+Fragen

[27.2.2013] Meanwhile I found a description for a DSM plugin to Sync Dropbox->Synology (unidirectional only). http://otmanix.de/?p=202  It references a beta plugin. Didn´t try it myself yet.

[29.7.2012] mangels direkter Dropbox Integration auf DSM habe ich mir für den Privatgebrauch den Workaround gebastelt wie im beigefügten Bild dargestellt. Evtl. hilft die Idee auch anderen weiter. Mir reicht das so als Backup Lösung, dezentrales Backup läuft zusätzlich separat. Das soll natürlich keine Empfehlung für Produktivbackups sein.
Meine Rahmenbedingungen:
– geringes Datenvolumen in der Dropbox
– Android Tablet verfügbar und liegt tagsüber ohnehin nutzlos zuhause rum
– kritische Daten sind in der Dropbox bereits per Boxcryptor verschlüsselt oder vorher lokal per TrueCrypt, daher kein weiterer Bedarf nach Verschlüsselung
– ich will nicht noch einen separaten PC starten müssen, damit Dropbox Kopien gezogen werden.

Erforderliche Android Apps und Kenntnisse
– Dropsync
– Tasker
– FTPSyncX Pro (läßt sich als PlugIn in Tasker integrieren)
– SSH Zugang zur Synology, Skripte anlegen, Cron Jobs pflegen

Dropbox Backup Overview

Dropbox Backup Overview

Zeitplan für meine Jobs dafür:
Each 30 min. Dropsync Dropbox -> Tablet
10:30 Backup Dropsync Tablet -> NAS
11:00 Create TabletVersion NAS -> NAS
11:30 TabletVersions (incl. Dropsync) NAS -> eSATA ext.

Wer noch eine tolle Möglichkeit kennt a) komplett-Backups eines Android Device zu erstellen ohne zu rooten b) wie man sowas geschickt überwacht, bisher habe ich nur die Möglichkeit Erfolgsmails zu senden, aber eigentlich möchte ich ja genau das umgekehrte Verhalten, nämlich ein Mail oder anderer Alert, wenn etwas schiefgegangen ist. bitte hier ergänzen….

Auch hier gepostet http://www.synology-forum.de/showthread.html?30785-HowTo-Workaround-f%FCr-Heimgebrauch-Dropbox-gt-Synology-Copy-via-Android-Device-Fragen&highlight=Howto+dropbox

Synchronizing my devices for reading stuff later

Meanwhile I have different devices and apps plus cloud automators to read stuff later,  the stuff being articles found on RSS feeds,  twitter,  google plus.
To retain some overview I created a simple picture

The services are:
www.wappwolf.com
www.ifttt.com If This Then That
www.getpocket.com (fka Read It Later)
read.amazon.com
www.crofflr.com

Browser PlugIns, Extensions:
Send To Kindle (Chrome)

Privacy and protection tools for safer browsing

Tools for ‘Safer Browsing’

…. helping to protect your privacy (to some extent….)

Protection Tools

Basics: you run antivirus (e.g. antivir.de offers a very good freeware one) and a firewall.

Additions: e.g. Computerbild Abzockschutz (translate yourself 😉 ). http://www.computerbild.de/artikel/cb-Special-Sicherheits-Center-Internet-Abzocke-COMPUTER-BILD-schuetzt-4406145.html

Fire Fox PlugIns  (Sequence follows my personal rank of importance…)

Chrome

  • Some of the above FF addons are available on Chrome or in slight variations

Tools outside browser

Beware – depending on your location the use of these tools is legally not permitted or e.g. prohibited by your company


also see https://plus.google.com/111122095369026968993/posts

Museen

  • Speyer
  • Frankfurt
  • Schwetzingen
    • Schloß / Park. Sicher sehr schön zu Pfingsten
  • Heidelberg
    • Kurpfälzisches Museum – http://www.museum-heidelberg.de/servlet/PB/menu/-1/index.html
      “Kultur und Kunst des Rhein-Neckar-Gebiets von der Urzeit bis zur Moderne. Der Barockbau beherbergt Schätze der Kurpfalz, eine Gemäldesammlung und

Wanderungen um Heidelberg

  1. Birkenau B7 Essen in Rohrbach (alter Bauernhof)
    Gorxheimertal: http://www.gpsies.com/map.do?fileId=pldrbwhonhhbhdso
  2. Trösel – Löhrbach (entlang B7-B4) – Buchklingen – Unterflockenbach – Trösel
  3. 3. Rimbach – R2 –Einkehren Juhöhe – über Bonsweiher zurück – 20km
    Rimbach – Juhöhe: http://www.gpsies.com/map.do?fileId=ymofbjodewplzedd
  4. Vier Burgen Neckartal Hirschhorn etc.
  5. Pfalz Hohe Loog
  6. Pfalz Lindelbrünn
  7. Hochschwarzwald
  8. Elsaß, zB Burg Löwenstein
  9. Pfalz Trifels

 

Besuchsprogramm in Heidelberg

 

  • Espresso bei Robert 🙂
  • Abends ?? IMax in Speyer?  Oder PLanetarium Mannheim? (Oder Staatstheater Mannheim… )
  • Brunch Lindbergh (Flughafen Mannheim)

siehe Besuchsprogramm