Saturday, April 5, 2014

SD card logger library with log rotation that fits on ATmega8

This library implements an SD card Data Logger that runs on ATmega.
It has a small footprint, so it can be loaded on an ATmega8, leaving space for user code.
It supports SD and microSD cards formatted with FAT16.
It also features log rotation.


The "Petit FAT File System Module" by ChaN (http://elm-chan.org/fsw/ff/00index_p.html) it is used to write on SD card. I've used the Petit library because i would like to build a small footprint logger. Even if it has a few limitations, those can be circumvented.

We had to format the card we would like to use with FAT16, and then load it with a predefined number of empty files of a know dimension. Once we have files on the card, we can write on those files.
You can create empty files by using the python helper provided in code.
Firmware side we will setup the file dimension, and file number. File dimension, can not be greater than 2^16 bytes because a uint16_t type variable it is used to store this information, also max number of files it is limited to 256, becayse a uint8_t variable it is used.

Given the number of files used by the logger and every file size, we just have to record the last written position and the file number we are using to implements the log rotation. When a file is filled up with logged data, it skips to the next one, if the file is the last one, we go back to first.
Note that Petit FAT File System Module write files using blocks, so the "seek to position" can be used only on the block dimension. Typically a block is 512 bytes.

Power loss failure are managed by recording to eeprom the above parameters every 60 seconds (you can setup this timing option in code). When power get's back, the logger will use the last file and position recorded.
File write failure will cause the file to skip to the next log file.

The sample code provided just logs time + a string.
Date and time are obtain using a DS1307 chip. In the code you can also find a python helper to configure date and time through UART.

The SD card board, is an ebay cheap one, it basically just has voltage regulation to prevent the SD card to be damaged.

You can find schematics below:


This library was developed on Eclipse, built with avr-gcc on Atmega8 @ 8MHz.


Code

Notes
  • read risk disclaimer
  • excuse my bad english



13 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Hello, I have a mysterious problem while building your project in AVR Studio 4 and in Eclipse Indigo too. I get the same error messages while building your your code:
    Programs say errors are in line number 9.

    http://pastebin.com/GkHp3ksw

    Errors messages:

    `)' required usi.S /avr_sdloggers/src/pfatfs line 49 C/C++ Problem
    garbage at end of line usi.S /avr_sdloggers/src/pfatfs line 49 C/C++ Problem
    make: *** [src/pfatfs/usi.o] Error 1 avr_sdloggers C/C++ Problem

    If you had all of these errors how could you fix them?

    ReplyDelete
    Replies
    1. Are you compiling with avr-gcc, what version?

      Delete
    2. I use WinAVR-20100110 with Win7 x86 enviroment.

      Delete
    3. These are included in the exlipse project:

      C:/WinAvr20100110/avr/include.
      C:/WinAvr20100110/lib/gcc/avr/4.3.3/include
      C:/WinAvr20100110/lib/gcc/avr/4.3.3/include-fixed

      Delete
    4. 90% It's the F_CPU number ending with UL problem.
      Just use this
      ${COMMAND} ${FLAGS} -DF_CPU=${AVRTARGETFCPU} ${OUTPUT_FLAG} ${OUTPUT_PREFIX}${OUTPUT} ${INPUTS}
      as your "C/C++ Build"/"Settings"/"AVR Assembler"/"Command line pattern:"
      Or add in "usi.S" after "#include " the line
      #define F_CPU 8000000
      (or you F_CPU clock) - note do not use the UL ending here.
      That way should works.

      Delete
    5. F_CPU was missing from the code, but the project contained it in the options. It's strange but it works if I type F_CPU in usi.S. Both IDEs succeded to build it now. I thank you for your help.

      Delete
  3. Well, I've been testing your sd logger project. First I successfully created my own functions to write float, integer, string on sd card.Then went ahead and adopted your project to atmega32. I could write those .log files you had provided in the project. All seemed to work as you published. What I want is to record bmp180 measurement datas. I can't see clearly why you use WDT in your code? I was using only your bmp180 project + my LCD library to show measurement datas. So bmp180 library seemed to work after I did level shifting even after 2 days runtime.

    I now as mentioned adopted your SD card project. I used no WDT in code for the very first time. Result: code was running for circa 20 mins then goes to infiinte loop somehwere.(It acts as if BMP180's SDA or SCL pins was disconnected ) I wonder whether you have faced the same problem and that's why you inserted WDT function into your code?

    ReplyDelete
    Replies
    1. I do not encount this error in other sd logger project and other bpm180 project. But i've never made a project that involves both sd logger and bmp180, so it could be a bmp180 / sdlogger incompatibiltity.
      Watchdog is here to prevent any software fail, I usually test firmware with watchdog disable, then in production i enable it.
      You could try to scope around the SDA / SCL pin after 20 mins. Let me know if you find the problem.
      Also check your memory heap. It could be that you are using too much resources for the mega32.

      Delete
  4. Imagine my electricity meter sending a package of data through an infrared LED every 1 second. I want to log this using a device like yours.

    Can I connect a photo diode or transistor to it and log smart meter data in SML (smart message language)?

    324 bytes, send at 104µs ~9600 bauds

    http://www.vde.com/en/fnn/extras/sym2/Infomaterial/documents/sml_080711_102_eng.pdf

    It would help a lot of people in my country to monitor their devices and power consumption. I live in an apartment building where I can only use something as small as arduino to log my power consumption.

    ReplyDelete
    Replies
    1. Hello, of course you can. You can use the template code of this project as startup. You have just to change the test line: strcat(logger_buffer, "test") with the reading decoded from your meter. You would have to write the bitbang decoder for your meter.

      Delete
    2. Thank you, I was unsure for a moment if Petit FS can append to the same file until it hits the 4GB FAT limitation. There are projects that would allow me to do this easier, but this seems more fun. Hopefully the less tech and code I use, the longer the battery of my logger will live.

      Delete
    3. My suggestion is to logrotate over many files, with lower dimension. That way, if something goes wrong, you do not loose all your files. Another suggestion, if possible, use a NET logger, something like the xively.com logger you can find here in my blog, or you can even write the server side code for a NET logger. It would not be difficult.

      Delete