Understanding the Project Scope

Building a data logger with a PIC microcontroller and an SD card is an excellent way to capture and store real-world measurements over extended periods. This type of system finds use in environmental monitoring (temperature, humidity, barometric pressure), agricultural soil tracking, equipment runtime logging, and even educational science experiments. The combination of low-cost PIC microcontrollers and widely available SD cards provides a robust platform that can record thousands of data points without needing a PC or cloud connection. This article walks through the entire design, from component selection and circuit assembly to programming, file system handling, and practical testing.

Selecting the Right PIC Microcontroller

PIC microcontrollers are available in a wide range of families with varying memory, pin counts, and peripheral capabilities. For an SD card data logger, the essential features include hardware SPI support, sufficient program memory (flash) for the code, and enough RAM to buffer data before writing to the card. The PIC16F877A remains a popular choice because it offers 8K words of flash memory, 368 bytes of RAM, and a built-in SPI module. However, more modern devices like the PIC18F46K22 or the PIC24FJ series provide larger RAM buffers, more timers, and lower power consumption—useful for battery-powered logging. The PIC18F46K22, for instance, has 3.8K bytes of RAM, allowing for a larger circular buffer that helps when the SD card is busy writing. Decide based on your sampling rate, sensor count, and power constraints.

Key Specifications Checklist

  • SPI module: Hardware SPI reduces processor load and simplifies code.
  • Flash memory: At least 4 KB for a basic logger; 16-32 KB recommended if adding FAT file system and RTC.
  • RAM: Minimum 256 bytes for buffering; preferably 1-2 KB to avoid data loss during write delays.
  • Operating voltage: 3.3V or 5V tolerance. SD cards require 3.3V signaling, so ensure I/O voltage compatibility or use level shifters.
  • Timer modules: At least one timer for fixed-interval sampling.

Understanding the SD Card Interface

SD cards communicate via SPI mode, which is well-supported by PIC microcontrollers. The interface requires four signals: SCK (serial clock), MOSI (master-out-slave-in), MISO (master-in-slave-out), and CS (chip select). A fifth line, CD/DAT3, is sometimes needed for card detection but is optional for basic logging. The SD card must be initialized using a specific sequence: reset the card with command CMD0, then send CMD8 to check voltage range, followed by ACMD41 to initiate initialization. After that, you can read or write data blocks. The card expects data in 512-byte sectors. Writing directly to raw sectors requires managing your own memory allocation, which is complex and not portable. A better approach is to use a lightweight FAT library such as Petit FatFs, which fits in under 4 KB of code space and supports FAT16/FAT32 on resource-constrained microcontrollers.

SPI Configuration Tips

Set the SPI clock speed to a maximum of 400 kHz during initialization phase (as required by SD card spec). After the card is ready, you can raise the clock speed to 8-10 MHz for faster data transfers. Use the PIC's SPI module in mode 0 (CPOL=0, CPHA=0) or mode 3 (CPOL=1, CPHA=1) — typically mode 0 works with most cards. Always enable internal pull-ups on the SD card control lines if your PIC supports them, or add external 10 kΩ resistors to VCC.

Circuit Design and Power Considerations

The SD card operates at 3.3V, while many PIC microcontrollers (like the 16F877A) run at 5V. Directly connecting 5V logic to a 3.3V SD card can damage the card. Use a voltage regulator (e.g., LM1117-3.3) to supply the SD card module, and place a 10 µF electrolytic capacitor plus a 100 nF ceramic capacitor close to the module. For level shifting of the SPI lines, a simple resistive divider (2.2 kΩ / 3.3 kΩ) on MOSI, SCK, and CS works at moderate speeds (up to 1 MHz). Alternatively, use a dedicated level shifter IC like the 74LVCH125 or TXB0104 for higher speeds. Sensors like the DHT22 (temperature/humidity) or the DS18B20 (digital temperature) have their own power supply requirements; typically a 4.7 kΩ pull-up resistor for the data line. For analog sensors (e.g., thermistor + voltage divider), connect the output directly to the PIC's analog input pin.

Power Management for Remote Logging

If the logger will operate on batteries, use a PIC variant with deep sleep modes. The PIC16F1829, for example, has a sleep current of 60 nA. Wake it using a watchdog timer or external interrupt from the RTC alarm. During active logging, keep the SD card powered down (use a MOSFET to cut its supply) when not writing. A typical sequence: wake PIC → power up sensor → take reading → power up SD card module → write data → power down module → go back to sleep. This can extend battery life from days to months.

Adding a Real-Time Clock (RTC) for Accurate Timestamps

A data logger without time information has limited value. Integrating an RTC chip such as the DS3231 or DS1307 provides accurate timestamps. These communicate via I²C, which most PICs support, or you can bit-bang the protocol. The DS3231 is preferred for its built-in temperature compensation and accuracy of ±2 ppm. Wire the SDA and SCL pins to the PIC's MSSP module (if available) with 4.7 kΩ pull-up resistors to 3.3V. In firmware, initialize the RTC once, then read the time before each data write. Format the timestamp as "YYYY-MM-DD HH:MM:SS" or a UNIX epoch 32-bit value to save space in the log file. If an RTC is not used, you can record a relative tick count from the PIC's timer, but that only works if the power stays on.

Firmware Architecture Overview

The firmware for the data logger follows a structured yet modular approach. Break the code into these functional blocks:

  • Initialization: Configure oscillator, GPIO, SPI, I²C (for RTC), ADC (if using analog sensors), and timers.
  • SD card driver: Implement low-level SPI read/write, command send, and CRC check (most modern cards accept CRC disabled for simplicity).
  • File system layer: Incorporate Petit FatFs (or a similar small library) to create, open, write, and close log files.
  • Sensor polling: Read sensors at defined intervals using timer interrupts or a delay loop.
  • Logging engine: Buffer sensor data and timestamps, then flush to SD card in 512-byte blocks.
  • Error handling: Check for SD card remove/reinsert, low voltage, or sensor read failures.

Buffer Management Strategy

Writing one record at a time to the SD card is inefficient and can cause wear on the card. Instead, allocate a circular buffer in RAM large enough to hold multiple log entries. For example, if each log entry is 20 bytes (timestamp + 3 float values), a 512-byte buffer holds 25 entries. When the buffer is full, write the entire block to the card in one go. This reduces SPI overhead and improves performance. On the PIC18F46K22, you can use a buffer of 1024 bytes for even better efficiency. Ensure the file is opened as "append" mode — Petit FatFs supports FA_OPEN_APPEND flag so that new data is written at the end of the file.

Sample Code Structure (Pseudocode)

The following pseudocode outlines the main loop. Actual implementation will differ based on your PIC and libraries.

1. Initialize oscillator to 20 MHz
2. SPI_init(SPI_MASTER, MODE_0, 400kHz)
3. RTC_init(I2C_100kHz)
4. while (SD_init() != OK)
   {
       blink_error_led();
       delay(1000);
   }
5. f_mount(&fs, "", 0);  // mount file system
6. f_open(&fil, "LOG.csv", FA_OPEN_APPEND | FA_WRITE);
7. if (file_size == 0) write_header();
8. while(1)
   {
        wait_timer_1s();
        read_RTC(&ts);
        read_sensors(&temp, &humid);
        format_string(datalog, "%s,%.1f,%.1f\r\n", ts, temp, humid);
        if (buffer_full)
        {
            f_write(&fil, log_buffer, BYTES_WRITTEN, &bw);
            if (bw != BYTES_WRITTEN) error_handler();
        }
   }
9. f_close(&fil);
10. f_mount(NULL, "", 0);

Handling File System and Log Format

Write data in CSV format to make it easy to analyze in spreadsheet software or Python. Include a header row at the top of the file after first opening. Example: Timestamp, Temperature_C, Humidity_pct. The file name can be fixed (e.g., "LOG00.CSV") or derived from the RTC data to avoid overwriting. If using Petit FatFs, note that it does not support long file names, so use 8.3 format (e.g., "LOG0001.CSV"). To manage multiple log files, assign a file number in EEPROM and increment it each time the logger starts. If the card is removed and reinserted, the microcontroller should remount the file system and create a new file to prevent data corruption.

Dealing with SD Card Wear and Data Integrity

SD cards have limited write cycles, but for a data logger writing once per minute, a typical Class 10 card will last years. To improve reliability, consider using a dedicated industrial-grade SD card (e.g., Swissbit or Apacer) with error correction. Always close the file and unmount the file system before powering down. If the microcontroller loses power unexpectedly, you may lose the last buffer (typically up to 512 bytes). To mitigate this, you can flush the buffer to the card after every 10-20 writes. Some PICs have brown-out detection; use it to interrupt normal flow and close the file before voltage drops too low.

Sensor Interfacing Examples

Temperature with DS18B20

The DS18B20 digital sensor uses the 1-Wire protocol. Connect its data pin to a PIC digital I/O (e.g., RD0) with a 4.7 kΩ pull-up to 3.3V. Implementation requires bit-banging the 1-Wire timing sequences: reset pulse, presence detect, skip ROM, start conversion, read scratchpad. The conversion time is up to 750 ms at 12-bit resolution. You can power the sensor with parasitic mode (data line provides power) or use an external 3.3V supply. For logging, start a conversion, then wait (or do other tasks) and read the result. The DS18B20 can be uniquely ID'd, allowing multiple sensors on the same wire.

Humidity with DHT22

The DHT22 (AM2302) provides both temperature and humidity with a single-wire serial interface. Similar to 1-Wire but with different timing. The microcontroller sends a start signal (low for at least 18 ms), then the sensor responds with a 40-bit data packet. Implement a state machine to sample the bits; a fast PIC running at 16 MHz or above is recommended because the bit times are short. The DHT22's accuracy is ±0.5°C and ±2% RH, sufficient for most logging applications.

Analog Light Sensor

For measuring ambient light, use a photoresistor (LDR) in a voltage divider with a fixed 10 kΩ resistor. Connect the midpoint to an ADC input. The PIC's 10-bit ADC (1024 levels) gives reasonable granularity. Calibrate by measuring the voltage under known lux levels using a standard light meter. Log the ADC count or convert to lux using an approximate formula.

Testing and Troubleshooting

Before deploying the logger, test each subsystem independently. Write a simple test program that toggles an LED to verify oscillator and timer accuracy. Next, test SPI communication with the SD card by initializing the card and reading the CID register. Use an oscilloscope or logic analyzer to check SPI signals — common issues include incorrect clock polarity, missing pull-ups, or loose connections. For the file system, test creating, writing, and reading back a small file on a PC after unplugging the card. Insert the card into a computer and open the CSV file to confirm data integrity. If you see garbage characters, check the baud rate (SPI speed) or the byte ordering (fatfs expects little-endian).

Common Pitfalls

  • SD card not initializing: Ensure VCC is 3.3V, not 5V. Some modules come with a 3.3V regulator; verify output. Try a different card (SanDisk or Kingston often work best).
  • File write failure: Did you create a partition (FAT32) on the card? Use a PC to format the card as FAT32 (not exFAT or NTFS). Petit FatFs requires a MBR with a FAT partition.
  • Data corruption: Long wires can cause signal reflections. Keep SPI lines under 10 cm. Add 47 pF capacitors to ground on each line near the card socket.
  • Timer drift: Crystal accuracy matters. Use an external 20 MHz crystal instead of internal oscillator for precise 1-second intervals. If using internal, calibrate using the OSCTUNE register.

Enhancing the Data Logger

Once the basic logger works, consider these additions:

  • Wireless upload: Add an ESP8266 or HC-05 Bluetooth module to transmit data to a smartphone or server in real time. The PIC can offload old data via UART while continuing to log new data.
  • User interface: Use a 16×2 character LCD (I²C or parallel) and a few buttons to set logging interval, view last reading, or trigger manual save.
  • Multiple log files: Rotate files every day or every 1000 records to limit file size and make retrieval easier.
  • Data compression: For high-frequency logging (e.g., 100 Hz), store data in binary format with a packed timestamp (e.g., 4-byte offset from start). Convert to CSV offline using a PC script. This greatly reduces card wear.
  • Over-the-air updates: Advanced users can implement a bootloader via serial to update firmware without removing the PIC.

Real-World Applications

The simple PIC/SD card data logger is the foundation for many practical systems. Industrial monitoring often requires logging vibration, temperature, and current for predictive maintenance. In environmental science, portable loggers are deployed in remote areas to track microclimate changes. Educational institutions use them to teach sensor integration and data analysis. The open-source community provides many complete projects on sites like DigiKey's Maker platform that you can adapt. The key is to start simple, then iteratively add features.

Conclusion

Building a simple data logger with a PIC microcontroller and an SD card is a rewarding project that blends hardware design, embedded programming, and practical data management. By carefully selecting components, understanding the SPI protocol, implementing a lightweight FAT file system, and adding an RTC, you can create a reliable logger for a wide range of applications. The skills developed here—interfacing with digital sensors, managing file I/O on resource-constrained devices, and optimizing power consumption—are directly transferable to more complex embedded systems. Start with a basic breadboard prototype, test thoroughly, and then build a custom PCB for field deployment. The satisfaction of retrieving an SD card full of clean, timestamped data is worth the effort.