Arduino Communications - SQLite

From Combustory
(Difference between revisions)
Jump to: navigation, search
(→Data Slam Method)
Line 22: Line 22:
  
 
<source lang="c">
 
<source lang="c">
 +
/*
 +
* Data Slam v.01
 +
* by <http://www.combustory.com> John Vaughters
 +
*/
 +
#include "Wire.h"
 +
// Global Variables
 +
unsigned int diff_mil_time;
 +
long last_mil_time;
 +
 +
void setup() {
 +
  Wire.begin();
 +
  Serial.begin(115200);
 +
}
 +
 +
void loop() {
 +
//  diff_mil_time = (int) (millis() - last_mil_time);
 +
  diff_mil_time = (unsigned int) (millis() - 1 +1);
 +
//  last_mil_time = millis();
 +
  Serial.print(diff_mil_time);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(2),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(3),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(4),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(5),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(6),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(7),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(8),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(9),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(10),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(11),DEC);
 +
  Serial.print(",");
 +
  Serial.print(digitalRead(12),DEC);
 +
  Serial.print(",");
 +
  Serial.print(analogRead(0),DEC);
 +
  Serial.print(",");
 +
  Serial.print(analogRead(1),DEC);
 +
  Serial.print(",");
 +
  Serial.print(analogRead(2),DEC);
 +
  Serial.print(",");
 +
  Serial.print(analogRead(3),DEC);
 +
  Serial.print(",");
 +
  Serial.print(analogRead(4),DEC);
 +
  Serial.print(",");
 +
  Serial.print(analogRead(5),DEC);
 +
  Serial.println("");
 +
  delay(45);
 +
}
 +
//*****************************************************The End***********************
  
 
</source>
 
</source>
Line 28: Line 85:
  
 
<source lang="bash">
 
<source lang="bash">
−
 
+
#!/bin/bash
−
 
+
# Data Slam Script v0.1
 +
# User define Function (UDF)
 +
LogLine(){
 +
  echo -E "`date +%s`,${line}"
 +
sqlite3 arduino.db3 "insert into arduino_data(time_stamp,millis,di2,di3,di4,di5,di6,di7,di8,di9,di10,di11,di12,ai1,ai2,ai3,ai4,ai5,ai6) values (`date +%s`,${line})"
 +
}
 +
### Main script stars here ###
 +
# Store file name
 +
FILE=""
 +
 +
# Make sure we get file name as command line argument
 +
# Else read it from standard input device
 +
stty -F /dev/ttyUSB0 cs8 115200 ignbrk -brkint -icrnl -imaxbel -opost -onlcr -isig -icanon -iexten -echo -echoe -echok -echoctl -echoke noflsh -ixon -crtscts
 +
if [ "$1" == "" ]; then
 +
  FILE="/dev/ttyUSB0"
 +
else
 +
  FILE="$1"
 +
  # make sure file (serial device) exist and is readable
 +
  if [ ! -f $FILE ]; then
 +
  echo "$FILE : does not exists"
 +
  exit 1
 +
  elif [ ! -r $FILE ]; then
 +
  echo "$FILE: can not read"
 +
  exit 2
 +
  fi
 +
fi
 +
# Create Database if it does not exist
 +
 +
if [ ! -f "arduino.db3" ]; then
 +
  echo "Creating database"
 +
        sqlite3 arduino.db3 "CREATE TABLE arduino_data (time_stamp integer ,millis integer, di2 integer, di3 integer, di4 integer, di5 integer, di6 integer,
 +
                          di7 integer, di8 integer, di9 integer, di10 integer, di11 integer, di12 integer,
 +
                          ai1 integer, ai2 integer, ai3 integer, ai4 integer, ai5 integer, ai6 integer);"
 +
fi
 +
exec 3<&0
 +
exec 0<"$FILE"
 +
while true
 +
do
 +
 +
# use $line variable to process line in processLine() function
 +
while read -r line
 +
        do
 +
              LogLine $line
 +
        done
 +
      usleep 50000 # This delay can be changed to match the delay of the Arduino
 +
done
 +
exec 0<&3
 +
exit 0
 
</source>
 
</source>
  
Line 51: Line 155:
  
 
<pre>
 
<pre>
−
 
+
1327361512,1752,0,0,0,0,0,0,0,0,0,0,0,506,499,491,483,1023,1023
 +
1327361512,1802,0,0,0,0,0,0,0,0,0,0,0,506,500,491,482,1023,1023
 +
1327361512,1852,0,0,0,0,0,0,0,0,0,0,0,507,500,490,482,1023,1023
 +
1327361512,1901,0,0,0,0,0,0,0,0,0,0,0,507,501,491,482,1023,1023
 +
1327361513,1951,0,0,0,0,0,0,0,0,0,0,0,507,499,491,482,1023,1023
 +
1327361513,2001,0,0,0,0,0,0,0,0,0,0,0,507,501,491,483,1023,1023
 +
1327361513,2052,0,0,0,0,0,0,0,0,0,0,0,506,499,492,484,1023,1023
 +
1327361513,2102,0,0,0,0,0,0,0,0,0,0,0,506,500,492,484,1023,1023
 +
1327361513,2152,0,0,0,0,0,0,0,0,0,0,0,507,500,491,483,1023,1023
 +
1327361513,2202,0,0,0,0,0,0,0,0,0,0,0,507,500,492,484,1023,1022
 
</pre>
 
</pre>
  

Revision as of 23:34, 23 January 2012

Welcome to Combustory


Any questions or comments:

  • Send them to - combustor@combustory.com

Contents

Overview

When writing code these days it is silly to not use a database to manage your data. It significantly cuts the amount of code needed to manage data. So why not just collect the Arduino output directly into a database and allow any software access to the database?

This article shows you how to use a database to collect Arduino data directly from the serial port and place the data into an SQLite database using scripts. This article will provide examples for Linux and Windows.

Technical Assumptions

  • Users are familiar with programming the Arduino
  • Users know how to install and use SQLite (Most Linux computers have it by default)
  • It would be helpful to be knowledgeable about bash scripting

Linux

Ok, well I am starting with Linux. Why? Well.... Linux is smooth! It treats all devices like files. Meaning once you know how to open a file in Linux, you basically know how to access any device. How great is that? Someone was thinking when they created that idea. The operating system drivers are made to make this possible. So in the case of a serial port all you have to do is an stty command to set the transmission rates and the communications parameters and BOOM... like magic a serial port can be treated like a file. This allows you to use common commands that are used on files on a serial port.

Data Slam Method

The goal here is to push out all the data of the Arduino as fast as we can and load it into the database. To do that use this Arduino Sketch: 

/*
 * Data Slam v.01
 * by <http://www.combustory.com> John Vaughters
*/
#include "Wire.h"
// Global Variables
unsigned int diff_mil_time;
long last_mil_time;
 
void setup() {
  Wire.begin();
  Serial.begin(115200);
}
 
void loop() {
//  diff_mil_time = (int) (millis() - last_mil_time);
  diff_mil_time = (unsigned int) (millis() - 1 +1);
//  last_mil_time = millis();
  Serial.print(diff_mil_time);
  Serial.print(",");
  Serial.print(digitalRead(2),DEC);
  Serial.print(",");
  Serial.print(digitalRead(3),DEC);
  Serial.print(",");
  Serial.print(digitalRead(4),DEC);
  Serial.print(",");
  Serial.print(digitalRead(5),DEC);
  Serial.print(",");
  Serial.print(digitalRead(6),DEC);
  Serial.print(",");
  Serial.print(digitalRead(7),DEC);
  Serial.print(",");
  Serial.print(digitalRead(8),DEC);
  Serial.print(",");
  Serial.print(digitalRead(9),DEC);
  Serial.print(",");
  Serial.print(digitalRead(10),DEC);
  Serial.print(",");
  Serial.print(digitalRead(11),DEC);
  Serial.print(",");
  Serial.print(digitalRead(12),DEC);
  Serial.print(",");
  Serial.print(analogRead(0),DEC);
  Serial.print(",");
  Serial.print(analogRead(1),DEC);
  Serial.print(",");
  Serial.print(analogRead(2),DEC);
  Serial.print(",");
  Serial.print(analogRead(3),DEC);
  Serial.print(",");
  Serial.print(analogRead(4),DEC);
  Serial.print(",");
  Serial.print(analogRead(5),DEC);
  Serial.println("");
  delay(45);
}
//*****************************************************The End***********************

Once the Arduino sketch is loaded, then use the code below to create a BASH script with a file named "read_arduino.sh". 

#!/bin/bash
# Data Slam Script v0.1
# User define Function (UDF)
LogLine(){
  echo -E "`date +%s`,${line}"
 sqlite3 arduino.db3 "insert into arduino_data(time_stamp,millis,di2,di3,di4,di5,di6,di7,di8,di9,di10,di11,di12,ai1,ai2,ai3,ai4,ai5,ai6) values (`date +%s`,${line})"
} 
### Main script stars here ###
# Store file name
FILE=""
 
# Make sure we get file name as command line argument
# Else read it from standard input device
stty -F /dev/ttyUSB0 cs8 115200 ignbrk -brkint -icrnl -imaxbel -opost -onlcr -isig -icanon -iexten -echo -echoe -echok -echoctl -echoke noflsh -ixon -crtscts
if [ "$1" == "" ]; then
   FILE="/dev/ttyUSB0"
else
   FILE="$1"
   # make sure file (serial device) exist and is readable
   if [ ! -f $FILE ]; then
  	echo "$FILE : does not exists"
  	exit 1
   elif [ ! -r $FILE ]; then
  	echo "$FILE: can not read"
  	exit 2
   fi
fi
# Create Database if it does not exist
 
if [ ! -f "arduino.db3" ]; then
  	echo "Creating database"
        sqlite3 arduino.db3 "CREATE TABLE arduino_data (time_stamp integer ,millis integer, di2 integer, di3 integer, di4 integer, di5 integer, di6 integer,
                          di7 integer, di8 integer, di9 integer, di10 integer, di11 integer, di12 integer,
                          ai1 integer, ai2 integer, ai3 integer, ai4 integer, ai5 integer, ai6 integer);"
fi
exec 3<&0
exec 0<"$FILE"
while true
do
 
	# use $line variable to process line in processLine() function
	while read -r line
        do
              LogLine $line
        done
       usleep 50000 # This delay can be changed to match the delay of the Arduino
done
exec 0<&3
exit 0

Now make the file executable with the following command. 

chmod 755 read_arduino.sh

Run the script with the following command 

./read_ardiuno.sh

At this point, if you are lucky, you are seeing data separated by commas spit out on your screen. This is the data being loaded into SQLite. To exit the script, hit ctrl-c (^c). Now check if the SQLite database was created with the command. 

ls -la arduino_data.db3

If you see the file, then run this command to see your data 

sqlite3 arduino_data.db3 'select * from arduino_data'

Sample Data: 

1327361512,1752,0,0,0,0,0,0,0,0,0,0,0,506,499,491,483,1023,1023
1327361512,1802,0,0,0,0,0,0,0,0,0,0,0,506,500,491,482,1023,1023
1327361512,1852,0,0,0,0,0,0,0,0,0,0,0,507,500,490,482,1023,1023
1327361512,1901,0,0,0,0,0,0,0,0,0,0,0,507,501,491,482,1023,1023
1327361513,1951,0,0,0,0,0,0,0,0,0,0,0,507,499,491,482,1023,1023
1327361513,2001,0,0,0,0,0,0,0,0,0,0,0,507,501,491,483,1023,1023
1327361513,2052,0,0,0,0,0,0,0,0,0,0,0,506,499,492,484,1023,1023
1327361513,2102,0,0,0,0,0,0,0,0,0,0,0,506,500,492,484,1023,1023
1327361513,2152,0,0,0,0,0,0,0,0,0,0,0,507,500,491,483,1023,1023
1327361513,2202,0,0,0,0,0,0,0,0,0,0,0,507,500,492,484,1023,1022

Troubleshooting

So you weren't lucky and you received errors somewhere in this process. Let's verify a few things first.

Windows

If you jumped straight to this section, I recommend that you read the Linux section and understand the overall concept. The problem with Windows is that the communication with serial ports is not as easy. The only way to communicate through software that you write is to have a license to the software objects that provide communication to serial ports. Well we cannot have that, so we will have to resort to an open source solution.

TeraTerm Pro is a terminal emulator that can communicate with serial ports as well as telnet and ssh. It also has a very rich macro language for automating terminal sessions that we will use to talk with the Arduino. There are many great applications for this software, like using it to control or configure devices that are accessed via ssh or telnet. So if you are new to TeraTerm Pro, it should significantly improve your toolbox of software utilities.

Retrieved from "http://combustory.com/wiki/index.php/Arduino_Communications_-_SQLite"
Personal tools
Sponsers
Your Ad Here
Your Ad Here