Networked Objects

PHP and web scraping

use language that makes sense to “outsiders”
- what the hell is an Arduino, Xbee, XPort, even potentiometer

what’d we learn? all we’re doing is sending serial data.

HTTP Requests and web scrape…

PHP

let’s do some PHP:
http://www.php.net

see a function, try it out, save it

lots of browse and borrow
good amount of

files on server, when you make a request to a php file it runs the script and sends the results.

PHP and Perl can swap because they assume everything comes out as ASCII in the end.
variables start with $

< ?php
for ($myVar = 0; $myVar < 10; $myVar++) {
echo “Hello”.$myVar.“
“;
}
?>

save to your server. then use telnet and an HTTP get request from terminal:

telnet IP.ADDRESS
GET /address/goes/here.php HTTP/1.1
HOST: itp.nyu.edu



From here we have the option to get or receive.

Age checker: age_checker.php in Making Things Talk

use http://makingthingstalk.com

making a date variable:
$date = date(”Ym-d h:i:s\t”);

You can use Coda and a local web server

< ?php
$date = date(”Y-m-d h:i:s\t”);
echo “”;
echo “Today’s date is $date”;
echo “”;
?>

regular expressions == using backslash if you want to have it forget the next one (echo “Today’s date is $date and I have \$45.”)

Web scraping:

http://sensorbase.org/

can save files…?

all available here: http://www.tigoe.net/pcomp/code at http://www.tigoe.net/pcomp/code/category/code/php/146

make a project, give it a name, give it a location, tags
then make tables and fields (with data type)
address, sensor reading, timestamp (be sure to put in timestamps: date and time) : sequential ID is automatically
upload some field info as CSV or XML (available in about section or manually)

get sensorbase_xml_slog.php file from tigoe’s website

quick concantenate:
$url .= $page
is the same as
$url = $url.$page

use ? to add parameters in a url (http://www.whatever.com/the_file.php?field=info&another_field=more_info&more=even_more)

using curl library: http://us3.php.net/curl

urlencode used to clean up url variable strings

when writing to microcontroller uses
echo “\0”

On the Arduino side:

gets the information from the sensor
saves it to variables
makes an HTTP GET request by printing serial data :

Serial.print(“GET /~myaccount/sensorbase_xml_slog.php?“);
Serial.print(“radio_ID=“);
Serial.print(radioID, DEC);
Serial.print(“&voltage=“);
Serial.print(voltage, DEC);
Serial.print(“ HTTP/1.0\n“);
// Fill in your server's name:
Serial.print(“HOST:itp.nyu.edu\n\n“);
// update the state of the program:
status = requesting;


Example for scraping:

http://www.Airnow.org
look at source code and find the line needed, strips tags, and store info
spits back to Arduino

< source code coming soon from Tom >

A little Wiring.C library action:
atoi(); function makes variable ASCII to integer