Over the past month, we’ve had the roof, windows/doors, garage floor, wiring/electrical, plumbing, and most of the siding added.
Our outdoor room is starting to shape up with the fireplace being installed. I was hesitant about the loss of interior square footage due to the outdoor room but I have to say that as it comes together, it’s becoming one of my most liked features of the house.
As the electrical and plumbing work wraps up, the next steps will be to finish the siding and putting up the drywall. Our move day is still planned for late October; we’re right on schedule.
Author: mozunk
Progress on the house
After permits were received, we’ve had great, fast progress with the house. We’ve gone from an empty lot to both levels almost entirely framed in less than a month. Foundation work started April 24th and we expect to have a roof on by June 2nd with electrical and plumbing completed by mid-June.
Foundation progress over the past month
Current state
Everything has been on-time and without any major hiccup thus far so here’s to hoping the rest of the build goes well. We’ve had the tailwind of perfect weather and that’s expected to continue for the next two weeks which means we should have much of the framing work completed without rain.
Logging router traffic and other changes
It’s been a slow couple months between the holidays, travelling, and work. I did manage to accomplish a few things with the Home Dashboard project, though. I redesigned the UI to move away from an exclusively mobile interface as the amount of data and type of data I’m including in the project now simply don’t all make sense to squeeze into a mobile UI. Sometime in early January, the system broke the 2 millionth record milestone — I’m unsure what I’ll do with some of the data I’m collecting at this point but I’ve learned a lot through collecting it and I’m sure I’ll learn more analyzing it at some point in the future.
This brings the list of events I’m collecting to:
- Indoor temperature and humidity
- Amazon Echo music events
- DirecTV program and DVR information
- Cell phone location and status details
- Local fire and police emergency events
- Home lights and other Wink hub events
- …and now home network information
Analyzing home network information
The biggest change was the addition of network event logging. After seeing that a foreign IP was accessing my LAN, I started logging each request to or from my home network until I was sure I had fixed the vulnerability. After that, I found the information interesting so I just kept logging it. For example, I was able to discover that an app on my phone was making repeated calls (~2,000 per day) to an app monitoring service (New Relic) which wasn’t doing my phone’s battery life any favors.
Collecting additional phone data
After launching Location to HTTP, I’ve been tinkering with additional data collection from my cellphone such as Bluetooth, WiFi, GPS, battery, and screen status. After collecting this information for a month or so, here are some useless data points from the most recent 30 days:
- I’ve actively used my phone for 104.5 hours (3.5hrs per day – I need to cut back on the work email…)
- Average battery level: 61%
- Average free memory: 516MB (12.6%)
- Average location accuracy: +/-31FT
- Average altitude: 154FT
- Average speed: 1MPH
- Average uptime: 153.3HRs
- Maximum uptime: 437.4HRs
I also improved the location history mapping to show different color map markers depending on the age of the record and phone details at the time the record was made:
Improved home climate visuals
I added some simple graphs of MoM temperature and humidity and also updated the heat-mappings for the daily climate information. These are a bit easier to read that those I had in the previous UI. It’s interesting to see the effectiveness of thermostat automation and our daily routines.
More detailed emergency event
Lastly, I expanded on the emergency event information to surface the top event types and the total number of events by type:
Collecting and Handling 911 Event Data
Seattle has a pretty awesome approach to data availability and transparency through data.Seattle.gov. The city has thousands of data sets available (from in-car police video records to land zoning to real-time emergency feeds) and Socrata, a Seattle-based company, has worked with the city (and many other cities) to allow developers to engage this data however they like. I spent some time playing around with some of the data sets and decided it’d be nice to know when police and fire events occurred near my apartment.
I setup a script to pull the fire and police calls for events occurring within 500 meters of my apartment and started storing them into a local database (Socrata makes it so simple – amazing work by that team). While reading it from the API, I check the proximity of the event to my address and also the type of event (burglary, suspicious person, traffic stop, etc) and trigger emails for the ones I really want to know about (such as a near by rape, burglary, shooting, vehicle theft, etc). I decided to store all events, even traffic stops, just because. I may find a use for it later – who knows…
After I’ve scrubbed through and sent any notifications for events I care about, I display the data in a simple table in my existing home dashboard and highlight red any rows for events which are within certain square area of my apartment.
To add a nice visual, I also plot the most recent events on a map using the Google Maps API. Police events are noted with blue pins, fire events are noted with red pins:
Clicking the pins will give us some details about the event:
All told, it was a pretty simple project which helped me gain some experience with the Google Maps API and also poke around with some of the data the city provides. I’m sure I’ll be doing a bit more of that in the future. These two projects have been integrated back into my home automation dashboard so I can continue to build on them in the future.
Location to HTTP – Send GPS data to remote script
I wanted to capture my phone’s location information and store it remotely (on my server) so I could do with it as I please. Most of the apps I found were severely limited in that they only mapped within the app, were riddled with ads, crashed frequently, or were untrustworthy (they’re storing my location, after all). So, I decided to build my own app that did what I wanted.
Location to HTTP
Location to HTTP is an app I’m working on which allows users to input a remote script URL to capture a device’s location information via POST.
How it works
- Create a remote script to capture the variables $lat and $lon via POST (example below).
- Download the app and enter the URL of that script.
- The app will automatically send the location information every 5-10 minutes to the script.
Possible Uses
- Track your phone’s location so you can retrace your steps if you lose it.
- Store your location history for displaying in custom interfaces.
- Share your location with friends/family when travelling or hiking.
- Trigger home automation events based on your location (turn on the heat, turn on the lights, unlock the doors, feed the pets, water the plants, etc.).
- Measure the amount of time you spend at specific locations.
Sample Script
The beauty (and utility) of this app is that you can do whatever you like with the output. In my use case, I want to store my location history to track my travel history. I do this using a simply PHP script and MySQL database.
Database Structure
CREATE TABLE IF NOT EXISTS `phone_location` ( `id` int(11) NOT NULL AUTO_INCREMENT, `lat` varchar(200) NOT NULL, `lon` varchar(200) NOT NULL, `time` varchar(200) NOT NULL, `address` varchar(200) NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=1;
PHP Script
<?
//Prevent caching and set content-type to json so app can display it correctly.
header("Cache-Control: no-store, no-cache, must-revalidate, max-age=0");
header("Cache-Control: post-check=0, pre-check=0", false);
header("Pragma: no-cache");
header('Content-Type: application/json');
//Pull address for latitude and longitude from Google API (max 25,000 calls per day for free)
function getaddress($lat,$lon) {
$url = 'http://maps.googleapis.com/maps/api/geocode/json?latlng='.trim($lat).','.trim($lon);
$json = @file_get_contents($url);
$data=json_decode($json);
$status = $data->status;
if($status=="OK"){
return $data->results[0]->formatted_address;
} else {
return false;
}
}
//Set the response we want to display in the app
date_default_timezone_set('America/Los_Angeles');
$timestamp = date('Y-m-d h:i:s');
$success_response = "Successfully captured location @ $timestamp PST!";
//Connect to MySQL database
$conn = mysqli_connect("", "", "", "");
if (!$conn) {
die("Connection failed: " . mysqli_connect_error());
}
//Capture the latitude ($lat) and longitude ($lon) passed from the app
$lat=$_REQUEST['lat'];
$lon=$_REQUEST['lon'];
//Get the address from Google API
$address= getaddress($lat,$lon);
if(!$address){
$address= "Not found";
}
//Insert record into database
$sql = "INSERT INTO phone_location (lat, lon, time, address)
VALUES ('$lat', '$lon', '$timestamp', '$address')";
if (mysqli_query($conn, $sql)) {
echo json_encode($success_response);
} else {
echo json_encode("Error: Unable to create posting.");
}
mysqli_close($conn);
?>
Google API Integration for Address Data
As shown in the example above, you can easily get address information using the Google API and reverse geocoding.
Webcam Captures Text and Stores in MySQL Database
While visiting my family recently, I saw my dad entering numbers from each of the 5-8 ticket receipts he receives daily to keep track of the work he’s done, report for payroll, etc. I knew there had to be an easier way to collect this information without having to key each ticket manually or without using a clunky, slow scanner. After a bit of research, I found an API for OCR from Haven OnDemand and I wrote a simple script to use the camera on his laptop to snap pictures of the tickets, scrape the text and position of the text from the tickets, store it all in a MySQL database, and retain the image of the tickets in a digital archive.
Demo: Snapping image via webcam and storing text
The script itself is actually very simple:
<?php
$con=mysqli_connect(localhost,<user>,<pw>,<db>);
$name = date('Y-m-d_H:i:s');
$newname="images/".$name.".jpg";
$file = file_put_contents( $newname, file_get_contents('php://input') );
if (!$file) {
print "Unable to write image to directory.";
exit();
}
else
{
$filePath = 'http://' . $_SERVER['HTTP_HOST'] . dirname($_SERVER['REQUEST_URI']) . '/' . $newname;
$result_json = file_get_contents("https://api.idolondemand.com/1/api/sync/ocrdocument/v1?apikey=<dedacted>&url=$filePath&mode=scene_photo");
$json_a=json_decode($result_json,true);
$result_left=0;
$result_top=0;
$result_widht=0;
$result_height=0;
foreach($json_a[text_block] as $p){
$result_text=htmlspecialchars($p[text]);
$result_left=$p[left];
$result_top=$p[top];
$result_width=$p[width];
$result_height=$p[height];
$sql="insert into image (name,pxleft,pxtop,pxwidth,pxheight,result) values ('$name','$result_left','$result_top','$result_width','$result_height','$result_text')";
$result=mysqli_query($con,$sql);
$value=mysqli_insert_id($con);
}
}
print "$filePath\n";
?>
While the script works well, the API isn’t great at picking up small text or parsing large amounts of text. It lacks the accuracy (only about 80% accurate) needed to confidently rely on its interpretation of the text. To compound the issues with the API, the images from the webcam are low quality – shaky hands, varying lighting, etc. drop the quality of the images I’m trying to scrape text from.
This will be a project I keep playing with to improve the accuracy and speed of snapshots and storage. Until I can figure out a way to improve the accuracy, though, this isn’t very practical.
Data Visualization and Demo
As mentioned previously, my goal wasn’t to just create a home controller/dashboard but to also collect as much data as possible while doing so. So tonight, I started playing around with a few different visualizations of the data I’ve collected thus far. It took a few hours but I’m satisfied with the current state.
I’m doing simple dumps of the most recent music played by my Amazon Echo; most recent programming watched via DirecTv; visualizing the daily average, minimum, and maximum temperature and humidity levels in my apartment; visualizing by hour of day the average, min, and max temperature for the current month vs the previous month; breaking down the amount of time I spend at home by day of week (and telling on myself that I like to leave work early on Fridays :)); and visualizing my TV watching habits by hour of day and day of week.
I recorded a video of this all and also included the DirecTv control demo at the end.
Home Automation and NFC Tags
NFC has proven to be a pretty useless technology for cell phones (unless you’re one of the people who use you Google Wallet/Apple Pay). Nevertheless, I decided to buy some tags and play with them because they’re so damn cheap (just over a dollar each, depending on the type).
One useful application of NFC tags is setting “scenes” using my existing home automation setup. By setting a tag where I usually place my phone at night, I can trigger several events all at once. When I play my phone on my nightstand, the following events are triggered:
- If it’s a weekday, set my alarm for 7:00 AM.
- If an alarm was set, the phone will adjust its volume and say “Alarm set to 7 AM”.
- Using the same text to speech, the phone will say “Goodnight, Kevin.”
- After pausing for a few seconds, it’ll POST to a simple script I wrote and turn all the lights in the apartment off before setting it’s volume to mute for the remainder of the night.
It’s a simple way of automating my night time routine and is likely the most practical use of NFC tags with home automation (and it’s not super practical, at that). If you want to recreate, here’s how I did it…
NFC Tag and App
I decided to go with the WhizTags brand because they boast more writeable space (888 bytes of writeable memory vs the standard 144 bytes). For reading and writing the tag, I went with the NFC Tools app. There’s no real reason for using this app – it just looked to be the most stable after a quick search.
Creating the Task
One of the benefits of the NFC Tools app is that you can export import json tasks. Here’s the json for the task I noted
[ {
"tasks.profile.name":"bedtime post",
"tasks.profile.date":"20161002T101631",
"tasks.profile.length":11,
"tasks.profile.size":226,
"tasks.profile.data":[ {
"tasks.profile.fields": {
"field1": "5"
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "53", "itemTask": "5", "itemDescription": "5", "itemHash": "16686621-29d1-46bf-80f0-2d8905abcfdb"
}
}
,
{
"tasks.profile.fields": {
"field1": "0"
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "819", "itemTask": "1", "itemHash": "b63e98a5-7385-4ae5-a71e-a76949092649", "itemDescription": "1 second"
}
}
,
{
"tasks.profile.fields": {
"field4": "true", "field1": "true", "field3": "true", "field8": "1", "field2": "true", "field5": "true", "field7": "false", "field6": "false"
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "92", "itemTask": "f9", "itemDescription": "MON,TUE,WED,THU,FRI\nPerform the tasks below", "itemHash": "eb2223cb-c5a2-4d59-a1a8-961e1bb13dde"
}
}
,
{
"tasks.profile.fields": {
"field3": "0", "field1": "7am", "field2": "7"
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "41", "itemTask": "7:0;7am", "itemDescription": "7am - 07:00", "itemHash": "ba40261a-84e3-437d-b1e4-b44dbc6e0b8f"
}
}
,
{
"tasks.profile.fields": {
"field1": "Alarm set for seven am."
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "85", "itemTask": "Alarm set for seven am.", "itemHash": "02fc2b51-36ba-497a-adea-1620b5799d8c", "itemDescription": "Alarm set for seven am."
}
}
,
{
"tasks.profile.fields": {
"field1": "1"
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "819", "itemTask": "2", "itemDescription": "2 seconds", "itemHash": "60c8cd67-00bb-4850-a5c2-cbe5883f3de4"
}
}
,
{
"tasks.profile.fields": {
"field1": "1"
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "90", "itemTask": "1", "itemHash": "78382993-811f-4692-b110-61ff20406731", "itemDescription": "Close your conditional block"
}
}
,
{
"tasks.profile.fields": {
"field1": "Goodnight, Kevin."
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "85", "itemTask": "Goodnight, Kevin.", "itemDescription": "Goodnight, Kevin.", "itemHash": "acab1f45-d379-4af4-99f9-0ccb63db4ae3"
}
}
,
{
"tasks.profile.fields": {
"field1": "2"
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "819", "itemTask": "3", "itemDescription": "3 seconds", "itemHash": "37320287-c034-4742-a959-705ac3399800"
}
}
,
{
"tasks.profile.fields": {
"field1": "0"
}
,
"tasks.profile.config": {
"itemUpdate": "false", "requestType": "53", "itemTask": "0", "itemHash": "d7a566bd-107b-4dae-aff4-f2ce832f06e6", "itemDescription": "0"
}
}
,
{
"tasks.profile.fields": {
"field1": "http:\/\/hellokevin.com\/nfc\/goodnight.php", "field2": "status=false;"
}
,
"tasks.profile.config": {
"itemUpdate": "true", "requestType": "110", "itemTask": "http:\/\/hellokevin.com\/nfc\/goodnight.php|status=false;", "itemHash": "e1e00d48-16fa-4942-9d9e-298b806a65c2", "itemDescription": "Request: http:\/\/hellokevin.com\/nfc\/goodnight.php\nPOST parameters :\nName: status \/ Value: false", "itemTaskExtra": null
}
}
]
}
]
The Script
The script used to turn the lights off is a modified version of this script, which I posted earlier. Instead of controlling a single device, I simply added in all the lights in my home and added in the POST var.
Expanding the Home Dashboard
In the previous post, I outlined the Home Dashboard touchscreen for controlling lights, temperature/humidity, displaying Amazon Echo information, displaying who’s home (via bluetooth sniffing), and displaying what was being watched on DirecTv. As this dashboard is intended to be a sudo remote control for my home, I thought it made sense to be able to actually control the TV with it.
After a bit of tweaking and some UI work, the end result is a super-quick (HTTP response is ~30 milliseconds which feels nearly as quick as the standard DirecTv remote) interaction between device and DirecTv receiver.
From my home dashboard/touchscreen controller, I can select the title that’s currently playing to launch the remove control (pictured above). You’ll notice that the touchscreen controller can do everything the standard remote can do, including guide browsing, DVR browsing, etc.
Home Dashboard using a Raspberry Pi
After creating a desktop home automation dashboard and, later, a live stream “digital picture frame”, I got the idea to combine the two into an always-on control panel that condenses everything I care about into a single kiosk which can sit on my end table or nightstand.
What it does
It’s essentially a condensed UI of the desktop version linked above which uses the same databases and processes.
- Current indoor temperature and humidity (via DHT11 sensor)
- If my Amazon Echo is playing music, it’ll display the artist, song, and album
- If I’m watching TV, it’ll show the title, channel, and image/movie poster
- Display unique icons for each person in the house (by sniffing for their phone’s bluetooth signal)
- It’ll show the status of my lights (on/off) and update if that status changes (using the Wink API)
- Through touch screen, allow me to control my lights in near real-time.
Materials Used
- A Pi3
- Pi display
- A case
- Indirectly, I also engage my Wink Hub, Amazon Echo, and DirecTv receiver.
How it works
Much of this (temperature, humidty, DirecTv and Wink control) is covered in “The Foundation” post. Specific to collecting information from the Amazon Echo, I use IfTTT and the Maker channel. Each time my Echo plays a song, I POST to a script similar to the one below which stores the song in a MySQL database. I can then query that, determine if the song is still playing, and publish it to the UI.
<?php
$conn = mysqli_connect(<credentials>);
if (!$conn) {
die("Connection failed: " . mysqli_connect_error());
}
$song=$_REQUEST['song'];
$artist=$_REQUEST['artist'];
$album=$_REQUEST['album'];
$timestamp=$_REQUEST['timestamp'];
$sql = "INSERT INTO echo_history (artist, song, album, timestamp)
VALUES ('$artist', '$song', '$album', '$timestamp')";
if (mysqli_query($conn, $sql)) {
echo "New record created successfully";
} else {
echo "Error: " . $sql . "<br>" . mysqli_error($conn);
}
mysqli_close($conn);
?>
My Maker recipe looks something like this: IF then to URL