Added Away/Home to unofficial-nest-api

I just finished adding a new simple feature to control the away status for a structure to my unofficial-nest-api published on GitHub and available as a node package (npm).

Usage is simple as calling setAway or setHome on the nest instance after authentication and a successful status has been returned (see commented calls below).

if (username && password) {
    username = trimQuotes(username);
    password = trimQuotes(password);
    nest.login(username, password, function (data) {
        if (!data) {
            console.log('Login failed.');
            process.exit(1);
            return;
        }
        console.log('Logged in.');
        nest.fetchStatus(function (data) {
            for (var deviceId in data.device) {
                if (data.device.hasOwnProperty(deviceId)) {
                    var device = data.shared[deviceId];
                    console.log(util.format("%s [%s], Current temperature = %d F target=%d",
                        device.name, deviceId,
                        nest.ctof(device.current_temperature),
                        nest.ctof(device.target_temperature)));
                }
            }
            subscribe();
            //nest.setAway();
            //nest.setHome();
        });
    });
}

Nest Thermostat API using Node JS and Nest API Update

I’ve been asked by a few people for more details on the API Nest Labs uses for their thermostats, especially regarding setting data (and not just polling).

The API uses mostly JSON formatted data POSTed to their web servers.

Authentication

To authenticate, POST the username and password, encoded as form url-encoded:

POST https://home.nest.com/user/login HTTP/1.1
Host: home.nest.com
Proxy-Connection: keep-alive
Accept-Encoding: gzip, deflate
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Accept-Language: en-us
Content-Length: {Length}
Accept: */*
Connection: keep-alive
User-Agent: Nest/3.0.1.15 (iOS) os=6.0 platform=iPad3,1

username={email}&password={password}

Adjust the email and password, and the content length to fit. You may need to remove the Accept-Encoding header value if your client cannot accept gzip or deflated responses.

The server responds with a healthy set of basic information (in JSON format):

{
    "is_superuser": false,
    "is_staff": false,
    "urls": {
        "transport_url": "https://{subdomain}.transport.nest.com:443",
        "rubyapi_url": "https://home.nest.com/",
        "weather_url": "http://www.wunderground.com/auto/nestlabs/geo/current/i?query=",
        "support_url": "https://nest.secure.force.com/support/webapp?"
    },
    "limits": {
        "thermostats_per_structure": 10,
        "structures": 2,
        "thermostats": 10
    },
    "access_token": "GIANT TOKEN STRING==",
    "userid": "1234",
    "expires_in": "Wed, 07-Oct-2012 12:08:00 GMT",
    "email": "user@example.com",
    "user": "user.1234"
}

There are a few things you’ll need from the response:

  • transport_url : this is the address for all of the later request that are made. I’d speculate it’s just a specific server in a server farm (likely with server affinity/session)
  • access_token : this is the key for all later requests and grants access to the API
  • userid/user : a unique user ID
  • expires_in : this is the timestamp for when the access token expires

I’m not sure why the “limits” are being sent back to the client.

You can obtain the service URLs at any time:

POST https://home.nest.com/user/service_urls HTTP/1.1
Host: home.nest.com
Authorization: Basic GIANT TOKEN STRING==
Accept-Encoding: gzip, deflate
Accept: */*
Content-Length: 0
Accept-Language: en-us
Connection: keep-alive
Proxy-Connection: keep-alive
User-Agent: Nest/3.0.1.15 (iOS) os=6.0 platform=iPad3,1

Just insert an Authorization header with the access_token value.

The response:

{ "urls": { "transport_url": "https://{subdomain}.transport.nest.com:443", "rubyapi_url": "https://home.nest.com/", "weather_url": "http://www.wunderground.com/auto/nestlabs/geo/current/i?query=", "support_url": "https://nest.secure.force.com/support/webapp?" }, "limits": { "thermostats_per_structure": 10, "structures": 2, "thermostats": 10 } }

Nest labs has a special URL at wunderground.com to access the weather.

One of the first requests you might want to send is to get a complete picture of the system:

GET https://{subdomain}.transport.nest.com:443/v2/mobile/#USER.ID# HTTP/1.1
Host: {subdomain}.transport.nest.com:443
Authorization: Basic GIANT TOKEN STRING==
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-us
Connection: keep-alive
X-nl-protocol-version: 1
X-nl-user-id: #USERID#
Proxy-Connection: keep-alive
User-Agent: Nest/3.0.1.15 (iOS) os=6.0 platform=iPad3,1

You’ll make the request to the transport_url and make sure that the Host, Authorization, and X-nl-user-id header values are set appropriately. The Url now must include though:

  • version
  • mobile
  • and the full user id (like user.1234)

So, it will look something like: /v2/mobile/user.1234

It will respond with the mother-load of all JSON payloads. I’ve trimmed the response for my house as I have three thermostats. But the pattern repeats exactly, so it’s easy to extrapolate how the pattern works:

{ "metadata": { "SERIALNUM1": { "$version": -1262653277, "$timestamp": 1349697004000, "last_connection": 1349697004683, "last_ip": "LAST.IP.ADDRESS" }, "SERIALNUM2": { "$version": -1868790132, "$timestamp": 1349696678000, "last_connection": 1349696678701, "last_ip": "LAST.IP.ADDRESS" }, "SERIALNUM3": { "$version": -1581663504, "$timestamp": 1349696680000, "last_connection": 1349696680647, "last_ip": "LAST.IP.ADDRESS" } }, "track": { "SERIALNUM2": { "$version": 1065037529, "$timestamp": 1349696725390, "online": true, "last_connection": 1349696725390, "last_ip": "LAST.IP.ADDRESS" }, "SERIALNUM3": { "$version": 981680556, "$timestamp": 1349696726266, "online": true, "last_connection": 1349696726266, "last_ip": "LAST.IP.ADDRESS" }, "SERIALNUM1": { "$version": 1421919505, "$timestamp": 1349697004728, "online": true, "last_connection": 1349697004728, "last_ip": "LAST.IP.ADDRESS" } }, "user_settings": { "#USERID#": { "$version": 370836640, "$timestamp": 1337481029003, "email_verified": true, "tos_accepted_version": 1319500800000, "receive_marketing_emails": true, "receive_nest_emails": true, "receive_support_emails": true, "max_structures": 2, "max_thermostats": 10, "max_thermostats_per_structure": 10, "tos_minimum_version": 1319500800000, "tos_current_version": 1319500800000, "lang": "en_US" } }, "structure": { "#STRUCTURE-UUID#": { "$version": 1797929878, "$timestamp": 1349689810000, "location": "Mount Horeb, WI", "renovation_date": "2000", "country_code": "US", "away_timestamp": 1349302501, "away": false, "house_type": "family", "name": "Home", "postal_code": "#POSTALCODE#", "creation_time": 1324159145719, "num_thermostats": "3", "devices": ["device.SERIALNUM3", "device.SERIALNUM1", "device.SERIALNUM2"], "user": "user.#USERID#", "away_setter": 1 } }, "link": { "SERIALNUM3": { "$version": 2122853931, "$timestamp": 1327246591000, "structure": "structure.#STRUCTURE-UUID#" }, "SERIALNUM2": { "$version": -1703839727, "$timestamp": 1324159215000, "structure": "structure.#STRUCTURE-UUID#" }, "SERIALNUM1": { "$version": -459415854, "$timestamp": 1325967612000, "structure": "structure.#STRUCTURE-UUID#" } }, "device": { "SERIALNUM1": { "$version": -81037153, "$timestamp": 1349696605000, "heatpump_setback_active": false, "emer_heat_enable": false, "local_ip": "LOCAL.IP.ADDRESS", "switch_system_off": false, "away_temperature_high": 27.778, "temperature_lock_high_temp": 22.222, "cooling_source": "electric", "leaf_threshold_cool": 0.0, "fan_cooling_state": false, "note_codes": [], "heater_source": "gas", "compressor_lockout_leaf": -17.8, "has_x3_heat": false, "target_humidity_enabled": false, "heat_x3_source": "gas", "alt_heat_delivery": "forced-air", "fan_mode": "auto", "has_x2_heat": false, "rssi": 67.0, "emer_heat_delivery": "forced-air", "heatpump_savings": "off", "pin_y2_description": "none", "filter_reminder_enabled": false, "capability_level": 3.0, "schedule_learning_reset": false, "has_x2_cool": false, "hvac_pins": "W1,Y1,C,Rc,G", "ob_orientation": "O", "range_enable": true, "auto_away_enable": true, "dual_fuel_breakpoint_override": "none", "lower_safety_temp_enabled": true, "has_fan": true, "dehumidifier_state": false, "range_mode": false, "nlclient_state": "", "emer_heat_source": "electric", "heatpump_ready": false, "available_locales": "en_US,fr_CA,es_US", "current_version": "3.0.1", "learning_state": "slow", "pin_ob_description": "none", "pin_rh_description": "none", "has_alt_heat": false, "pin_y1_description": "cool", "humidifier_state": false, "backplate_serial_number": "#BACKPLATE-SERIALNUMBER1#", "has_x2_alt_heat": false, "heat_x3_delivery": "forced-air", "leaf_threshold_heat": 19.336, "has_emer_heat": false, "learning_mode": true, "leaf_learning": "ready", "has_aux_heat": false, "aux_heat_source": "electric", "backplate_bsl_info": "BSL", "alt_heat_x2_source": "gas", "pin_c_description": "power", "humidifier_type": "unknown", "pin_w2aux_description": "none", "country_code": "US", "target_humidity": 35.0, "heat_x2_delivery": "forced-air", "lower_safety_temp": 4.444, "cooling_x2_source": "electric", "equipment_type": "gas", "heat_pump_aux_threshold": 10.0, "alt_heat_x2_delivery": "forced-air", "heat_pump_comp_threshold": -31.5, "learning_days_completed_cool": 116, "backplate_bsl_version": "1.1", "current_schedule_mode": "HEAT", "hvac_wires": "Heat,Cool,Fan,Common Wire,Rc", "leaf": false, "type": "TBD", "pin_g_description": "fan", "switch_preconditioning_control": false, "click_sound": "on", "aux_heat_delivery": "forced-air", "away_temperature_low_enabled": true, "heat_pump_comp_threshold_enabled": false, "preconditioning_ready": true, "has_dehumidifier": false, "fan_cooling_enabled": true, "leaf_away_high": 28.88, "fan_cooling_readiness": "ready", "device_locale": "en_US", "temperature_scale": "F", "error_code": "", "preconditioning_active": false, "battery_level": 3.93, "away_temperature_high_enabled": true, "learning_days_completed_heat": 149, "pin_star_description": "none", "upper_safety_temp_enabled": false, "preconditioning_enabled": true, "current_humidity": 45, "dual_fuel_breakpoint": -1.0, "postal_code": "#POSTALCODE#", "backplate_mono_version": "4.0.5", "alt_heat_source": "gas", "aux_lockout_leaf": 10.0, "has_heat_pump": false, "heater_delivery": "forced-air", "auto_away_reset": false, "away_temperature_low": 14.444, "radiant_control_enabled": false, "temperature_lock": false, "upper_safety_temp": 35.0, "time_to_target_training": "ready", "dehumidifier_type": "unknown", "target_time_confidence": 1.0, "temperature_lock_low_temp": 20.0, "pin_w1_description": "heat", "forced_air": true, "temperature_lock_pin_hash": "", "leaf_type": 1, "backplate_mono_info": "TFE (BP_DVT) 4.0.5 (root@bamboo) 2012-09-18 18:18:23", "has_dual_fuel": false, "learning_time": 2113, "creation_time": 1325966794212, "has_humidifier": false, "learning_days_completed_range": 0, "leaf_schedule_delta": 1.11, "user_brightness": "auto", "leaf_away_low": 13.92, "pin_rc_description": "power", "serial_number": "SERIALNUM1", "mac_address": "18b43004f391", "heat_x2_source": "gas", "time_to_target": 0, "backplate_model": "Backplate-1.9", "model_version": "Diamond-1.10", "heat_pump_aux_threshold_enabled": true }, "SERIALNUM3": { "$version": 2134103145, "$timestamp": 1349695665000, /* same as previous */ "backplate_model": "Backplate-1.9", "model_version": "Diamond-1.10", "heat_pump_aux_threshold_enabled": true }, "SERIALNUM2": { "$version": -1340728480, "$timestamp": 1349692217000, /* same as previous */ "backplate_model": "Backplate-1.9", "model_version": "Diamond-1.10", "heat_pump_aux_threshold_enabled": true } }, "schedule": { "SERIALNUM3": { "$version": -1130522241, "$timestamp": 1349692663000, "days": { "3": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "2": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "1": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "0": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "6": { "3": { "time": 67500, "entry_type": "setpoint", "temp": 18.333, "type": "HEAT" }, "2": { "time": 28800, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 24300, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 75600, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "5": { "3": { "time": 67500, "entry_type": "setpoint", "temp": 18.333, "type": "HEAT" }, "2": { "time": 28800, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 24300, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 75600, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } }, "4": { "3": { "time": 74700, "entry_type": "setpoint", "temp": 20.0, "type": "HEAT" }, "2": { "time": 23400, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" }, "1": { "time": 19800, "entry_type": "setpoint", "temp": 17.222, "type": "HEAT" }, "0": { "touched_by": 1, "time": 0, "touched_tzo": -18000, "entry_type": "continuation", "temp": 14.444, "type": "HEAT", "touched_at": 1349285499 }, "4": { "time": 78300, "entry_type": "setpoint", "temp": 14.444, "type": "HEAT" } } }, "schedule_mode": "HEAT", "name": "Basement Current Schedule", "ver": 2 }, "SERIALNUM2": { "$version": -462155699, "$timestamp": 1349674697000, "days": { /* SAME AS ABOVE */ } }, "schedule_mode": "HEAT", "name": "Second Floor Current Schedule", "ver": 2 }, "SERIALNUM1": { "$version": 2014520777, "$timestamp": 1349695806000, "days": { /* SAME AS ABOVE */ } }, "schedule_mode": "HEAT", "name": "First Floor Current Schedule", "ver": 2 } }, "shared": { "SERIALNUM3": { "$version": -493517056, "$timestamp": 1349696367000, "auto_away": 0, "auto_away_learning": "training", "hvac_heat_x3_state": false, "hvac_alt_heat_state": false, "compressor_lockout_enabled": false, "target_temperature_type": "heat", "hvac_heater_state": false, "hvac_emer_heat_state": false, "can_heat": true, "compressor_lockout_timeout": 0, "hvac_cool_x2_state": false, "target_temperature_high": 24.0, "hvac_aux_heater_state": false, "hvac_heat_x2_state": false, "target_temperature_low": 20.0, "target_temperature": 14.444, "hvac_ac_state": false, "hvac_fan_state": false, "target_change_pending": false, "name": "Basement", "current_temperature": 18.11, "hvac_alt_heat_x2_state": false, "can_cool": true }, "SERIALNUM1": { "$version": -1432433268, "$timestamp": 1349696363000 /* SAME AS ABOVE */ }, "SERIALNUM2": { "$version": 2060664119, "$timestamp": 1349696709000 /* SAME AS ABOVE */ } }, "user_alert_dialog": { "#USERID#": { "$version": -1852987123, "$timestamp": 1327246591000, "dialog_data": "", "dialog_id": "confirm-pairing" } }, "user": { "#USERID#": { "$version": 209478897, "$timestamp": 1324159145000, "name": "EMAILADDRESS", "structures": ["structure.#STRUCTURE-UUID#"] } } }

Setting a Temperature

Changing a thermostat’s current set point is easy.

You’ll need the Serial Number (shown as SERIALNUM1 in the JSON above) of the thermostat.

POST https://275be85a.transport.nest.com:443/v2/put/shared.01AA01AB4611009P HTTP/1.1
Host: 275be85a.transport.nest.com:443
Accept-Language: en-us
User-Agent: Nest/3.0.1.15 (iOS) os=6.0 platform=iPad3,1
X-nl-base-version: 2060664119
Accept: */*
Content-Type: application/json
X-nl-protocol-version: 1
X-nl-user-id: 7236
X-nl-session-id: ios-7236-371385438.528577
Connection: keep-alive
X-nl-merge-payload: true
Authorization: Basic GIANT TOKEN STRING==
Content-Length: 60
Proxy-Connection: keep-alive
Accept-Encoding: gzip, deflate

{"target_change_pending":true,"target_temperature":16.11111}

Set the temperature in Celsius.

There’s also a polling subscription that happens. It’s extremely chatty and from the amount of polling it does, you’d think that the UI was doing live graphing of micro-temperature changes.

Essentially, the polling sends a series of keys, with timestamps, representing the various types of data being requested.

It looks something like this:

{ "keys": [{ "key": "user.#USERID#", "version": 209478897, "timestamp": 1324159145000 }, { "key": "user_settings.#USERID#", "version": 370836640, "timestamp": 1337481029003 }, { "key": "user_alert_dialog.#USERID#", "version": -1852987123, "timestamp": 1327246591000 }, { 

It repeats for sections such as “shared”, “message”, “device”, “track”, and more. For my purposes, shared is the winner as it contains the current temperature.

 

Node API

I decided to write a new demonstration application and polish it up a bit, this time using Node.

I’m not going to document the API that I created here (not right now), but here’s a sample of how it can be used:

var username = process.argv[2];
var password = process.argv[3];

if (username && password) {
    username = trimQuotes(username);
    password = trimQuotes(password);
    nest.login(username, password, function (data) {
        if (!data) {
            console.log('Login failed.');
            process.exit(1);
            return;
        }
        console.log('Logged in.');
        nest.fetchStatus(function (data) {
            for (var deviceId in data.device) {
                if (data.device.hasOwnProperty(deviceId)) {
                    var device = data.shared[deviceId];

                    console.log(util.format("%s [%s], Current temperature = %d F target=%d",
                        device.name, deviceId,
                        nest.ctof(device.current_temperature),
                        nest.ctof(device.target_temperature)));
                }
            }
            subscribe();
        });
    });
}

function subscribe() {
    nest.subscribe(subscribeDone);
}

function subscribeDone(deviceId, data) {
    if (deviceId) {
        console.log('Device: ' + deviceId)
        console.log(JSON.stringify(data));
    }
    setTimeout(subscribe, 2000);
}

The example code runs forever. Smile

I’ve included two methods in the API, “get” and “post” which make it simple to call additional web services that I haven’t yet provided.

Find the code here: https://github.com/wiredprairie/unofficial_nodejs_nest.

Update: there’s a npm as well now (Dec 20, 2012)

.NET API for Nest Thermostat

I just finished a preliminary read-only (think version 0.1) wrapper around the Nest Thermostat API that is used by their mobile phone and web applications. As Nest doesn’t have a formal API yet, the code could break at any time and may not be suitable for any use. However, it is working today. Smile

The project is hosted on GitHub. It uses JSON.NET for parsing the return values from the Nest servers.

There are three projects, with the lib containing the assembly that is used by the two test applications. One is a console app and the other a simple WPF application:

SNAGHTML88bff0b3

(My thermostats are named Zero, One, and Two).

If there’s interest, I plan on adding some methods to the library which allow modification of data (such as the current temperature), and ideally, support for live updates from the devices if I can make sense of the data that is returned.

FYI: I’ve now written a Node version of the API, detailed here.

Nest Thermostat API/Protocol

While Nest Labs hasn’t released a formal (documented & supported) API, I thought I’d do a bit of digging to see how they’re using the network and what might be achievable.

A few things are going on, the majority as you’d probably expect.

The web interface is using a long polling technique apparently to watch for updates to the schedule, temperature, etc.

image

I haven’t determined what the frequency is though, or the wait time. It’s very inconsistent, even when I wouldn’t expect much new “live” data to be available on the network, it frequently updates and polls again.

There are a few constants set in the HOME page script:

C.ABSENT_USER_THRESHOLD     = +('300') || 0;  // seconds
C.DEAD_DEVICE_THRESHOLD     = +('300') || 0;  // seconds
C.pollingInterval           = +('2500') || 0;       // ms
C.WEATHER_POLLING_INTERVAL  = +('120000') || 0; // ms

 

If the C.pollingInterval value were for the subscribe endpoint mentioned above, I’d see a LOT more calls than I do – so I’m still not clear how the polling interval is decided.

The API calls, for the most part are using JSONP syntax over an HTTPS connection.

The most frequent request is to “subscribe.” It sends as part of the GET request a large block of encoded JSON (using encodeURIComponent and then JSON.stringify).

I’m not familiar with the key/value system that they’re using (it may just be something they’ve constructed in-house – although given the number of open source JavaScript libraries they’re using, I thought someone might recognize it):

key”, “{actualkey}.{value}”

I don’t understand why they’ve redundantly specified “key” in a list of keys when it’s evident that the actual key is contained within the value as a delimited string. It’s more data to send and more data to parse this way. So, again, maybe it’s based on some DB or model system I’m not familiar with. (Anyone recognize it?)

I’ve substituted the actual values (as they are serial numbers of my devices) with text representations of what the value represented below:

{"keys":
    [{"key":"user.#USERID#",
        "version":209478897,"timestamp":1324159145000},
    {"key":"user_alert_dialog.#USERID#",
        "version":-1320296685,"timestamp":1325967612000},
    {"key":"structure.#STRUCTURE-GUID#",
        "version":656192675,"timestamp":1325967612000},
    {"key":"device.#DEVICE 1 SERIAL NUMBER#",
        "version":1485027516,"timestamp":1326034984000},
    {"key":"shared.#DEVICE 1 SERIAL NUMBER#",
        "version":588844038,"timestamp":1326034818000},
    {"key":"schedule.#DEVICE 1 SERIAL NUMBER#",
        "version":1187107985,"timestamp":1326005677000},
    {"key":"track.#DEVICE 1 SERIAL NUMBER#",
        "timestamp":1326035650601,"version":1041047847},
    {"key":"device.#DEVICE 2 SERIAL NUMBER#",
        "version":149169270,"timestamp":1326034820000},
    {"key":"shared.#DEVICE 2 SERIAL NUMBER#",
        "version":659841570,"timestamp":1326034820000},
    {"key":"schedule.#DEVICE 2 SERIAL NUMBER#",
        "version":-2016290692,"timestamp":1326005625000},
    {"key":"track.#DEVICE 2 SERIAL NUMBER#",
        "timestamp":1326035650862,"version":528978433},
    {"key":"device.#DEVICE 3 SERIAL NUMBER#",
        "version":1637112547,"timestamp":1326035399000},
    {"key":"shared.#DEVICE 3 SERIAL NUMBER#",
        "version":760504326,"timestamp":1326035397000},
    {"key":"schedule.#DEVICE 3 SERIAL NUMBER#",
        "version":-314552357,"timestamp":1326003402000},
    {"key":"track.#DEVICE 3 SERIAL NUMBER#",
        "version":-645931164,"timestamp":1326035531802}]}"

We’ve got three thermostats, so there are always three sets of subscription requests for each call to subscribe.

Using my iPad, I adjusted the set point for our second story (#DEVICE 2#) down one degree Fahrenheit (to 67°).

Within approximately a second, the most recent pending subscribe request returned with a far more interesting payload:

jQuery17108417355176061392_1326035646750(
    { "status": 200,
        "headers": {
            "X-nl-skv-key": "shared.#DEVICE 2 SERIAL NUMBER#",
            "X-nl-skv-version": 869022424,
            "X-nl-skv-timestamp": 1326038279000,
            "X-nl-service-timestamp": 1326038279825
        },
        "payload": {
            "current_temperature": 19.98,
            "hvac_fan_state": false,
            "name": "TWO", "hvac_heat_x2_state": false,
            "hvac_ac_state": false,
            "can_cool": true,
            "auto_away": 0,
            "compressor_lockout_enabled": false,
            "target_temperature_low": 16.66667,
            "target_temperature_high": 26.66667,
            "compressor_lockout_timeout": 0,
            "hvac_heater_state": false,
            "hvac_aux_heater_state": false,
            "target_temperature": 19.44444,
            "can_heat": true,
            "target_temperature_type": "heat",
            "target_change_pending": true
        }
    });

Everything above is needed to update the current state of the UI. As you can see, the current temperature (returned as Celsius apparently) is 19.98 (67.964°F). The current temperature as displayed on the thermostat and the web UI was 68.

Seeing these return values makes me think that they may be using Ruby and Rails (as the naming convention tends to follow Rails naming using underscores between words). I know for example, I wouldn’t name variables/columns that way when building a C#/JavaScript MVC project.

Rather than just a delta payload of what’s changed, they’ve currently opted for a full update of all information related to the thermostat state.

Several seconds later, a much larger payload was returned to a subscribe request:

"status": 200,
"headers": {
    "X-nl-skv-key": "device.#DEVICE 2 SERIAL NUMBER#",
    "X-nl-skv-version": -2086438581,
    "X-nl-skv-timestamp": 1326038378000,
    "X-nl-service-timestamp": 1326038379023
},
"payload": {
    "ob_orientation": "O",
    "upper_safety_temp": 1000.0,
    "forced_air": true,
    "creation_time": 1324142042019,
    "switch_preconditioning_control": false,
    "click_sound": "on",
    "leaf": false, "user_brightness": "auto",
    "learning_state": "steady",
    "heat_pump_comp_threshold": -1000.0,
    "local_ip": "10.0.0.205",
    "backplate_serial_number": "#SHOULD BE DEVICE 2 SERIAL NUMBER, BUT ISN'T?#",
    "capability_level": 1.03,
    "postal_code": "#POSTALCODE#",
    "upper_safety_temp_enabled": false,
    "heat_pump_aux_threshold": 10.0,
    "lower_safety_temp_enabled": true,
    "serial_number": "#DEVICE 2 SERIAL NUMBER#",
    "temperature_lock": false,
    "learning_time": 1002,
    "current_version": "1.0.4",
    "model_version": "Diamond-1.10",
    "backplate_bsl_info": "BSL",
    "auto_away_enable": true,
    "heat_pump_comp_threshold_enabled": false,
    "fan_mode": "auto",
    "range_enable": false,
    "temperature_scale": "F",
    "backplate_mono_info": "TFE (BP_DVT) 3.5.2 (ehs@ubuntu) 2011-11-05 12:00:00",
    "backplate_bsl_version": "1.1",
    "equipment_type": "gas",
    "range_mode": false,
    "lower_safety_temp": 7.0,
    "has_fan": true,
    "hvac_wires": "Heat,Cool,Fan,Common Wire,Rc",
    "learning_mode": true,
    "away_temperature_high": 32.0,
    "switch_system_off": false,
    "time_to_target": 1326039444,
    "away_temperature_low": 14.444444444444445,
    "current_humidity": 45,
    "mac_address": "#MACADDR#",
    "backplate_mono_version": "3.5.2",
    "has_aux_heat": false,
    "type": "TBD",
    "hvac_pins": "W1,Y1,C,Rc,G",
    "has_heat_pump": false,
    "heat_pump_aux_threshold_enabled": true,
    "battery_level": 3.945,
    "target_time_confidence": 1.0
}

 

A few things to note:

  • Upper_safety_temperature is just a bit beyond my comfort zone at 1832°F. I don’t know why it’s sending a value like that to the client, and why it’s stupidly high.
  • The backplate serial number doesn’t match with the thermostat according to the payload response. I don’t know why this might be as I confirmed that the numbers matched through visual inspection of the device just now.
  • The majority of these details are exposed in one way or another in the details area of the web UI.
  • Time to target (payload.time_to_target) is unusual in that it’s a JavaScript Date value, divided by 1000. So, in the example above, the time to target is: new Date(1326039444 * 1000).toString() = >"Sun Jan 08 2012 10:17:24 GMT-0600 (Central Standard Time)"

    Next, a payload is returned with the new status:

    "status": 200,
    "headers": {
        "X-nl-skv-key": "shared.#DEVICE 2 SERIAL NUMBER#",
        "X-nl-skv-version": 1689916148,
        "X-nl-skv-timestamp": 1326038378000,
        "X-nl-service-timestamp": 1326038379151
    },
    "payload": {
        "hvac_fan_state": false,
        "name": "TWO",
        "hvac_heat_x2_state": false,
        "hvac_ac_state": false,
        "can_cool": true,
        "auto_away": 0,
        "compressor_lockout_enabled": false,
        "target_temperature_low": 16.66667,
        "current_temperature": 19.53,
        "target_temperature_high": 26.66667,
        "compressor_lockout_timeout": 0,
        "target_change_pending": false,
        "hvac_aux_heater_state": false,
        "target_temperature": 20.55556,
        "can_heat": true,
        "target_temperature_type": "heat",
        "hvac_heater_state": true
    }

     

    Here, the hvac_heater_state is set to true. The furnace is on.

    A little while later, that value is set to false.

    Occasionally, the payload includes the complete schedule for the thermostat. I’m not going to reproduce the entire payload here as it’s too large, and quite boring. Here’s a snippet of what it returns:

    "schedule": {
        "#DEVICE 2 SERIAL NUMBER#": {
            "$version": 1187107985,
            "$timestamp": 1326005677000,
            "name": "One Current Schedule",
            "days": {
                "0": {
                    "0": {
                        "type": "HEAT",
                        "temp": 14.445,
                        "time": 0,
                        "entry_type": "continuation"
                    },
                    "1": {
                        "type": "HEAT",
                        "temp": 14.445,
                        "time": 27900,
                        "entry_type": "setpoint"
                    },
                    "2": {
                        "type": "HEAT",
                        "temp": 20.556,
                        "time": 63000,
                        "entry_type": "setpoint"
                    },
                    "3": {
                        "type": "HEAT",
                        "temp": 14.445,
                        "time": 70200,
                        "entry_type": "setpoint"
                    }
                },
                "1": {
                    "0": {
                        "type": "HEAT",
                        "temp": 14.445,
                        "time": 0,
                        "entry_type": "continuation"
                    },
                    "1": {
                        "type": "HEAT",
                        "temp": 18.889,
                        "time": 20700,
                        "entry_type": "setpoint"
                    },

     

    It’s a basic table structure. The first set point of the day is at 0, and is a “continuation.” These don’t show up in the UI.

    Here’s what the day 1 looks like on the Nest thermostat UI:

    image

    When changing a temperature setpoint, I’m a bit disappointed to see that the entire schedule is sent with every request apparently. I just wouldn’t have expected that given that the more setpoints that there are, the bigger the payload must be. The UI is often sluggish when rapidly making adjustments in the schedule, and this could be one of the factors.

    In the example below (which I’ve snipped most of the payload sent again as a JSONP request), I’ve set the first set point to 57F.

        "payload": {
            "days": {
                "0": {
                    "0": {
                        "type": "HEAT",
                        "temp": 14.685,
                        "time": 0,
                        "entry_type": "continuation"
                    },
                    "1": {
                        "type": "HEAT",
                        "temp": 15.000444444444444,
                        "time": 24300,
                        "entry_type": "setpoint"
                    },

    For the JSONP requests sent as “MAKE CHANGE” (easily could have been PUT), they each contained the following attributes as shown below. All JSONP requests are apparently routed on the web server using “headers” rather than a RESTful URL based system:

        },
        "headers": {
            "X-nl-client-timestamp": 1326041210566,
            "X-nl-session-id": "#SESSION ID#",
            "X-nl-protocol-version": 1,
            "Authorization": "Basic #BASIC AUTH#"
        },
        "path": "/v1/put/schedule.#DEVICE 2 SERIAL NUMBER#",
        "redir": "https://home.nest.com/post_jsonp",
        "jsonp": "4_"
    }

    It’s RESTful in spirit as there is a route (“path”), but it’s managed by some internal routing engine. (Now, I think that they’re not using Ruby and Rails).

    For something simple, like changing the current temperature of a thermostat, the request is thankfully simple:

    {
        "payload": {
            "shared": {
                "#DEVICE 2 SERIAL NUMBER#": {
                    "target_temperature": 18.333333333333336
                }
            }
        },
        "headers": {
            "X-nl-client-timestamp": 1326041744556,
            "X-nl-session-id": "#SESSION ID#",
            "X-nl-protocol-version": 1,
            "Authorization": "Basic #BASIC AUTH#"
        },
        "path": "/v1/put",
        "redir": "https://home.nest.com/post_jsonp",
        "jsonp": "14_"
    }

    While, I haven’t taken the time to try to write a custom UI for this undocumented API yet, it looks like it should be relatively easy to do, especially as it relates to the schedule and current temperature settings. I know there’s been some Siri proxy stuff that’s been written – but I don’t have any interest in trying to get that to work.

    As with most APIs like this, the trick is often getting authorization correct. For Nest, it appears that making a POST request to https://home.nest.com/accounts/login/ with username and password as form data, that the server responds with 2 cookies:

    1. sessionid == used in X-nl-session-id in headers
    2. cztoken == used as the Authorization in headers (prepended with the text “Basic “

    FYI: I also have a Node version of the API that is more up to date than this.