Friday, October 17, 2014

Operating your Hue remotely using Hue Remote API – a working Proof of Concept

Introduction

To operate your Hue system on local network you can send REST based messages like described in the Philips hue API to your Hue bridge. However if you want to do this in your custom application over the internet you have a problem. No API, no official documentation about how to sent messages through the Hue Portal that forwards it to your Hue bridge.

This blog post will show how you can send messages to your Hue bridge over the internet (without using port forwards). In this proof of concept I reuse the access token of IFTTT. The code samples are in C#, however you could make the same REST calls using the language of your choice.

BTW: I assume you have an account on the Hue portal and that you registered your Hue bridge.

What others write about this topic

Just a few people have some information about this like:
http://blog.paulshi.me/technical/2013/11/27/Philips-Hue-Remote-API-Explained.html
Hacking Lightbulbs: Security Evaluation of the Philips Hue Personal Wireless Lighting System by Nitesh Dhanjani
http://stackoverflow.com/questions/19900657/how-to-connect-backend-service-with-philips-hue-bridge-remotely

I investigated some of the findings from above resources. The parts about the getting the access token is not working, it’s probably out of date because of changes to the Hue Portal. So if you don’t have an access token you cannot send commands to the Hue Portal.

Local REST Messages

Sending a REST based message to your local bridge is done by sending a http request in the following format.

URL http://<your bridge ip>/api/<username>/<some resource>
Method GET/PUT/POST/DELETE
Body JSON formatted data

To turn on light 1 you will send a message like:

URL /api/<username>/lights/1/state
Method PUT
Body {
“on”: true
}

Remote REST Message

Now it gets interesting. Sending a message to your bridge through the Hue Portal is a bit different. You should always use POST and send the request to the Hue portal. It should also include a valid access token. The body contains a clipmessage which contains a clipCommand that encapsulates the JSON command you normally sent to your bridge (in local network mode).

The format is as follows:

URL https://www.meethue.com/api/sendmessage?token=<AccessToken>
Method always POST
Body clipmessage=<JSON formatted clipCommand>

clipCommand==>
{
   “clipCommand” :
         {
            “url”: “api/0/<some resource>”,
            “method”: “GET/PUT/POST/DELETE”,
            “body”: JSON formatted data
         }
}
Header encoding: UFT-8
application/x-www-form-urlencoded

Note: the url in clipCommand uses ‘0’ what in a local scenario would be the username.

Below an example of a command that will turn on light 1.

URL https://www.meethue.com/api/sendmessage?token=<AccessToken>
Method POST
Body clipmessage=
{
    “clipCommand” :
         {
            “url”: “/api/0/lights/1/state”,
            “method”: “PUT”,
            “body”:
               { “on”: true }
         }
}

To retrieve the status of the Hue system you need to sent a message as follows:

URL https://www.meethue.com/api/getbridge?token=<AccessToken>
Method GET

The result will be a JSON string representing the current state of your Hue system.

AccessToken

The biggest challenge in all this is to obtain a valid access token. In the next step I’m going to get the access token that IFTTT is using. But you could also use the access token of the official Philips Hue app.

Steps:

1- Go to IFTTT

2- Search for Channel ‘Philips Hue’

3- Activate Channel, this will bring you to the Hue Portal and log you in, and ask you to trust IFTTT. Click ‘Yes’. The Hue Portal will generate an access token and IFTTT will use that access token to send command to your Hue bridge.

4- When channel is activated logon to Hue Portal

5- Go to ‘Settings’

6- Click on ‘My Apps’ and you will see a list of apps you trusted for accessing your Hue bridge.

image

7- Copy the link of the ‘(De-activate)’ link. It will contain the access token. Here’s an example of my token: https://www.meethue.com/en-us/user/preferencesappsrevoke?tokenId=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaTT0%3D
The access token you should send in the remote message should be: “aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaTT0=” (without the double quotes and make sure to URL encode the whole thing).

BTW: If you revoke the access token by clicking ‘De-activate’ the access token cannot be used anymore. You will get a JSON response containing the message “I don't know that token”.

Code samples

We now have the message structure and the access token. We can now put this together in a C# sample.

   1: dynamic command = new ExpandoObject();
   2: command.clipCommand = new
   3: {
   4:     //change state of light 1
   5:     url = "/api/0/lights/1/state",
   6:     method = "PUT",
   7:     body = new
   8:     {
   9:       //set the light state changes
  10:       on = true
  11:     }
  12: };
  13:  
  14: SendMessage(command).Wait();

Lines 1-12 builds the clipCommand.
Line 14: sends the command

   1: private static async Task SendMessage(dynamic command)
   2: {
   3:   string jsonMessage = JsonConvert.SerializeObject(command);
   4:  
   5:   HttpClient client = new HttpClient();      
   6:   HttpContent content = new StringContent("clipmessage=" + jsonMessage, Encoding.UTF8, "application/x-www-form-urlencoded");
   7:  
   8:   Uri remoteMessageUrl = new Uri("https://www.meethue.com/api/sendmessage?token=<YOUR ACCESS TOKEN>");
   9:   var result = await client.PostAsync(remoteMessageUrl, content).ConfigureAwait(false);
  10:   var jsonResult = await result.Content.ReadAsStringAsync().ConfigureAwait(false);  
  11: }

Line 3: creates JSON formatted string of the command.
line 6: creates the body of the message, including the encoding and mediatype
line 8: creates URI including the access token
Line 9: sends request to Hue Portal
line 10: will contain results of the request.
Possible results could be:
success: "{\"code\":200,\"message\":\"ok\",\"result\":\"ok\"}"
failure: "{\"code\":109,\"message\":\"I don\\u0027t know that token.\",\"result\":\"error\"}"
failure: "{\"code\":113,\"message\":\"Invalid JSON.\",\"result\":\"error\"}"

 

Show me the Code

Get the C# sample from https://github.com/koenvanderlinden/RemoteHueDemo that contains sample for sending a message to change lights and how to retrieve bridge status.
Let me know if you think this blog post helped you in building a Hue enable application ;-).

TODO: getting the access token is still done by hand and it reuses another popular Hue application. Next step will be how to make the Hue Portal trust your own third party Hue app so you can use your own access token.

Wednesday, February 5, 2014

SharePoint issue “The context has expired and can no longer be used. Exception from HRESULT 0x80090317”

Problem
A client had problems with some workflow instances. Most instances did complete without a problem others failed.  Especially workflow that continued after a few days reported the following error in the workflow history log:
“{Microsoft.SharePoint.SPException: The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317) ---> System.Runtime.InteropServices.COMException (0x80090317): The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317)
   at Microsoft.SharePoint.Library.SPRequestInternalClass.SetHttpParameters(String bstrHttpMethod, String “

I did some googling and found some hits that found a solution in synchronizing the date/time of the WFE’s and admin servers. In our case that wasn’t the the case.

Cause
After digging into the problem I noticed particular workflow activity opened a site collection using an SPUserToken.
using (SPSite site = new SPSite(WorkflowProperties.SiteId, InitializationData.UserToken))
{
//do some work
}
The SPUserToken was part of the workflow context and was populated when the workflow instance was first created. Default the SPUserToken is valid for 1440 minutes (1 day). In case the workflow instance is de-hydrated after more than 1 day the SPUserToken is expired and isn’t valid anymore. You will get the message “The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317” and are not allowed to open the sitecollection.

Conclusion
So when you get this specific error make sure to check if you use an old SPUserToken! And be aware the the SPUserToken can expire.

Detailed exception information
Full error message: "The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317)"
Sample Stacktrace:
{Microsoft.SharePoint.SPException: The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317) ---> System.Runtime.InteropServices.COMException (0x80090317): The context has expired and can no longer be used. (Exception from HRESULT: 0x80090317)
   at Microsoft.SharePoint.Library.SPRequestInternalClass.SetHttpParameters(String bstrHttpMethod, String bstrRequestDigest, UInt32 flags, Guid gTranLockerId, Byte[]& ppsaImpersonateUserToken, Boolean bIgnoreTimeout, String bstrUserLogin, String bstrUserKey, UInt32 ulRoleCount, String bstrRoles, Boolean bWindowsMode, ApplicationPrincipalInfo& pAppUserInfo, Boolean bInvalidateCachedConfigurationProperties, Int32 lAppDomainId, ISPManagedObjectFactory pFactory, Boolean bCallstack)
   at Microsoft.SharePoint.Library.SPRequest.SetHttpParameters(String bstrHttpMethod, String bstrRequestDigest, UInt32 flags, Guid gTranLockerId, Byte[]& ppsaImpersonateUserToken, Boolean bIgnoreTimeout, String bstrUserLogin, String bstrUserKey, UInt32 ulRoleCount, String bstrRoles, Boolean bWindowsMode, ApplicationPrincipalInfo& pAppUserInfo, Boolean bInvalidateCachedConfigurationProperties, Int32 lAppDomainId, ISPManagedObjectFactory pFactory, Boolean bCallstack)
   --- End of inner exception stack trace ---
   at Microsoft.SharePoint.SPGlobal.HandleComException(COMException comEx)
   at Microsoft.SharePoint.Library.SPRequest.SetHttpParameters(String bstrHttpMethod, String bstrRequestDigest, UInt32 flags, Guid gTranLockerId, Byte[]& ppsaImpersonateUserToken, Boolean bIgnoreTimeout, String bstrUserLogin, String bstrUserKey, UInt32 ulRoleCount, String bstrRoles, Boolean bWindowsMode, ApplicationPrincipalInfo& pAppUserInfo, Boolean bInvalidateCachedConfigurationProperties, Int32 lAppDomainId, ISPManagedObjectFactory pFactory, Boolean bCallstack)
   at Microsoft.SharePoint.SPGlobal.CreateSPRequestAndSetIdentity(SPSite site, String name, Boolean bNotGlobalAdminCode, String strUrl, Boolean bNotAddToContext, Byte[] UserToken, String userName, Boolean bIgnoreTokenTimeout, Boolean bAsAnonymous)
   at Microsoft.SharePoint.SPWeb.InitializeSPRequest()
   at Microsoft.SharePoint.SPWeb.InitWebPublic()
   at Microsoft.SharePoint.SPWeb.get_CurrentUser()

Thursday, January 30, 2014

How to handle ‘multicultural’ issues with the formula of a SharePoint SPFieldCalculated

Setting a formula in code for a SharePoint SPFieldCalculated is a real nasty task. I spent a lot of time figuring out why some formula’s did work and others did not in various cases.

When designing a formula you may think you need to take a few things into account like language and separator. During testing I came to the conclusion that the following has impact:
- thread culture
- separator character used for separating parameters in the formula
- SPContext.Current == null or not
- SharePoint default language (what language was the original SharePoint setup)
- and that the language of the formula method (like CONCATENATE) is not important at all.

Simple Test Case
First let me tell you about my test case I used. Two text fields, one is a company tag, the other is the first name of a contact person. A third will be the calculated field. It will need to show the following: ‘<Company Tag> - <First Name>’, without the ‘.
SPFieldText internalName: companyTag, DisplayName (English) = “Company Tag”, DisplayName (Dutch) = “Tag bedrijf”
SPFieldText internalName: contactFirstName, DisplayName (English) = “First Name”, DisplayName (Dutch) = “Voornaam”,
The creation of the calculated field should be in code (in this case a feature receiver). The deployment should work in various cases like:
Case 1: Activating feature by hand in UI.
Case 2: Feature is activated by onet.xml of a web template.

The problem:
The problem arises when you need it to work for case 1 and case 2 in different cultures. In my case in English and Dutch. By testing the creation using the following scenario’s you will see where it fails.

Scenario 1: Case 1 in English thread culture
Formula: “=CONCATENATE([Company Tag],\” – \”,[First Name])”
Separator: , (English)
Display names:  all English
Result: It works…

Scenario 2: Case 1 in Dutch thread culture
Formula: “=CONCATENATE([Tag bedrijf];\” – \”;[Voornaam])”
Separator: ; (Dutch)
Display names:  all Dutch
Result: It works…

Scenario 3: Case 2 in English thread culture
Same as Scenario 1:
Result: It works…

Scenario 4: Case 2 in Dutch thread culture
Same as Scenario 2:
Result: It does NOT work. 

Why doesn’t scenario 4 work? Threads are all Dutch, passing all the same parameters for the formula as scenario 2. So why…?
Also tried various combinations on the formula like:
- English display names, English and Dutch separator,
- Using Internal names, English and Dutch separator,
- Using Dutch formula method names like “=TEKST.SAMENVOEGEN([Tag bedrijf];\” – \”;[Voornaam])”

Only thing that seems to work was setting the separator to the default English (the default SharePoint installation culture) variant:
Formula: “=CONCATENATE([Tag bedrijf];\” – \”;[Voornaam])”
Separator: , (English)
Display names:  all Dutch

The main difference I detected between case 1 and case 2 was the SPContext.Current was null in case 2.
In case you design a formula and need to determine the separator, do not only check the Thread culture, but also the existence of SPContext.Current.

Guidelines regarding formula
Adding all variations of testing and failures together I came up with the following guideline when designing the formula for the SPFieldCalculated.
- Formula string should start with a '=' (not shown in above cases)
- always encapsulate the display name with brackets [], better safe than sorry (not shown in above cases)
- in case SPContext.Current is NOT null
* use field display names for the Thread.CurrentThread.CurrentUICulture
* use separator for the Thread.CurrentThread.CurrentUICulture  (Dutch= ‘;’ English=”,”)
- in case SPContext.Current==null (e.g.: when feature code via activated by an onet.xml of a webtemplate)
* use field display names for the Thread.CurrentThread.CurrentUICulture
* use separator that matched the SharePoint Initial Installation Language (English on my development machine. Contact me if you know how to determine this in code!)
- Methods names in the formula are culture independent, you can use English/Dutch/Whatever without a problem. (e.g. CONCATENATE and TEKST.SAMENVOEGEN work in English and Dutch thread culture)

I hope I saved you some valuable time in creating formula’s for SPFieldCalculated fields in code.
New insights on the formula are welcome.