One step forward, two steps back

It's been a while since the last update, mainly because things haven't progressed a great deal.
I reinstalled the previous version of Ubuntu and it did allow the system to run, however, I started encountering problems with the IR driver locking up. There was nothing obviously wrong (and nothing on the manufacturer website) so I decided to start again, Reinstall version 9.10 of Ubuntu again but not apply any updates.

Busy weekends and long working weeks conspired to delay this but I managed to do this last Friday and it appeared to work.

I then spent the weekend implementing (most of) the configuration file system to optimise startup times (in the end I didn't use Boost, but went for a simpler solution - a shame since that was what started all these problems). I also took the opportunity to rework the channel picker control so you now have to press a button to activate the selection. Everything seemed to be working well, but when I disconnected all the development kit and reinstalled the server in the lounge, the IR crash problems started to appear again.
I've managed to do some investigations and found that the daemon appears to lock and refuse to send any IR signals - the only solution is to restart the daemon. More testing revealed this happens quite quickly under some circumstances, but doesn't seem to happen at all at other times. My conculsion is that the way in which I'm interfacing to the driver must be problematic, so I need to revisit that.

So, I need to look at the way I'm driving the IR device through the driver - possibly write some test code to try different techniques of making it work and see which ones are the most reliable. I also need to finish the configuration file mechanism and also do some more optimisations for speed of network connection. Finally, I want to enhance the control layout mechanism so I can test drive some better layouts for controlling devices.

Error. Redo from start

To help develop the file transfer system I decided was needed for Elmo, I decided to install the C++ Boost library on the server (Boost is particularly useful for cross-platform development).
Since my Ubuntu package manager had been complaining that my version was very old, I decided to upgrade to the new version 10 - Lardy Lumpfish, or whatever it's called.

BIG Mistake.

There are no graphics drivers for my hardware in this version. Having searched all over the place I still can't find a decent solution and the way things are at the moment means I can't boot into the system without getting an error and having to select to rebuild the display config (even when no display is connected). This is pretty hopeless for a system that isn't supposed to have a display.

After much deliberation, I've decided to reinstall from scratch the previous version - Karmic Koala - and reapply all the patches. This does give me the opportunity to document the process, which is probably no bad thing.

Obviously, I can't make any other progress until this is resolved

Process of continual improvement

So, Elmo has been installed and running for about three weeks now. This has allowed time to see how it really behaves when used in anger. Technically it has been generally working well, with just a couple of glitches to do with booting Ubuntu, however there are a number of "issues" - not really problems - that diminish its usefulness.

The biggest problem is latency. I've noticed that reconnecting to the server after you've initially powered up is actually taking quite a tangible amount of time. This is probably because the iPhone has to get back onto the WiFi network and then reconnect to the server. The upshot is that when you've been watching TV for a few minutes and decide to change channel, its not as good and experience as I'd hoped, there is a noticable delay. I have a few ideas about how to improve this, mainly around trying to predict when the user is about to press a button - I'm thinking of when the phone is unlocked as one trigger - although that probably won't give me enough time - or possibly using the accelerometer to detect when the phone is picked up.

An associated problem is that when you want to change between different items of equipment the phone has to download the new screen layout from the server. This again takes time and makes the process clunky. The simple solution is to store all of the screen layouts on the phone - only refreshing them at startup. This should be reasonably straightforward.

The screen layouts definitely need attention because the main screen is far too busy and the channel change lists aren't really useable. Because the iOS picker control basically selects an item when you stop scrolling, it means that the channel picker always comes up with a number of different channels before you get to the one you want. As each channel takes around a second to be selected by the server (3 digits transmitted over IR), this ends up being a bit tedious.

The IR transmitter is also very sensitive to positioning. I've found that changing the TV angle by about 10 degrees so the face of the TV is more closely aligned with the server helps greatly in the reliability of the signal transmission. On the whole the IR is pretty reliable now, but it still seems to go through patches where it struggles. Probably some more work is needed on the low-level IR capture. There are a small number of codes that have stopped working altogether as well.

Although there are a number of issues here, none of them are major. I intend to try to work to correct these over the next few weeks and see how sweetly I can get Elmo to sing. First step is to cache the screen layouts (because it should be simple) and then look into the connection latency issues.

Tantrums

Elmo has been throwing a few tantrums.
I left the system set up this week and encouraged my wife to use it in my absence. It wasn't entirely successful because she moved the IR transmitter which stopped the whole thing from working at all!
However, I've also noticed that the iPhone quite often loses connection to the server and won't reconnect. There is also a threading problem in the server which shouldn't be critical, but needs sorting out.
So, with all these things combined, I managed to grab some time this weekend to rework a lot of the network connection process and make it more robust. The new system relies on the responses from the server to let it know it's connected. If the responses don't arrive the iPhone starts a reconnection process. This all seems to make the whole thing far more reliable - although I did find some nasty differences between the Ubuntu and Windows implementations of the socket layer - which caused a fair bit of consternation and wasted a load of time.
While I was at it, I also added the capability to shut down the Elmo server from the iPhone - because the FitPC box runs so hot, I don't like to leave it on all the time.
It took more time than I hoped, but the results are very encouraging - both the server and iPhone now manage their socket connections in a much better way and the iPhone will queue up commands until it reconnects to the server.
I also took the opportunity to tweak a few of the IR codes - basically making the sequence longer, which is equivalent to holding the remote control button down for more time. I've done this to try to improve the reliability of turning equipment on and off.

Elmo is once again installed in the living room and we will see if this week is more successful.

The Dawn of a New Age

A few days off work has done great things for Elmo.
There was more work left to do than I realised, and lots of small details that all needed to come together to make everything work.
Having got the code support complete, I also had to set up the configuration files to link the IR up with the user interface on the phone. Quite a tedious process but straightforward.
Elmo is now resplendent in the living room and is controlling the AV kit including the amplifier, TV, Bluray and Sky box. I am generally quite pleased with the results and the way the system functions in real life. I have found the angle of the transmitter dongle is quite critical - it seems to have a narrow angle of transmission - and also that reliability suffers when the sun is shining through the window - only to be expected, I guess.
The final insult was that my Provisioning certificate for my development iPhone had expired. This wasn't immediately obvious and I searched high and low for the reason my software wouldn't deploy to the phone before noticing this. It was easy to regenerate the certificate and then everything worked straight away.
I now plan to leave Elmo in-situ and get a real feel for what works well and what doesn't and alsonwhat problems come up - there's nothing like using a system in earnest to shake out all the bugs.
It's very good news, though, because after all the work and setbacks, the system that I envisaged nearly 2 years ago is now a reality.

Channelling

Finally, some good progress to report.
The Status engine is up and running on the server simulator, as is the sequencer. I managed to spend a good bit of time over the weekend working to complete these and also started one other item that I'd forgotten about, namely the channel selector.
I had some rudimentary ideas about how the TV channel was going to be selected but after spending nearly an hour going through all the Sky channels and mapping channel numbers to names, it seemed that the only sensible way to do this was going to be in a list and use a Picker to select the channel. Because there are so many channels, I'll arrange them into groups and use a split Picker control on the iPhone to select first the category and then the channel.
I managed to dig out some sample code from my days of learning from the iPhone book and have got the appropriate Picker view up and running. I also have an XML file listing the channel names, numbers and categories.
Next step is to update the server simulator to serve the channels file and the iPhone app to consume it and link it into the Picker. I then need to decide how to drive the commands to change the channel (since it requires a sequence of button presses) and how to store the selected channel.
I have some time off work this week and hope to spend some of it doing this and also getting the new code running on the real server hardware. Then I'd like to run some actual live tests and see how the system hangs together in use.

A Standpoint of status

Over the weekend I managed to integrate my recent server changes onto the Linux box and get it up and running with the iPhone simulator. That means it should just be a short step to a configurable system running on the real hardware.

I also started looking at the other things needed to get the system up and running in a basic form and the main part of the puzzle missing is a Status engine.

The Status engine remembers the state of various parts of the system it is controlling so, for instance, if you set the TV to external input and the amplifier to DVD, the status engine remembers this and won't set these settings again. This then allows activity based control where you tell the system that you want to watch a DVD rather than selecting the inputs on the different devices manually. It also allows control of things such as Sky TV guide where the controls have different functions depending upon which menu you're in. Initially I intend to use this only in a basic way and then experiment with how far it can take the system.

The final part of the initial system will be the sequencer, which allows a single button press to initiate a number of commands.

I hope to get both the Status engine and sequencer working this week and then perform some testing at the weekend.

Fit the Second

I've decided to dub this latest version of Elmo v0.2.
Most of the work I've been doing has been on the iPhone app - generally this is more problematic than the server because of my unfamiliarity of the development system. I've got to a point now where I've separated the raw network protocol from the Elmo specific protocol and the application logic - although it's not a brilliant implementation because the structure has evolved as I've learnt, it is along the right lines and should allow quite a bit of flexibility in what I do.
The low level network component has been stable for some time and most of my work has been on the Elmo protocol - which is now working well in both directions. The View rendering has been rewritten to use the configuration downloaded from the server rather than hard-coded data.
The final piece of the puzzle was in the server and that involved hooking up the commands sent by the iPhone to the actual IR codes that need to be transmitted. I decided to map these because it will make it more flexible later on, so there is a translation file on the server that takes the command from the iPhone and maps it to the device file and IR code that needs to be transmitted.
Last night I ran the whole edifice up on the iPhone and windows server simulators and it all hung together - so I now have fully configurable iPhone screens, command mapping and IR code generation. It looks the same as it did before but under the hood it's far more capable.
Next step is to get the whole lot working on the real server and iPhone and then work out what I need to do to make it into a practical system to do some initial real-life testing.

Network Nastiness

It's been very quiet on Elmo recently.
Despite my previous post it's still been difficult getting time to work on it. This hasn't been helped by long, hard days at work which make it more difficult to spend a few hours working on yet more computers in the evening.
However, when I have been able to spend time on Elmo, I've spent a lot of it rather stuck with some network problems. My problem manifested itself when I started to transfer larger amounts of data for m configuration management. It seemed that small numbers of characters were trouble free but anything over a few hundred would cause problems.
Eventually I traced the problem to packetisation - where the transport was splitting things into smaller packets - and so implemented some code to cope with this - rather poorly as it turned out. It took me a week to track down the problems in the packeting code, but once I did I had a reliable transport.
I can now transfer my configuration files from the Elmo server onto the iPhone app and have started to use my homegrown XML parser to pull the information out of these files. This is requiring a fair bit of refactoring of code but it's progressing quite well.
When I finish this part of the work, all the control layouts will be held on the server rather than the iPhone - which is a far more flexible solution.
Next step is to rework the view control render code to pull data from the configuration file.

Moving out of the slow lane

Finally, the home computer is reborn. The failure of the main household computer has meant my free time has been spent building and commissioning it's replacement. At a time when free time itself has been hard to come by, this has meant very slow progress on Elmo.

I have managed to do some work on the configuration transport. This is all XML based and so I've written a lightweight, cross-platform XML parser which helps with passing the information around. I have incorporated this into my test server and test client applications which are both hosted on Windows and that is working well.

The next step is taking rather longer because I need to port my configuration classes across to iOS. I knew this was technically possible, but I've had to research the specific details and hash up some test code to try it all out. In the event, it turned out to be very easy to do in the end but some wrong turns and mistakes meant it took a few evenings to get the details of the solution worked out.

I am now at the stage where I am ready to start porting the configuration transport code across to the iPhone. This will need to be wrapped in an objective C++ class so that the rest of the app can talk to it. I then intend to expand on the current configuration capabilities and then start to work on the other aspects of configuration, eventually getting to the point where I can define functions based on activities rather than individual device commands. Still a way to go yet.

Slowed by the real World

It's been a rather quiet time in the World of Elmo of late. The real World has, annoyingly, been encroaching with Birthdays and failed computers stealing a lot of my time away.
I have been working on the configuration of the remote app. I intend to store all the screen layouts on the server and download them to the device when needed since this gives a number of benefits in terms of system characteristics. I have managed to dynamically create tab views on the iPhone and am now working on a transport mechanism to get the information from the server on to the phone. Hopefully I will be able to devote a bit more time to this in the next couple of weeks.
Moving on, I've also been considering different control methods on the iPhone itself. A button box is only one of the possible solutions and I'd like to be a little more imaginative if I can.
I'm still treating this code as very much prototype but am starting to see ways of building in quite a lot of functionality in some rather interesting ways. Time will tell if these will work or not.

The next step is to complete the transposition of the iPhone software from a hardcoded button box to the genesis of a control browser, downloading content from the server.

The (illustrated) story so far

Picking up this project after it had been idle for a year meant going back to the drawing board and rediscovering many things. The last 2 months have seen a great deal of progress after the frustrations of my previous attempts. Here are some quick highlights.

April
Having identified the problems with receiving IR through the wrong sensor, there was much work to understand the IR codes being received. This is an early attempt at analysing them:

Early May
After many false starts, I eventually found some techniques to learn (most) IR codes that I encountered:

This was coupled with a new Linux program to capture the IR in the first place:


Mid May
Successful testing of the IR transmission:


The initial prototype iPhone app running (teathered):


Late May
In the past few days, I have developed a new test app which allows all the functions of my AV equipment to be controlled. Here is the advanced design system in use:

As I write this, it is about to go into testing:

The Real Deal

I had a 4 day weekend and although there were many things to do, I managed to squeeze a few hours of Elmo into the proceedings.

During the week I had managed to create a simple iOS App which basically consisted of a number of buttons (rather hopefully labelled with various satellite box functions). On Thursday I hooked this up with some hastily engineered network code to produce something that, on the simulator at least, could drive the Elmo Linux box to produce IR signals.

Flushed with this success, I embarked on the painful process required to run this creation on a real device. It soon became clear that I'd need to update my version of XCode (4Gb of download) as well as update my development iPhone 3G (650Mb download). On my woeful broadband connection this took somewhere around 5-6 hours to download.

So it was quite late on Thursday that I fired up the provisioning portal on the Apple dev site and, after two attempts, managed to get to a point where I could run my software on my iPhone via the tethered debug interface. Wow! Not as painful a process as I was expecting.

About half-an-hour of research and fiddling on Friday morning got me to the point of being able to run the App standalone without the teather.

Finally, on Saturday morning, I set the system up in the lounge and successfully controlled the satellite box with my iPhone via the Linux box. The real set of devices were now communicating! There was only one screen of hard-coded buttons, but all the parts I need for the full system are now working - fantastic news indeed.

I managed to grab a couple of hours over the rest of the weekend which allowed me to start developing a more fully featured App which has multiple pages of buttons and that development continues into this week whilst I'm away from home.

Elmo is Born

There has been a great deal of work over the past week. This has mostly been concerned with the network comms side of things. Because I decided to run an XML based protocol I needed to implement some support code to drive it all. The result is rather pleasing in its functionality and simplicity.

Having got the basic comms up and running, I then wanted to be able to query the Elmo server for the available devices and commands, so a Catalogue was needed to keep track of this information and the appropriate XML formats designed.

I wrote the client in Windows (using MFC - that takes me back) and integrated my previous socket client into the new program. I then added my socket server to the server side and was able to get command catalogue information passed between the two machines. I also got a cross-compiled Windows-based test server so I could continue to develop away from home.

Come the weekend it was time to link this up to the IR side of things. That took a few hours on Saturday, so on Sunday the big moment of truth. I set all the test kit up in the lounge, ran up the client on the laptop and tried to send some commands. It didn't work.

After lots of investigation and fiddling around, everything looked correct but the signals weren't being transmitted. Eventually I realised that the new server program had the wrong version of the IR Protocol library. Correcting this brought Elmo to life - finally.

The system now accepts button presses for various remote control functions on the laptop, passes them over the wireless network to the server where they are converted to the appropriate infrared signals and transmitted from the dongle. It sounds quite simple, but represents rather a lot of work, both investigating, prototyping and developing code.

There are still some limitations to the IR codes I can learn, but I intend to push on with the development now and start work on the iOS client program that will allow control from the iPhone.

You don't need pants for the victory dance

A good deal of effort and testing during the week led me to the conclusion that I need to re-implement code to decode the actual protocol of the IR signals, rather than try to do some clever, generic capturing. Some hunting on the web found 2 main protocols, a pulse width scheme and a level-based (bi-phase) scheme. Manual decoding soon showed that, broadly, all of the signals I'd captured could be classed as one of these two.

Some frenzied late evening work resulted in a new analysis program which could decode both of these schemes. Feeding in my test code showed that the results were highly consistent - giving the same values for each command from a single remote - a very promising development.

Some further tweaking before the weekend and then some quick testing revealed partial success. We had visitors at the weekend, so there was limited time to code or test but I quickly realised that the Sony devices sent their signals multiple times. I managed to grab a couple of hours to add repeat code detection and then retransmit them.

A hastily set up test rig resulted in complete success - all my codes worked and successfully controlled the equipment. I can command the TV, amplifier, satellite box and blu-ray player. Finally, the result I'd been looking for.

Having used up as much time away for our visitors as I was allowed, I didn't have any more time to work over the weekend, but I feel very satisfied that the infra-red is now working. I will need to add functionality for things like volume, which are repeating signals, but that can come later.

Next step is to integrate the new IR functionality with the network server program and to write a test client to run on the laptop. This will demonstrate a prototype of the complete Elmo system. Exciting times.

Return of the Jedi

I managed to spend a fair amount of time over the 4 day weekend working on Elmo (thank you Royal Wedding).

After initially having great success with learning the codes from the Sky remote, the amplifier was very elusive. In the end I had to write some new signal analysis software, capture the raw data on the development box and then process it on my laptop, using a bit of human intervention as well.

The results have been very encouraging as I can now control 3 different devices using the codes I captured. The only downside is the reliability of the control is not good and this is because I am always transmitting the same code, whereas most IR remotes alternate between two codes each time you press the button (so the device can tell the difference between holding down a button and pressing it twice in rapid succession). I had anticipated this on the transmission side, but do not yet have a capture algorithm that can find the two versions of the signal.

My tasks during the week whilst away from home are firstly to develop an algorithm to detect the two codes. To this end, I have captured raw data from the 4 devices I originally want to control and will feed this through my new software as I develop it. Secondly, I need to develop the client-server software so I can actually control some devices - initially using the laptop. When I can achieve both of these, a fledgling Elmo will have been born.

Time and Tide

As anticipated, there was little time to work on Elmo over the Easter weekend.

My time during the week is firstly being spent on getting a cross-platform version running on my Windows development machine and then on working on the analysis of the received signals. Hoping that the next long weekend coming up will see me able to devote more time.

Early Learning

Another weekend with little time to work on the project.
During the week I'd written an analyser which took previously captured codes and worked out the commonalities. With what I learned from this, I wrote some test code to see if the real capture process worked the way I thought it did and, broadly, it does.

It's ended up being rather more involved than I wanted but I think I have a clear view of what needs to happen. Due to the inaccuracies that creep in with timing and ambient conditions the results from the IR receiver are not always identical. This means that some degree of fuzzy logic needs to exist in the learning process and fuzziness is something that's difficult for computers to do.

I managed to get as far as working out the best frequency fit for each remote control and selecting candidate signals that looked basically compatible. The next challenge is to work out when a signal is a slightly different version of one I've already received and when it's a different signal altogether. Some remotes alternate signals and so this has to be taken into account as well. This basically looks as though it should work, but some tweaking of the parameters will probably be required in order to obtain the most reliable results.

Next weekend is Easter, so I doubt much will get done, but I'm trying to make at least a little progress each week to keep the momentum going.

I fought I.R. and ... I won

My previous assessment of interfacing directly to the IR device was, surprisingly, confirmed this weekend.

Due to the glorious weather this weekend, I didn't have a great deal of time, but I managed to take the basic code I'd written in the week, port it onto the Linux box and successfully transmit codes to the Sky box (the digit 1). Having done that, with some more work I managed to capture IR from the remote control and confirm the numbers I was capturing matched those I'd previously sent. The trickiest part was getting all the data in the correct format since it doesn't seem to be consistent for different messages.

The upshot of all this is that Elmo can now capture IR signals from a remote control and regurgitate them to control devices (so far, only the Sky box). LIRC, the bane of my life, has been abandoned and I'm communicating directly with the IR device driver via a pipe. Initial indications are that the codes captured are very consistent.

I've taken the time to capture codes from 4 different remotes so I can work out the best way to analyse and store the codes - which is the task for this week whilst I'm away from home. If I can get a reasonable mechanism to do this then the most difficult part should be over. It's taken a very long time to get this far!

Next step: Analyse the captured remote codes and decide on the best way to record and store them. Write some code to support this, ready for porting to Linux at the weekend.

Signs of life

I managed to find some time on Monday to look through the IR dongle utilities code. To my surprise, using the device seems to be reasonably straightforward (a first on this project). There are a number of utility functions which can be reused to drive the basic protocol which just leaves the higher, packet, level to drive.

I was lucky enough to grab some time yesterday and quickly put together a program using the utility code to open a connection and read some data from the dongle. It worked! Knowing this, I can now work on some basic learning functionality so I can program up the remote controls I have. My plan is to keep this fairly simple for now and refine it later. The focus is to be able to issue individual commands to multiple individual devices. I already have a demonstration of the networking ability, so that can be put to one side for now.

Right now it feels like the project once again has a future, but this will be tempered by the reliability I manage to get from the learned codes. I fear there may still be a way to go yet.

Wonders of USB

My previous idea of using third-party IR codes was all very well until I found that the majority of remote controls I have do not seem to have files available on the net. So, I'm back to having to capture my own IR.

During my usual weekend frenzy of work on the project, I discovered the problem with capturing IR was that, once again, LIRC has been using the internal IR receiver rather than the external dongle I want it to use. I finally managed to disable the internal device by blacklisting the driver but now LIRC doesn't want to work at all!

I think there is a problem within the system because both the interal and external receiver use USB, but I cannot confirm this. Anyhow, I then decided to revert to using the tools supplied by the dongle manufacturer and finally had some success! I've managed to record some IR and retransmit it, successfully controlling the satellite box!

The process is currently rather tedious and manual - requiring copying dats from one tool into another, but I've downloaded the source code and am hoping to find the interfaces I need to access the interfaces programmatically. Hopefully this will then allow me to reliably record and retransmit IR signals. I'm quite excited by the prospect but I've thought I was at this point before and it all came to naught. The good thing is that I can examine the source during the week when I'm away from home.

Light dawns

After a slight change of tactic over the weekend, there has been some success with Elmo.
I decided to set aside the issue of decoding the infrared signals and switch to a set of IR codes produced by a third party. This led to some moderate success in controlling my satellite reciever. With a number of tweaks and tricks I was able to get moderate reliability of reception from the satellite box. I then went in search of some alternate codes and found a set which works very well with the box. The new set are actually decoded fully, rather than relying on the raw codes that the other file used - this looks like the way to go.

I then married my previous work with a socket server together with my infrared code and got to a point where I could issue commands on one machine and get the Linux box to send the appropriate IR codes. This is essential for the final solution since I don't want a display plugged into the Linux box.

I've not been able to find files for my other equipment so I'm back to trying to decode the IR signals in order to create my own config file. I'm hoping that the example file I already have will help in this process.

It's a good step forward, though, and has illustrated that the whole idea looks practical and achievable.

Close, but no cigar

A weekend of frenzied Elmoness has seen progress on both receiving and sending IR signals. My test programs can now do both, the only problem being that the two aren't compatible - codes recorded don't activate the equipment as expected. When I use the numbers captured by the real LIRC programs, everything is fine. Although frustrating, this means that the basic mechanisms are there, I'm just missing something on the reception side. I'm wondering about the sample frequency of the receiver (and transmitter).

My receive and record program can get consistent results and distinguish different buttons on the remote which is better than I've managed before. Transmit is cheating by using a program provided by the IR transceiver manufacturer.

If I can get the reception calibrated in line with transmission, the learning process I've come up with will work much better than the LIRC default one, which is rather long-winded. I'm hoping this doesn't prove too difficult, but only time will tell.

Hiatus

Elmo has been in hibernation for over a year now due to other commitments and loss of momentum. However the purchase of some new equipment has prompted me to start work once again.
I've finally fathomed out my problems with IR reception (down to my own misunderstanding of the protocol) and I've managed to recover the development environment to the point of building test code again. I've also spent some time learning about iOS programming.

I suspect progress will be slow since I'm away from home alot at the moment.

Having managed to receive some meaningful data on the IR device, my next step is to write a tool that makes some sense of the data in preparation for retransmission. I intend to focus on a single IR remote control for now and get to the point of being able to send a few simple commands. This should be reasonable simple, but with IR it never seems to quite work that way.

More news, as it happens.