Rubikon

By taking inspiration from the original Rubik's Cube we propose a new way of interacting with a personal computer. Rubikon is an independent interactive device which can, not only be used as a controller, but also a personal device. Over the period of 8 weeks we created a prototype which can be used to control a number of applications on a user's personal computer, as shown in the video below:

For any new interactive device it's success depends on user feedback; Rubikon is no different. A group of fifteen individuals were asked to independently carry out various tasks, such as navigating through the file system, using the Rubikon and. They were then asked to rate their experience on a scale of 1 - 5 with 5 being very intuitive. The results of the experiments can be seen in the graph below:

The general consensus among the participants was that it was very straight forward for them to interact with the personal computer using their basic knowledge of physics and how various applications worked.

This work in progress device can be made standalone and platform-independent. This can be achieved by adding wireless connectivity, replacing the current Arduino board with a smaller integrated controller board, and adding a portable power source.
In conclusion, Rubikon is a highly configurable independent device enabling a diverse set of interactions. Despite having many degrees of freedom and a large number of possible rotations, use remains intuitive and easy to master.

Software

Having finished building the cube I decided to help out the software team. For the demo Vlad, Rupert and Max had implemented an OpenFrameworks application which took input from Arduino code. However under certain circumstances the input received by OpenFrameworks wasn't exactly what was sent by Arduino, so I suggested a way of getting around that.

Apart from that, me and Vlad spent a good few hours researching how we could run the OpenFrameworks application as a service in background so that when Windows is locked, the user can simply use Rubikon to log in. This however proved unsuccessful, but it is nonetheless still achievable when the entire application is written as a service.

Modifying the core - Part 2

Now that I was able to reassemble the Rubik's Void around the core and all the encoders were in place, the next step was to attach the custom-built mechanism to the cube so that when a side of the cube is rotated, it rotates the corresponding encoder as well.

Upon attaching the mechanism it turned out that it's bottom layer was thick and therefore prevented movement of parts of the cube (as discussed in Designing parts of the cube). This is caused by the shape of the core; since the core is spherical some part of it comes out of the centre of Rubik's Void (shown in image below).

At first I spent couple of hours grazing the bottom layers along the path where each side moves. This however proved to be unsuccessful. So I was faced with 2 choices, either design the bottom layer of the mechanism to be spherical in shape, so it could perfectly fit the shape of the sphere, or modify the core so that the area around the encoders is flat, to match the shape of the bottom layer of the mechanism. Images below.

I explained the problem to Vlad and he suggested modifying the core because we couldn't think of a way to make a spherical bottom layer. So after spending couple of hours making areas around the encoders flat, I was finally able to reassemble the cube and perfectly fit the mechanism around all the encoders. The mechanics of the cube worked as intended, i.e. when each side of the cube was moved it in turn moved the encoder it was attached to, and each encoder could also be used as a button. Below is the picture of the final Rubikon:

Modifying the core - Part 1

Having built the core for the cube and the mechanism which will rotate the encoders when sides of cubes are rotated it was time to build the Rubik's Void around the core and put the mechanism in place.

After me and Madalina built the cube around the core by de-assembling the Rubik's Void and then reassembling it around the core we came across a bit of a hurdle. When drilling holes in the core for the encoders we weren't very precise about their location. This, as I insisted during the process of drilling the holes, has caused the encoders to be misaligned with respect to the sides of the cube (image below).

At this point Madalina had to leave so I now had two options: I either buy a new sphere and rebuild the core from scratch, or just modify the existing core by turning the encoder holes into rectangles along which the encoders could slid. The latter option was more appealing as it required less time and since the encoders came with a bolt it was easy to tighten them in the place best suited with respect to the side of the cube.

After few hours of modification to the sphere I reassembled the Rubik's Void around the core and tightened the encoders in place when they were centered with respect to the side of the Rubik's Void (image below).

Designing parts for the cube

Having built the core and tested the functionality of the encoders inside it the next step was to build mechanisms which will rotate the encoder when the corresponding side of Rubik's void is rotated. Currently a side of the cube looks like this:

The mechanism will go where the arrow is pointing. Together me, Vlad and Madalina gathered ideas about how to build such a mechanism. Upon taking apart the Rubik's void, I noticed that each small cube on one side of Rubik's void slides along the square below. The arrows indicate the path in which the orange cube can move along.

So our mechanism would need to be attached to the inside of the square (in the ring). We could simply design a circle as shown in the image above however as we can see in figure 1, due to size constraints the encoder comes quite far out of the side. Therefore the circle will need to be really thick. This however creates further problem since the encoder varies in shape along it's height; therefore we needed to design different layers and put them together (image below).

The top-left design is for the top layer, bottom-left design is for the middle layer and the one of the left is for the bottom layer. The reason the top 2 layers are smaller in diameter is because when smaller cubes are moved they slide over a part of the circular hole. We decided to make the above layers using a laser cutter as it's accurate and fast giving us time to perform trial and error. Vlad designed the above layers using Adobe Illustrator and then we used the laser cutter in MVB HCI lab to build them. The material used was acrylic, however due to the shape of each encoder the height of layers was varied. The top and bottom layer was 3mm each where as the middle layer was 5mm.
Once we built the layers we realised that we needed the middle layer to be a bit thicker, but since we didn't have any thicker acrylic sheet we used an additional 3mm layer of the same design. Following is an image of all the layers super-glued together:

Building the core - Part 2

Next we needed to decide which ball to use as our core and make some holes in it. I proposed to go with the following ball as it was largest in size (allowing for more space for the encoders and the wires) and was also slightly deformable (with high resilience) which meant it would be easier to make modifications to it further down the line if need be.

First we dissected one side of the sphere to allow for wires to come out from. Then after quickly getting trained to use the drill machines in the workshop, me and Madalina made 5 more holes in the sphere. Since our calculations weren't precise the holes were misaligned and I was quite concerned about this. However, the sphere was made out of easily deformable material, therefore I decided to let go of it for now and make modifications to it later when building the cube on top of the core. That way I will know precisely where the holes need to be. Below is the image of our core after Madalina fitted all the encoders inside it:

The next task was to connect all the encoders to the Arduino. We had 40 (5 x 8 ) wires to connect, however the Arduino we had been provided with only had 13 ports. After Vlad borrowed a bigger Arduino, Madalina spent few hours connecting all the wires and writing a program to communicate with all 5 rotary encoders, their RGB LED's and their push buttons. Below is an image of the core fully connected to the Arduino:

Building the core - Part 1

The next step in the process was to decide how we were going to build a sphere with 6 holes (5 for encoders and 1 for wires). Since the video demonstrated fitting a golf ball inside the Rubik's void, I suggested we buy a golf ball and modify it to suit our needs. Madalina then went and bought a few golf balls, however upon partially dissecting one she realised that it'd quite difficult to make the ball completely hollow. So we decided to shop for a hollow ball of the size of a golf ball (or ever so slightly bigger).

Since it was quite late (past 5pm) Madalina proposed that we could try use the 3D printer instead; I was on board with the idea as it would allow us to be precise about the locations of the holes (very important further down the line) and also give us an opportunity to develop new skills.

Vlad, however, suggested that since building the sphere is going to be more of a trial and error process (as we weren't sure about the exact size) we shouldn't use the 3D printer as it could take many hours to build a model. He insisted on shopping for a hollow ball. So me and Vlad went to cabot circus and after sometime decided to purchase the following 3 balls as they were all able to fit inside the Rubik's void.

Designing the core - Part 2

Having decided that the core of our device will need to be a cube rather than a cross we realised that it would be quite challenging to build the original Rubik's cube around the core as it has no room inside it, as shown in the image below.

Instead I suggested we use Rubik's Void (at least for the prototype) which has a lot of room inside it meaning that we wouldn't have to worry about building the cube. The image below shows the insides of Rubik's Void (left) and Rubik's Cube (right). Clearly Rubik's Void has a lot of free space inside it.

Now that we are using the Rubik's void instead of the original Rubik's cube, I further suggested that our core should be spherical instead of cuboid. The reasoning behind this is that the inside of the Rubik's void is spherical, so having a spherical core would give us more space which is a big concern for us as we have 5 encoders each with 8 wires to fit inside the core. The idea came from watching the following video:

Designing the core - Part 1

After me and Rupert soldered wires to one of the encoders and received some input from it using the Arduino, Madalina took the kit and soldered wires to rest of the encoders.

The next task was to make a core out of all the encoders which would then go inside the cube. Since it would have been very challenging to make the device wireless in the given time, we decided to put encoders only on 5 sides of the cube and leave one side open for wires to come out and connect to the Arduino. The flip side of this decision is that we can't detect movement from one of the sides, this we decided is an acceptable compromise for a prototype.

We wanted to build the core of our device as close to that of the original cube as possible to later allow us to easily replicate the mechanism of a Rubik's cube. The core is to hold the encoders in place while freely allowing each of the faces to be moved, much like the core of the original Rubik's cube.

In the above image, on the left we have the core of the original Rubik's cube and on the right is the first core made by Madalina using aluminium foil. However the problem with using aluminium, as Madalina pointed out, is that it might interfere with current from the rotary encoder. Nevertheless it was a good starting point.

The next day we went to the workshop to seek advice from one of the technicians about possible ways to design such a core. We got told that we'd have to make a cross, similar to the core of the original Rubik's cube and screw the encoders to it. Me and Madalina thought this was infeasible because we needed the centre of the core to be empty to leave room for wires. So the only other option was to use a small cube, drill holes on each side and attach encoders to it.

Rotary encoder

Having decided what functionalities our device should incorporate we decided to split the group into 2 teams: software side and hardware. Since I haven't had much experience with electronics I opted to work on the hardware team along with Madalina, while Rupert, Max and Vlad decided to work on the software side of things.

After receiving the rotary encoders I wanted to get started with building the device as soon as possible just in case we had to drastically change our design further down the line. However, Madalina had other commitments at the time so me and Rupert decided to make a start.

The two of us went to the workshop to solder wires to each of the eight pins of an encoder but soon came to a disagreement about how the wires should be connected. We had two options, either solder the encoder to a breadboard and then solder wires to the breadboard, or skip the breadboard and solder the wires directly to the encoder.
Rupert wanted to use the breadboard because it would be more stable whereas I thought using the breadboard would give us problems further down the line when we have space constraints, since all the 6 encoders along with eight wires for each of them would need to fit inside a Rubik's cube.
Since we couldn't come to an agreement I proposed we make one of each, get the group's input on it and solder rest of the encoder on a later date.

After the workshop we went back to the MVB with our the two soldered rotary encoders to see which one the rest of the group preferred. Although Madalina and Vlad both acknowledged Rupert's concern about not using a breadboard we agreed in the end that it's best if we just solder the wires directly to the encoder to save some space.

Having agreed on the design for our encoders, me and Rupert set about connecting the rotary encoder to the Arduino board by following this guide. We soon came to a halt and asked Vlad to help us out as he's had some previous experience with Arduinos.
With Vlad's help we were able to connect the rotary encoder to the Arduino and receive data when the encoder was rotated. The two of us then decided to take it a step further and incorporate the encoder's push button feature as well. Soon enough our program was able to receive information about both, the button and the rotation. The image below shows our rotary encoder connected to the Laptop via an Arduino.

Research Papers Relating to the Project (Group post)

In order to gather more information and possible ideas to include in our prototype, as a group we searched the Internet for, possibly recent, research papers which were related to our project. Unsurprisingly, there are quite a few academic papers on interacting with computers using alternative devices, which is our aim. Furthermore, many papers can be found relating to authentication, this being one of the primary features we plan to implement.

Below are the 10 best papers we have found:
1) Spontaneous Interaction with Everyday Devices Using a PDA
2) SideSight: multi-"touch" interaction around small devices
3) Reality-based interaction: a framework for post-WIMP interfaces
4) The cubic mouse: a new device for three-dimensional input
5) Real-time 3D interaction with ActiveCube
6) Sculpting: an interactive volumetric modelling technique
7) Reducing shoulder-surfing by using gaze-based password entry
8 ) An Association-Based Graphical Password Design Resistant to Shoulder-Surfing Attack
9) Novel Shoulder-Surfing Resistant Haptic-based Graphical Password
10) Weighted finite-state transducers in speech recognition

Initial Rubik's Cube design

After deciding to go ahead with the Rubik's cube idea we started putting designs together as to what the cube should look like and what functionalities it should incorporate.

The first challenge was to be able to detect each side's movement. Since I proposed the idea of Rubik's cube I had done some background research as to how this might be achieved. A possible mechanism which we could use is the encoder from a computer mouse. Each side of the cube, when rotated, could in turn rotate the encoder associated with it (image below).

We pitched our concept to Diego and realised that the mechanism need not be so sophisticated. Diego suggested that we could try and use a potentiometer instead. Not only this saved us time on building the mechanism from scratch it also gave us a good starting point since we were easily able to get our hands on a potentiometer, connect it to the Arduino and get some input from it.

Potentiometer, unfortunately, wasn't suitable for our device as it could not be rotated more than a certain degree. For our device we wanted something which could rotate as much as needed so a user isn't required to return a cube's side back to the original state after every move. After some research Madalina found that a rotary encoder would tick all the requirements.

Having decided how we would obtain basic input from the Rubik's cube, we started discussing additional features which should be present in our prototype. Some group members suggested incorporating RGB LED's in each of the smaller 27 cubes. Even though we had some ideas about how we might implement it, the task was quite challenging for a first prototype. So instead we decided to only have one RGB LED per side.
This task was fortunately simplified when Madalina said she found a rotary encoder which is also an RGB LED and a button (image below). So we decided that for our prototype we will use the rotary encoders to detect movement on each side and the RGB LED to display some sort of feedback to the user.

Final Idea

Following our discussion during the last workshop I went on a hunt to find an idea for our interactive device. After much research, and my passion of solving puzzles, I came up with the idea of creating an interactive rubik's cube.

The idea is to be able to control your computer using a rubik's cube. This includes simple tasks such as zooming in or scrolling down when reading a paper, unlocking your laptop using certain combinations, controlling a 3D simulation, etc.

I purchased a couple of rubik's cube and held a meeting with the group the next day. Everyone was on board and very excited about the idea. During the meeting I disassembled one of the cubes to explain to the group how we might go about designing the cube.
We then went to the HCI lab and pitched our idea to some of the research assistants and got positive feedback and techniques in which we can refine our device.

The uses of this device vary from accessing media player to scrolling pages on a web browser or even controlling a 3D simulation. The way it differs from keyboard or mouse is that first it's wireless, second it can be personalized to suit user's needs and third its just a fun way of interacting with your computer.

The device moves us away from WIMP interfaces and provides an opportunity to users to personalise the way they interact with their personal computers. The idea can further be extended such that the cube becomes more than just a controller; the cube itself can be a personal device where each of the 27 smaller cubes are screens. Then it can be used to do things like answering your calls, an alarm clock or even checking your emails.

Workshop 2 – Arduino and Further brainstorming

During this session we were given access to several tools to give us an understanding of what we can do with the kit provided to us and also to further develop our ideas.

We started by installing the SDK on the laptop provided to us and then ran through couple of examples which gave us a chance to play around with the LEDs and the temperature sensors.

Prior to the workshop I carried out some independent research regarding our two ideas from the previous workshop. It turned out that our idea of creating a bat mapper was quite ambitious and perhaps unachievable in the 5 weeks period. Me and Vlad spoke to a few research assistants about our concerns and based on their feedback we decided that it is perhaps best to not pursue this idea.

This left us with the subliminal controller, which I personally wasn't a big fond of as I couldn't see many applications of it. Since we couldn't think of any other ideas, we decided as a team that unless I come up with something better before Friday, we'll go with the subliminal controller.

During the workshop we also used Occulus Rift and Leap motion and discussed with the research assistants about how they work. This not only helped us understand its functionality but also inspired us to think about how we can incorporate them in our interactive device.

Workshop 1 - Final two ideas (Group post)

As a group we crystallised our ideas into two main ones. The first idea was inspired from the movie The Dark Knight (Batman), where the main character uses a cluster of mobile phones as an alternative to sonar devices to map and visualise an enclosed environment.

Since the mobile devices in the market today are not sensitive enough to reliably intercept the sound signals, our approach to this is going to be to use sonar devices instead as a proof of concept. This technology would be beneficial in environments where vision is limited or inexistant, such as:

• Looking around a corner or behind a wall.
• Fire rescue and combat, where vision is quite limited (smoke).
• Helping the visually impaired.

The second idea is a subliminal controller. The goal is to provide subliminal feedback to the user and input to the machine. The main output is a “RGB scent printer” which would be able to create a range of smells based on the user's experience within the application and an IllumiRoom-type of visual output which would be synced with the scents. The subliminal input is a bracelet which detects the user’s pulse and changes the in-game experience based on it. This technology could be used in:

• Gaming
• Cinemas
• Combat Simulations

Below is the video of us presenting our ideas to rest of the class.

Workshop 1 – Brainstorming ideas

During this workshop we gathered together as a group for the first time and discussed the ideas for our interactive device. This workshop was structured to give each of us an opportunity to share our ideas between the group as well as an entire group . First we were all asked to think of up to 10 ideas. This gave each of us a chance to think about and express what we want to achieve by the end of this unit. However, it was quite hard to come up with these ideas when put on the spot. Nevertheless I proposed the following three ideas:

• Tangible surface – a surface made up of an array of thin metal rods which move up or down to form a shape such as buildings or terrain.
• Bat mapper – using a cluster of sonar devices to detect the surrounding environment.
• Mobile control plane – though not much of a challenge, the idea was to build up on existing remote control devices and enable them to be controlled over the internet.

We then all shared our ideas with the rest of the group and categorized them by the devices they would be using. This was a quite challenging as the ideas ranged from something as simple as a mobile controllable plane to something as ambitious as teleportation. Having categorized our ideas we then filtered the list down to just the following 10 ideas:

1. Bat mapper
2. Tangible surface
3. Occulus Rift as an extension to Kinect
4. Wristband to monitor heartbeat, temperature and pressure to change video game dynamics
5. Touch-screen jacket
6. Modifying terrain through vibration/electronic impulses
7. Tablet and TV interaction with Kinect - using eye tracking to scroll webpages on TV and gestures to move between pages for books
8. Smell output from games
9. Water clone/water pressure jetpack
10. Hologram which could be moved in the space and resized

We then further narrowed the list down to 2 ideas only. This was quite challenging as we had few good ideas and some of the team members felt strongly about couple of ideas. We overcame this challenge by discussing uses and complexity related to each device. The final 2 ideas, their uses and and diagrams are mentioned in the group post.

Paper review 2 - Reality Based Interaction: A Framework for Post-WIMP Interfaces

In recent years we have seen an increase in the human-computer interaction techniques towards post-WIMP devices due to a greater understanding of human psychology and availability of powerful hardware. These new devices are intuitive as they build on users' pre-existing knowledge of everyday, non-digital world to a much greater extent than before.

This paper proposes the notion of Reality-Based Interaction (RBI) and claims that much of the new post-WIMP interaction research is based on this framework. The framework focuses specifically on the following four themes from the real-world: Naive Physics, Body Awareness and Skills, Environment Awareness and Skills and Social Awareness and Skills. The author thoroughly explains the four themes and their relation to human-computer interaction.

I like the amount of evidence shown by the authors to prove their claim. They not only refer to research where each of the four themes are found but also examples which consist of more than one of these themes. Furthermore, they also refer to a CHI 2006 workshop in which they observed that researchers generally agreed that focus is moving away from WIMP interfaces. After this they carried out an informal field study, in which they noticed the themes of RBI in work of researchers who were completely oblivious to the notion of RBI.

The authors then suggest that the move towards Reality-Based interaction is a positive one. This in my opinion is not their place to say it, I think what they should have done instead is mentioned its advantages and left it for the reader to decide whether the move is really a positive one or not.

Although its obvious, the authors do point out that its not enough to make a RBI device. These interfaces should also include some unrealistic and artificial features which differs the experience from that of real-world. This might often come at cost of trade off of RBI principles. These trade offs are Expressive Power, Efficiency, Versatility, Ergonomics, Accessibility and Practicality.

The authors then give 4 case-studies which along with incorporating each of RBI themes maintain acceptable trade offs.

Today we might think of RBI as being an obvious way forward, but given that the paper was published in 2008 it definitely brought a new notion to the community e.g the authors talk about gaze control which is something we have only seen this year in a consumer product (Galaxy S4). I also like the amount of research done by the authors to prove their claims.



I would have liked to see authors mention a few disadvantages as a result of RBI, e.g. toddlers pinching magazine papers because they are used to iPad (video above). However, overall the paper is quite thorough and establishes the notion of RBI very well, mentioning not only the themes but also the trade offs.

Paper review – The Power of Mobile Notifications to Increase Wellbeing

This paper is a research into how mobile notifications can increase wellbeing logging behaviour. It starts with emphasising self-logging as a solution for wellbeing. A straightforward example in the paper is of obesity; in the US, over one third of the population is obese; a mobile application which is regularly logged into can help raise self-awareness about wellbeing.

Then the author explains how given enough data they can provide users with feedback such as “You gain weight on Sundays”, “You eat more on days when you walk less” or “You are happier on days when its colder”.

While acknowledging that a lot of information for such feedback can be acquired automatically, the author highlights the need of self-logging to obtain data such as food intake, pain levels and mood. Naturally, users lack discipline to self-log and hence the motivation for this paper.

The paper then talks about related work that has been done in this area and proved unsuccessful as it required users to enter thorough details (e.g. amount of calories) about their intake. Users also reported disliking the disruptive nature of SMS as a reminder to log.

By reviewing user feedback about related work done in this field the authors created a very simple mobile app which made use of a FitBit to log step count and sleep as well as a WiThings Wi-Fi scale to log user’s weight. Calendar data and location was automatically captured and user could manually log food intake each day. To give users more flexibility they could choose things such as the time and frequency at which they should be asked to log their food intake; and the notification was non-interrupting as it would not ring or vibrate but instead just put a small icon in the notification tray.

The widget for inputting food intake is over-simplified; it consists of 3 questions, “How much did you eat today?”, “How healthy was the food you ate?” and “Did you prepare most of your food at home, or did you eat out?”. The user provided answers to this by means of a slide bar. Therefore the application is simplified at the cost of accuracy.

Their findings were quite positive and showed a 5 times increase in self-logging with 55-65% users consistently logging throughout the course of the study (one month). The user feedback was quite affirmative and users in general liked the polite reminders.

In conclusion the paper shows that users are more likely to self-log if Mobile Phone notifications are not disruptive and time and frequency of notifications was user configurable.