Serafina: Hi, I’m Serafina and this is The HealthMakers Show. I’m here with my guest, Dr. Paulo Pinheiro to discuss his company, HOOBOX. Paulo, thank you so much for being here today.

Paulo: Thanks for being here, Serafina.

Serafina: So, tell me about HOOBOX.

Paulo: HOOBOX is a startup company that uses artificial intelligence and machine learning for improving the human well-being. We start with just solving the problem of mobility and autonomy of quadriplegics and people diagnosed with ALS, or anyone who cannot control their wheelchair with their hands.

Serafina: So, you’re talking about controlling a motorized wheelchair with just your facial expressions?

Paulo: We had to develop almost the technology from scratch because we didn’t have a technology that was able to capture facial expressions regardless of the light condition. We didn’t have the technology that was able to capture facial expressions even if I mount a camera on the side of the face of the user, so we had to create this technology from scratch. The Wheelie now is so precise in capturing facial expressions that we are also able to capture some human behaviors.

Serafina: What happens about non-voluntary facial expressions, like a sneeze, or a cough?

Paulo: That’s a great question. We have an algorithm to detect it. So, we can detect that you are going to sneeze or cough before you sneeze or cough. So, we can disable the interface and then you can enable the interface using, for example, a combination of expressions. Let’s say you are going to sneeze. We disable the interface and then you can use a raised eyebrow for three seconds, like this: one, two, three, to enable the interface again.  It’s the same to talk to someone. If you want to talk to someone, you can one, two, three, and you disable the interface, and then you can talk, and then you can enable the interface after that.

Serafina: Tell us what triggered the idea.

Paulo: Two years ago, I was at the airport, I was just waiting for my flight, and I saw this girl in the wheelchair and she couldn’t move her legs or her hands. Her father was helping her to control the wheelchair, but she had a great smile, and you know, when you see someone smiling and you cannot stop thinking about that smile? I thought, ok, I don’t know how to do this but, definitely, I could try to translate that smile into commands to control a wheelchair.  This was the insight for the Wheelie.

Serafina: Once this idea came to mind, how did you get the ball rolling?

Paulo: The first thing I did was got my flight back and ran an Idea Validation. So, I set up a meeting with some colleagues that I had luck to have met during my Ph.D. research, quadriplegic people and people diagnosed with ALS. I told them about the idea, ‘Ok, so, I think I have a solution to use facial expressions to drive a wheelchair just using a camera and our technology.  Would you like to use this technology? There is no body sensor, so you don’t have to wear some kind of body sensor to control the wheelchair.’ We got excellent feedback, so we started to design the technology, and a few weeks later I was giving up my postdoc position that I was about to take in Sweden, along with my brother as my co-founder, we founded HOOBOX to develop this technology. So, we created the Wheelie as the first computer program ever capable of translating facial expressions, like a smile, a kiss, into commands to control a wheelchair just using a camera mounted on the wheelchair.

Serafina: So, how long did that research and development take you to build out that software, pitch the application, and then build your team? Tell me everything.

Paulo: Yeah, it was almost two years. We founded HOOBOX in 2016, we have seven people right now, it’s me and my co-founder, and we have a team that are very, very expert in machine learning artificial intelligence.  We take from the idea to the prototype in almost two years. Right now, we have more than 50 early adopters here in the United States that are trying the Wheelie on a daily basis.

Serafina: So, what are your plans for commercialization?

Paulo: Well, we’re finally dating the business model, and so far, it works very well with the kind of subscription, the monthly subscription. So, we send a kit for free and then we charge a monthly subscription. We are not selling the wheelchair, we are not selling a robotic or a special wheelchair. We are selling just a kit that we call Wheelie 7, because it only takes seven minutes to install on any motorized wheelchair available on the market. You do not have to buy a new wheelchair, just our kit, and you can install it in seven minutes. Next month we are going to release the Wheelie 6, because it only takes six minutes to install it on any motorized wheelchair available on the market. So, this is our plan to go to the market.

Serafina: So, what does this technology look like?

Paulo: You receive a kit at your home. It’s a box like this, and you have the camera that you can mount on the wheelchair; you have the on-board computer, that’s a very, very small computer that you put underneath the wheelchair; and there is a robot, a little robot that you can place over the joystick of the wheelchair. So, we have the camera here, and when you perform some facial expressions, like say, a kiss to move forward, when I kiss, the command is sent to this little robot and the little robot controls the joystick to move forward. So, this is how it works.

Serafina: How customizable is this technology? Can the user program what facial expressions they want to use to direct movement?

Paulo: Yeah, sure. So far, we have fourteen facial expressions that the users can pick up to control the wheelchair.

Serafina: What are some examples of those?

Paulo: A smile, kiss, a raised eyebrow, tongue out, and even a half smile for users who have suffered a stroke we have these kind of different facial expressions. You have to pick up five facial expressions: one to move forward, backward, turn left, turn right, and to stop. We have fourteen facial expressions in our catalog.

Serafina: So, there’s some variety there.

Paulo: Yeah, you can pick up the most comfortable for you to drive the wheelchair.

Serafina: So, are there commands aside from the movement of the wheelchair that the technology is adaptable for?

Paulo: Yes. So far, the users can use the Wheelie to turn the lights on and off. We can do this with the Philips lights, and also to call some family members. So, we have those two extra options for those users.

Serafina: So, what are the future applications that you guys are looking into?

Paulo: We are a health tech company, so we are using this technology to solve problems in this segment. We capture facial expressions with a high precision, and then we start to capture human behaviors. We are taking this technology to intensive care unit beds to monitor critical patients over there, and so far we are able to detect level of pain, sedation, agitation levels and also deliriums.  Also, of course, we can license our technology. We have several clients in other segments like retail transportation and even agrobusiness in Brazil.

Serafina: So, is collecting all this data part of your long-term business model?

Paulo: Yes, for sure.  We are capturing the human behaviors, how they behave on a daily basis, how they interact with the other people in the electronic device, and we are going to explore this data sometime in the future.

Serafina: Share with us, ideally, where the company will be, say, five years from today.

Paulo: We think in the future, many medical devices and other products in the healthcare industry are going to have some kind of facial recognition feature embedded. We want to make HOOBOX the biggest supplier of this technology for those other companies so they can solve their problems.  Of course, we want to solve our problems also. We want to develop our own products. Three months from now we are going to embed this same technology that we put inside the wheelchair, inside an intensive care unit bed, inside a baby monitor, so we can track newborn babies facial expressions and track their sleeping record just using the camera and our technology. So, we want to provide our technology to other companies but also build those kind of products that can help people.

Serafina: Well, thank you so much, Paulo, this has been immensely educational.

To learn more about HealthTech @ JLABS, click here.https://jlabs.jnjinnovation.com/blog/introducing-health-tech-jlabs