Talk to your Umbraco website with Google Home

On October 24th Google launched Google Home in Dutch. From then on I can talk in my mother tongue with this smart speaker in my own living room. At Perplex we believe that voice will play a big role in our daily life on the internet in the future. We started experimenting over a year ago to prepare for this (r)evolution. What can we do, and what do we want to do with Google Home? How can we use Google Home in our daily work? Can Google Home help us work more efficiently? We will present our proof-of-concept Umbraco package and our Google Home app in this blog for our favourite CMS: Umbraco.

What does the package do?

With this package and Google Home, you can edit your content in Umbraco using speech. Get rid of that keyboard because with this package you won't be needing it anymore. From now on you only need Google Home and your own voice. You can add and edit voice-activated pages in your Umbraco installation, so you have time to do other things!

Demo?

You definitely want to see this package in action because you probably won't believe us until you see it for yourself. First things first, you need a Google Home. Installed your smart speaker? Let's start talking to our Umbraco app.

Unfortunately our app is pending approval by Google. If you want to test our app we will need to add your Google account as a beta tester, and after you can start using it. If you're interested we're more than willing to add you as a tester. Please send me an email on jeffrey@perplex.nl

A video to demonstrate how easy it will be in the future to add or edit content on your Umbraco website.

Let's get started

Authorisation-step: If want to start using our app, you'll have to install the package in your website first. If you've installed it, you can connect the app to your website.

But of course you could 'borrow' one of our testwebsites as well. We have setup a testing environment on https://talkto.perplex.eu where our package is already installed and you can authenticate yourself in the Google Home app with the username perplexrocks@perplex.nl and the password perplexrocks.

Editing content

If you were successful, let's start editing some content! In the current website, we have three overview pages: FAQ Items, news items, and animals. You can talk to Google Home and ask to add one of these items.

Speak! 

Sentences that you could say to Google Home are for example: 

  • ''I want to add a new animal''
  • ''I would like to add a new FAQ''
  • ''Please add a new newsitem to my site''  

The app will try make a match based on the documenttype. These are the underlined words in the example sentences. All example sentences contain existing document types. Once the app has identified the correct document type Google Home will iterate through all properties of the document type.

For each of these properties, Google Home will ask which value it needs to set, depending on the datatype of the property.

Datatypes

Our package currently supports the following datatypes:

  • Textstring
  • Image
  • Date time
  • Dropdown list
  • Check box

The animal node, for example, will look like this:

Answering additional questions

Google Home will then ask you some follow up questions like :

  • “What is the animal name?”
  • “What is the natural habitat?”
  • “When is it first seen?”

When you've answered these questions our app will save your answers and your conversation is over.

Simulator

The code is written so generically that if we add a 'Car overview' with a document type 'Car' below it, it can also be automatically added via voice without having to change anything in the code. That is pretty neat!

If you do not have a Google Home (yet) and you would like to edit an Umbraco installation with your voice, you can also use this simulator. This simulates the Google Home via a chat interface, but of course, this is less spectacular.

I want this on my own website!

Well, then you are in luck because we created an Umbraco-package that allows adding a voice-app to your website without programming a single line of code.

  • Download the package on Our Umbraco
  • Install the package in your Umbraco website
  • You can now access the new section 'Google Home'. Here you can see all the authenticated accounts. This list is still empty at the moment, but later you will be able to see the various users that are authenticated and their tokens here.
  • Then you can start talking with our app by using the keyphrase "Ok Google, talk to my Umbraco website". The only condition is that the website has to be accessible via the internet. So if you are developing locally (localhost, or another internal domain name) then, unfortunately, Google Home cannot make a connection and will not work.
Perplex - Jeffrey Schoemaker

Some background information

Two years ago I attended the Umbraco Retreat in the weekend before Codegarden. During this weekend I was working on the idea of making Umbraco as accessible as possible. The idea was that you should only be able to manage Umbraco through the use of your keyboard. The goal was that visually impaired persons should be able to maintain their own website without having to put too much effort into this. #inclusiveness

But after having done some tests, I learned that I was taking the wrong approach to solving this problem. Wouldn't it be great if you didn't need a computer to use a website? Why do we force people to use a computer and other tools like keyboards to edit their website? I want to be able to talk to my website. That way (almost) everyone can maintain their website; all you have to do is know what you can do within your website and simply speak.

Besides, most content editors still speak faster then they type. Of course voice technology isn't perfect yet, and some conversations with computers still feel a bit awkward, but that's a matter of time, isn't it?

I totally agree. Do you have more demos?

Yes, we have. Because Umbraco can be fully managed through their services, we were able to 'easily' create this proof-of-concept app. (It wasn't easy actually. It took a trainee more than 5 months, under the guidance of 2 Umbraco MVP’s). But we wanted to take it a step further... An informative website with only text is just boring, so we wanted to also add images to the website. But how do you do that without a computer and only your voice?

Well, that's 'easy'... We pass the converted voice command to Google Image Search, retrieve the first result and add it to the Media library in Umbraco. In that way we can even add images with our voice!

Technology

To convert our voice into actionable requests for Umbraco, we need a piece of software that can do that. The software we use is DialogFlow (formerly this was called api.ai, and is now owned by Google). In DialogFlow we have configured several phrases to ensure that the app can have a conversation with you. DialogFlow calls a webhook on our webserver, the so-called Perplex API Gateway. This looks at which Umbraco website the request should be distributed to. This happens based on the token that we stored during the authorization step.

This token is constantly used by Google Home so now we know which Umbraco website should be addressed. Schematically it would look something like this:

The Perplex Gateway is set up in a generic way so it can handle various structures in different Umbraco websites.

Where do we go from here?

Start by talking to our app via Google Home by using the keyphrase "Ok Google, talk to my Umbraco website" (currently you'll need to be added as a beta tester, because the app is still pending approval by Google. Please send an email to jeffrey@perplex.nl) or use the simulator offered by Google. The result of your voice assignment can then be found in your own website or on our testwebsite (https://talkto.perplex.eu/umbraco/) where you can login with 'perplexrocks@perplex.nl’ and the password ‘perplexrocks’.

This blogpost and our package are a proof-of-concept package and not yet meant to directly manage your website, but it would be possible. The package will be developed further occasionally, but it's a pretty time-consuming activity. If you're interested in helping us improving the package and app, then please get into contact! Or if you have a visual limitation and you want to use our package, please get in contact as well and maywe we can help each other!

Read more

We won prizes at CodeGarden 2018!