FamilyNotes is really a Home windows 10 Universal Home windows Platform (UWP) application that implements an organization noticeboard. The aim of this app developer ended up being to showcase the different Home windows 10 input and interaction features which allow an individual and individualized computing experience.
This is actually the third of three blogs that describe the characteristics, design and technical challenges we faced once we developed the application.
Our first publish, FamilyNotes: Presenting a Home windows UWP sample using ink speech and face recognition, introduced the application and described the general design and architecture.
The 2nd publish, FamilyNotes: While using camera to identify a person, covered the impressive facial recognition features supplied by Microsoft Cognitive Services.
Within this publish, we’ll cover Cortana voice instructions, speech recognition and inking.
The Visual Studio task for FamilyNotes obtainable from GitHub at https://github.com/Microsoft/Home windows-appsample-familynotes.
We attempted to create FamilyNotes an example using speech inside a UWP app developer, and you will find a double edged sword for this. The very first is to help make the application react to instructions, and the second reason is to permit the consumer to dictate content in to the application. For part one, we used Cortana.
Cortana may be the Home windows digital assistant with functionality and expertise that may be extended through integration together with your application. This could ensure both you and your application would be the experts that Cortana turns to when customers require info or services that Cortana can’t provide natively.
Home windows 10 enabled Cortana to assist customers across various Home windows products including mobile, desktop and team SKUs. Due to this distinctively broad device support, we felt the FamilyNotes application, using its shared and collaborative encounters across products, would be a prime candidate for integration with Cortana.
Despite the fact that FamilyNotes is meant like a full-screen, exclusive mode kiosk application on the tablet device, we expected there to become occasions once the application wasn’t running like a user contacted. Of these instances, we thought about being able start the application using a deep link using either the Cortana interface or, when enabled with a user, the system’s ability to stay in constant listening mode and to do this if this listens to the saying “Hey Cortana.”
Stretching Cortana together with your application requires you to definitely implement voice instructions.
Note: Despite the fact that “voice” instructions suggests support for speech recognition only, additionally they support text instructions through keyboard input. Actually, any voice instructions you define are supported equally by speech and text. It is really an important point when talking about “more personal computing” and “natural UI,” as every tool and input type should be considered equally when creating and building an application. Who knows what type of products, capabilities or preferences that customers will provide.
But seriously, speech rules, text drools.
Voice instructions are defined inside a Voice Command Definition (VCD) file, that is a XML file explained the Voice Command Definition (VCD) elements and characteristics v1.2 schema. This file is put into any project and it is incorporated within the application package whenever your application is installed.