New electroacoustic products, like mobile phones, tablets and smart speakers, continue to push the design limitations. The newest smartphone needs to fit in ever more impressive features to succeed in a crowded marketplace; and meet international voice capture and reproduction telephone expectations. And whether consumers want open-backed headphones or flat frequency response speakers, as promised by the brand, companies need to deliver the quality of sound which customers expect. An examination of the sound produced and recorded during R&D is important to deliver quality sound. For accurate sound reproduction a multitude of parameters are crucial.
A Short History…
Precise sound reproduction is a very new topic, because it is heavily dependent on other enabling technologies that are also fresh. Of these fresh technologies, the electronic amplification is the most noticeable. Before that, opportunities were small, but before amplification many of the basics that we consider modern had been in place. Methods for the perceptual assessment of audio signals for a wide range of application areas have been developed during the last decades. They authorize assessment of time variant efficiency, nonlinear systems. As such, as used in the telecommunication and music industries, they are important for quality evaluation of low bit rate sound and audio coding.
Smart Home Devices…
Voice-operated smart home devices took households by storm. Universal products which provide the user with a variety of different features. They are intended to capture the user’s voice, provide sufficient speech recognition in natural language and facilitate the user’s requested actions.
Audio or Video playback is also possible with smart home devices. Both the built-in loudspeaker or wireless connectivity to other devices allow all kinds of music and video to be played. Also, in various types of environments the audio playback feature needs to work up to the expectations of the user.
As a result, a number of tests and simulations are required to properly evaluate and validate the output of devices. There are methodologies for performing these types of tests in a highly automated and efficient way. These experiments include simulating different room types (space simulation), different types of background noise conditions, as well as different types of talkers and distracting voices at various positions.
Testing Smart Devices…
Audio equipment evaluation is the assessment of the sound quality by objective and/or subjective means. The findings of those studies are published in newspapers, magazines, blogs, and other outlets. Those who test and assess equipment can be divided into two groups; “Objectivists,” who claim that all perceptible variations in audio equipment can be objectively explained by calculation and double-blind listening tests; and the “subjectivists” who claim that the human ear can hear the details and variations that cannot be calculated directly.
Smart device testing needs to cover a range of different technologies which have been used separately for different device types in the past. Many features are integrated inside smart devices. The testing of these devices includes sophisticated, versatile and fully automated testing procedures that assist in evaluating the devices under all conditions.
The main areas of testing are:
- Speech Recognition Testing
Speech recognition analysis of devices is specifically focused towards the efficiency of the smart home system rather than the common speech recognition engine available at the back end. And usually a relatively small collection of commands, sentences, etc. are used in cohesion with the network-based speech recognition engine to verify the performance of the smart system.
- Speech Communication Testing
The methods for evaluating the quality of speech communication in general are well known. There are proper standards in ESTI (European Telecommunications Standards Institute). These standards are ‘user-centric’ and are not influenced so much by the technical limitations of product development. All related performance parameters have to be checked, primarily listing speech quality (when background noise is present) and echo rate (in the presence of ambient reverberation) with varying user positions.
- Audio Quality Testing
Compared with speech recognition and speech communication tests, audio quality assessment is probably the easiest challenge throughout smart home devices testing. The test signals can be sent to the device for verifying audio measurements such as:
- Signal-to-Noise Ratio
- Frequency response
- Rub & Buzz