We’ll break out the low-pitched parts, the next-lowest-pitched-parts, and so on.

To make this data easier for a neural network to process, we are going to break apart this complex sound wave into it’s component parts. But taken all together, these different frequencies mix together to make up the complex sound of human speech. There’s some low sounds, some mid-range sounds, and even some high-pitched sounds sprinkled in. But even this short recording is a complex mish-mash of different frequencies of sound. This recording is only 1/50th of a second long. Here’s our first 20 milliseconds of audio (i.e., our first 320 samples): Let’s start by grouping our sampled audio into 20-millisecond-long chunks. Instead, we can make the problem easier by doing some pre-processing on the audio data. But trying to recognize speech patterns by processing these samples directly is difficult. We could feed these numbers right into a neural network. We now have an array of numbers with each number representing the sound wave’s amplitude at 1/16,000th of a second intervals. I mention this only because nearly everyone gets this wrong and assumes that using higher sampling rates always leads to better audio quality. In Part 3, we learned how to take an image and treat it as an array of numbers so that we can feed directly into a neural network for image recognition:Ĭan digital samples perfectly recreate the original analog sound wave? What about those gaps?īut thanks to the Nyquist theorem, we know that we can use math to perfectly reconstruct the original sound wave from the spaced-out samples - as long as we sample at least twice as fast as the highest frequency we want to record. The first step in speech recognition is obvious - we need to feed sound waves into a computer. Let’s see how it works! Turning Sounds into Bits To work around this, we have to use some special tricks and extra precessing in addition to a deep neural network. Both both sound files should be recognized as exactly the same text - “hello!” Automatically aligning audio files of various lengths to a fixed-length piece of text turns out to be pretty hard. One person might say “hello!” very quickly and another person might say “heeeelllllllllllllooooo!” very slowly, producing a much longer sound file with much more data. The big problem is that speech varies in speed.

That’s the holy grail of speech recognition with deep learning, but we aren’t quite there yet (at least at the time that I wrote this - I bet that we will be in a couple of years).

Fun text to voice