A Audio Visualization App That Makes Deafness Irrelevant

There are or as well as thence rattling proficient programmers amid the readers – this could hold upwards a fun exercise. I simply accidentally "chatted" amongst a deaf guy on Twitter as well as decided to pass a few to a greater extent than minutes past times thinking well-nigh or as well as thence high-brow, hi-tech technological ways to assist the deaf people – who may represent, according to or as well as thence counting, upwards to 5% of the Blue Planet population.

It's a huge market. It could hold upwards interesting to do a helpful app that effectively "returns the hearing" past times maximally visualizing the information contained inwards the sound. That app could hold upwards used amongst or as well as thence phones but maybe amongst or as well as thence "Google Glasses" or a similar gadget that the deaf folks could uncovering to a greater extent than helpful than others.




First, the app should endeavour to do phonation recognition - similar if you lot click the microphone icon at Google.com. Second, it should endeavour to a alternative a maximum total of information well-nigh the frequencies represented inwards the acquaint good as well as noise.




That information should hold upwards chop-chop Fourier-analyzed – each divide 2d of the good – as well as the Fourier components should undergo or as well as thence additional processing that extracts the information that is maximally helpful for communication or distingushing thing, as well as that information would hold upwards represented past times or as well as thence image.

The paradigm obtained from the good could contain non simply curves or spectrograms (see a spectrogram of a "song" that looks similar cool images amongst creatures; graph-like pictures for fun sounds; spectrograms of vowels).

Colors could hold upwards everywhere to stand upwards for the divulge of frequency peaks, the ratios of dominant frequencies, as well as lots of other things. The graphical representation should hold upwards helpful to distinguish known vowels as well as consonants, aside from another of import sounds, as well as their frequencies. You could approach the work inwards or as well as thence genuinely sophisticated mode – equally a motorcar learning problem. Some criteria that may maximally distinguish sounds could hold upwards graphed inwards or as well as thence way. Also, a detailed stereoanalysis of the sources' place could hold upwards made as well as influence the pictures.

Music could hold upwards contrary engineered to notes – but mayhap inwards or as well as thence to a greater extent than "biological" mode as well as thence that the programmer who does it doesn't convey to code whatever precise notation. Instead, the users volition spontaneously acquire to recognize what is at nowadays incorporated inwards the sound.

If the good is pictured inwards a sufficiently comprehensive, readable way, isn't it obvious that a (deaf) human existence could acquire to "hear" the good past times his eyes also equally others tin flaming ask heed it amongst their ears? Do you lot concur amongst my intuition that a human existence gets much to a greater extent than information from the eyes than the ears? Compare the good as well as visual information within a video file. I mean value it's simply a interrogation how this should hold upwards done optimally, non whether it's possible at all.

The visualization could honour the superposition regulation to or as well as thence degrees – as well as thence that if 2 people are speaking at the same moment, you lot tin flaming inwards regulation read both of them simultaneously.