Monday, March 12, 2012

Speech Recognition By Java :: A sphinx4 implementation

Speech recognition is one of the challenging areas in computer science, a lot of pattern recognition methodology tried to resolve a good way and higher percentage of recognition. I tried lots of technology. 
VB.net, C#.net etc.  But ultimately found Java the most simpler and easy way. 

One of the best ways to be use is Hidden Markov Model :





"The process of speech recognition is to find the best possible sequence of words (or units) that will fit the given input speech. It is a search problem, and in the case of HMM-based recognizers, a graph search problem. The graph represents all possible sequences of phonemes in the entire language of the task under consideration. The graph is typically composed of the HMMs of sound units concatenated in a guided manner, as specified by the grammar of the task."

There are a lot of Java Based Speech Recognition Engines, you can find them here:

In this post I will select one of the famous open source Java speech recognition software that I used in my final year project.

1) Download the software:

Here is the download link:

2) Create new Java project:

Add to class-path 3 jars from the lib folder: jsapi.jar (if not exist jsapi.exe will extract it on windows or jsapi.sh will extract it) ,sphinx4.jar (APIs Jar),WSJ_8gau_13dCep_16k_40mel_130Hz_6800Hz.jar (this is the jar containing the speech)
3) Create a new class to write the logic on it: e.g. ListenToMe.java
Let the package name dipayan_java

4) Add the file dict.gram to the same package: dipayan_java
It should looks like:
#JSGF V1.0;
grammar hello; 
public = (Good morning | Hello | Hi | Welcome) ( Dipayan | Paul | Philip | Rita | Will );

The structure is to write the alternative words between () and separated by |.

The grammar specs in this URL:
A quick tutorial you can find in my blog !

 5) Add the config file myconfig.xml to the same package: dipayan_java
You can get the file structure from the jar in bin folder HelloWorld.jar in the location "HelloWorld.jar\edu\cmu\sphinx\demo\helloworld\helloworld.config.xml"

You need to edit the following entry:
<component name="jsgfGrammar" type="edu.cmu.sphinx.jsgf.JSGFGrammar">
<property name="dictionary" value="dictionary"/>
<property name="grammarLocation"
value="resource:/dipayan_java/"/>
<property name="grammarName" value="dict"/>
<property name="logMath" value="logMath"/>
</component>

6) Write the following code in the created class:

import edu.cmu.sphinx.frontend.util.Microphone;
import edu.cmu.sphinx.recognizer.Recognizer;
import edu.cmu.sphinx.result.Result;
import edu.cmu.sphinx.util.props.ConfigurationManager;

public static void main(String args[]){
ConfigurationManager cm = new ConfigurationManager(ListenToMe.class.getResource("myconfig.xml"));
Recognizer recognizer = (Recognizer)cm.lookup("recognizer");
recognizer.allocate();
Microphone microphone = (Microphone)cm.lookup("microphone");
if(!microphone.startRecording()){
System.out.println("Cannot start microphone.");
recognizer.deallocate();
System.exit(1);
}
System.out.println("Say: (Good morning | Hello | Hi | Welcome) (Dipayan | Paul | Philip | Rita | Will )");
do {
System.out.println("Start speaking. Press Ctrl-C to quit.\n");
Result result = recognizer.recognize();
if(result != null) {
String resultText = result.getBestFinalResultNoFiller();
System.out.println((new StringBuilder()).append("You said: ").append(resultText).append('\n').toString());
} else {
System.out.println("I can't hear what you said.\n");
}
} while(true);
}


For more details about the usage you can refer to this URL:



NOTE :: I did this project on ecllipse . If you want to do it on the same, while running the program,you will find a OutOfMemory exception. This is because , the available Virtual Memory is not enough for Sphinx4.

The thing you have to do, is just increase the heap memory of ecllipse . Go to Virtual memory arguement ,  type    " -Xmx256m  "


Thanks :)
For any query feel free to mail me @dev.dipayan16@gmail.com








7 comments:

Anonymous said...

Awesome try brother :) Really a tough implementation in CS world .

Anonymous said...

hiii these are my error plz help me out

Exception in thread "main" java.lang.RuntimeException: Allocation of search manager resources failed
at edu.cmu.sphinx.decoder.search.SimpleBreadthFirstSearchManager.allocate(SimpleBreadthFirstSearchManager.java:650)
at edu.cmu.sphinx.decoder.AbstractDecoder.allocate(AbstractDecoder.java:87)
at edu.cmu.sphinx.recognizer.Recognizer.allocate(Recognizer.java:168)
at pkg.ListenToMe.main(ListenToMe.java:18)
Caused by: java.io.IOException: edu.cmu.sphinx.jsgf.JSGFGrammarParseException
at edu.cmu.sphinx.jsgf.JSGFGrammar.createGrammar(JSGFGrammar.java:304)
at edu.cmu.sphinx.linguist.language.grammar.Grammar.allocate(Grammar.java:116)
at edu.cmu.sphinx.linguist.flat.FlatLinguist.allocate(FlatLinguist.java:300)
at edu.cmu.sphinx.decoder.search.SimpleBreadthFirstSearchManager.allocate(SimpleBreadthFirstSearchManager.java:646)
... 3 more
Caused by: edu.cmu.sphinx.jsgf.JSGFGrammarParseException
at edu.cmu.sphinx.jsgf.parser.JSGFParser.newGrammarFromJSGF(JSGFParser.java:132)
at edu.cmu.sphinx.jsgf.parser.JSGFParser.newGrammarFromJSGF(JSGFParser.java:241)
at edu.cmu.sphinx.jsgf.JSGFGrammar.loadNamedGrammar(JSGFGrammar.java:697)
at edu.cmu.sphinx.jsgf.JSGFGrammar.commitChanges(JSGFGrammar.java:613)
at edu.cmu.sphinx.jsgf.JSGFGrammar.createGrammar(JSGFGrammar.java:300)
... 6 more

Dipayan Dev said...
This comment has been removed by the author.
Dipayan Dev said...

Where did you place all the .jar files?

Unknown said...

I have the same problem regarding jar files. i placed the 3 mentioned jars (i.e. jsapi, sphinx4 and wsj) in a folder called lib where i have all the other jar files currently used by my system. Here's my structure in general:

Progect Name
src
- package 1 including class listenToMe and files config.xml, dict.gram
JRE
ReferencedLibraries
data
lib
- containing jsapi, sphinx4 and wsj together with other jars required by the system.

I tried copying the 3 jars to my workspace lib folder and adding them from there and I also tried referencing them from the sphinx folder I downloaded but same error is given.

Hope you can help!
Thanks :)

Unknown said...

Never mind this comment...the problem was in the grammer.
public = (Good morning | ... seems to be incorrect. You should add or any other word after public example:

public = (Good morning | ...

Anonymous said...

you have to change the grammar file by :
#JSGF V1.0;
grammar hello;
public = (Good morning | Hello | Hi | Welcome) ( Osama | Paul | Philip | Rita | Will );