Semantic language model is a technique that utilizes the semantic structure of an utterance to better rank the likelihood of words compos-ing the sentence. When used in a conversa-tional system, one can dynamically integrate the dialog state and domain semantics into the semantic language model to better guide the speech recognizer executing the decoding process. We describe one such application that employs semantic language model to cope with spontaneous speech in a robust manner. The semantic language model, though can be manually crafted without data, can benefit significantly from data driven machine learning techniques. An example based approach is also described here to demonstrate a viable approach.