We describe our experience in developing ConciergeBot, an industrial strength question-answering bot for hotels. The bot automatically suggests answers for information-seeking questions over an input knowledge base of facts about the hotel and its amenities. We demonstrate how ConciergeBot handles unique challenges that arise in our setting. More specifically, we show how our system trains effective models with limited training data, how it can be deployed in different hotels with almost no hotel-specific tuning, and how it manages heterogeneity in questions and data. Our experiments validate that ConciergeBot achieves high precision (78%) and good recall (71%) with as few as 1,300 questions for training purposes.