Complex Question Answering, or CQA for short, has gained growing focus in recent years. It tries to give accurate answers for complex questions that need the informative context, like text, pictures, or videos. In this thesis, we study four important questions from data to method, and make significant contributions on (1) gathering more data by trained models; (2) teaching CQA models to do basic math, like addition and subtraction; (3) understanding the meaning of complex questions by breaking it into single steps; (4) learning from different types of modalities. We're tackling these aspects to make CQA models even better.