Mathematical physics is the study of physics using mathematics. It might be argued that all of theoretical physics is mathematical physics, but in practice, most physics is done on a more intuitive/approximate or even questionable level.
Mathematical physics tries to study physics on a more abstract and rigorous level than typical and respect the strict rules of rigor that are usual in mathematics, while theoretical physics tries to maintain a closer contact with experimental physics than with mathematics. Because of the high requirements, mathematical physicists often deal with questions that theoretical physicists have considered to be solved for decades. However, the mathematical physicists can sometimes (but rarely) show that the solution was incorrect.
Quantum mechanics cannot be understood without a good knowledge of mathematics. It is not surprising then that its developed version under the name of quantum field theory is the most abstract, mathematically based and backward-influential to mathematics area of physical sciences.
