This thesis focuses on improving medical image analysis by developing advanced AI techniques using various medical data. It starts by identifying the main challenges in analyzing medical images including label scarcity, variability, and modality incompleteness. For each challenge, specific solutions are then proposed to address the issues. The thesis finally highlights the key contributions and discusses future directions for this research. Ultimately, the goal is to help create large-scale medical AI models that can be used effectively in real-world clinical applications.