Traffic signal control (TSC) is an essential and effective approach to reduce traffic delay. Reinforcement Learning (RL) provides a new way of designing TSC systems that allow agents to learn optimal control policy through interacting with the environment without models. However, developing an RL-based TSC algorithm for a large-scale network with practical constraints is still an open question. This thesis develops multi-agent RL algorithms with fast learning speed, strong robustness, and scalability for large-scale networks. This thesis provides insights for transport engineers to develop efficient, scalable, and robust RL methods for networked TSC systems in a real traffic environment.