Corrosion costs roughly 3.4% of GDP, representing US$2.5 trillion worldwide per annum. The energy used to convert ores into alloys is constantly driving corrosion back to the oxidised state. Currently the most common corrosion assessment technique is visual inspection. Limitations to visual inspection include the requirement for expert opinions, subjectivity in this opinion, human error, and potentially hazardous access for inspectors. Our research is focused on using deep learning to automate the detection of corrosion. This presentation focuses on the challenges of deep learning for detecting rust, which has no defined form.