Milk is a complex emulsion of fat and water with proteins (such as caseins and whey), vitamins, minerals and lactose dissolved within. The purpose of this study is to automatically distinguish different dairy residues on substrates commonly used in the food industry using hyperspectral imaging. Fourier transform infrared (FT-IR) and Raman hyperspectral imaging were compared as candidate techniques to achieve this goal. Aluminium and stainless-steel, types 304-2B and 316-2B, were chosen as surfaces due to their widespread use in food production. Spectra of dried samples of whole, skimmed, protein, butter milk and butter were compared. The spectroscopic information collected was not only affected by the chemical signal of the milk composition, but also by surface signals, evident as baseline and multiplicative effects. In addition, the combination of the spectral information with spatial information can improve data interpretation in terms of characterising spatial variability of the selected surfaces.