Nonlinear optical frequency conversion, observed more than half a century ago, is a corner stone in modern applications of nonlinear and quantum optics. It is well known that frequency conversion processes are constrained by conservation laws, such as momentum conservation that requires phase matching conditions for e cient conversion. However, conservation laws alone could not fully capture the features of nonlinear frequency conversion. Here it is shown that topology can provide additional constraints in nonlinear multi-frequency conversion processes. Unlike conservation laws, a topological constraint concerns with the conserved properties under continuous deformation, and can be regarded as a new indispensable degree of freedom to describe multi-frequency processes. We illustrate such a paradigm by considering sum frequency generation under a multi-frequency pump wave, showing that, akin topological phases in topological insulators, topological phase transitions can be observed in the frequency conversion process both at classical and quantum level.