Getting an all-optical AI to handle non-linear math

Instead of sensing photons and processing the results, why not process the photons?