Consider:
A tablecloth with a wine stain...
A tablecloth with a wine stain on it...
In both sentences, with is understood to mean "having". As there are not many ways a tablecloth can be "with" a wine stain, "on it" is unnecessary; it simply reinforces the idea.
In the same way, a picture with a caption and a picture with a caption on it mean much the same thing, with the possible minor exception that someone might think "under" or "beneath" or "above" a more suitable preposition than "on", since captions usually do not appear directly "on" the picture. But "on" would be understood to mean "applied to it".