I recognize that the summation of the comments is "Sometimes things are like that because things are just like that. Sometimes." Language is created, and we fall into phrasal habits.
But I think there is a distinction that can be made, at least in the case of "coffee with milk." I think we can agree that "coffee and milk" would be odd. Is there any reason besides idiomatic habit?
I would argue that the constructions that use "and" tend to be either a pairing of equal but distinct things or somewhat the opposite: a pairing that becomes phrasally established as a single concept. "Gin and tonic," "peanut butter and jelly," "fish and chips": these pairings are recognized as coherent entities, and outrepresent their component parts. And (depending how you mix your drinks), they are unions of more or less equal parts. "Coffee with milk" is (with further requisite humor about drink mixing) more of a single main item plus a "garnish." And although café con leche (and for that matter au lait) is rather phrasally established, it's still seen as a variation on a theme; it's a "type of coffee" or a treatment of coffee. "Chicken with black bean sauce" continues the trend.
I'm at a loss to come up with a "___ with ___" construction that is either a phrasal pairing or a union of equals. (At least in English—arroz con pollo suggests itself.) The concept holds true outside of food, as well. "Simon and Garfunkel" are a team, but "Paul Simon with the Boston Pops" makes it clear who is the headliner.