"Go from house to house" would be fine:
They still go from house to house, questioning people.
You go from house to house collecting subscriptions.
That was the Christmas tradition- where you go from house to house collecting your presents.
It's true that home can mean a building, just like house. But those homes are not the milkman's ones (someone's home is the house or flat where they live). However, construction companies, for example, might say, "We build homes," not meaning their own homes. But in this case it sounds like homes for somebody, for people, for their customers. You don't need that kind of accent in your sentence about the milkman. So, "from house to house" is just fine.
Let me end by giving a few examples of "home to home:"
Mason Geary was a foster kid who grew up bouncing from home to home. (from one family to another)
We went form home to home and offered people the gospel of John. (again, from family to family)