As a native speaker of American English, with (reportedly) an above-average vocabulary and functional understanding, but a below-average technical knowledge of prescriptive grammar, "to" sounds more correct.
Additional background: I was born and raised in California, and now live in Oregon. I have some college, with two technical Associate's degrees, and my mother is a published author.
To my "ear", saying the delivery was made "at" the house defines a location that the transaction (handing over of a package, presumably) occurred, but is more vague about the intended destination. (We can assume the location and destination are the same, but it's an assumption.) But saying it was made "to" the house suggests that the specific house - and by extension, its implied residents - were collectively the intended destination of the package. It does leave ambiguous if any specific person has in fact received said delivery.
This may contradict others' interpretations, because even among native speakers language usage varies widely. And it may not follow a strict rule. But in the question's essence of: "which one sounds more correct to a native speaker?", that's my take on it. Both prepositions make assumptions, but the "to" variant sounds better to me.
As is common in both English in general, and in the nature of Native Speakers of any language, a small change to the initial sentence structure would alter my opinion, and make it more "native" in the process. "The mailman dropped a package at the house." The listener still makes an assumption that the house in question is one of principal interest to the speaker - the speaker's own house, or if not the same, perhaps the listener's own house - but using "to" here would feel quite wrong.