The issue causing you confusion appears to revolve around the interpretation of the interrogative pronoun what in the prompt "Help what." You are correct that its referent doesn't necessarily have to be a noun - but it does have to be something that functions as a noun. Sometimes prepositional phrases can function as nouns: an example would be
By the creek is my favorite picnic spot.
In the sentence above, the prepositional phrase by the creek functions as a noun and fills the role of subject of the sentence.
Recall that the definition of a noun is a person, place, thing or idea. When your instructor or textbook says that a grammatical object fills the role of what or whom, that means that the pronoun whom stands for a person, while what stands for a place, thing or idea. So, if you're trying to find the direct object of the verb "help" by answering the question "Help whom or what," you will be looking for something that fits into one of these categories (like the phrase "By the creek" in the example sentence, which refers to a place).
So, a sentence where a prepositional phrase is the direct object could look like this:
Q: What is your favorite picnic spot? A: I like by the creek best!
There, "by the creek" is the DO of "like."
So, why can't "with your housework" be a prepositional-phrase direct object?
This is because it answers a different kind of "what". Instead of what thing, the prepositional phrase "with your housework" can only answer the question "in what way."
In what way does Sally help you? She helps with your housework.
Thus, you can see that the role of "with your housework" (even when it comes directly after "help," without the word "you" occurring anywhere in the sentence) is not that of a noun, but that of an adverb. Something that functions as an adverb cannot be a Direct Object, because it's a modifier and not a substantive. (Substantive = anything that functions as a noun.)
So, since "with your housework" is an adverbial prepositional phrase in your quoted sentence, that only leaves "you" as the candidate for the direct object of "help."
Another way of looking at it is by analogy: it's easy to visualize what it means to help a person with something. If Sally helps you, you are receiving assistance from her (with something). But if Sally helped with your housework, is with your housework some entity which is receiving assistance with something? If so, could you, Sally, and with your housework all go on a picnic together?
The reason that interpretation doesn't work is that, since with your housework is adverbial and not substantive, even if the sentence were just
Sally helped with your housework
The phrase would still not be a direct object. Instead, the verb "help" would be described as functioning intransitively (meaning it doesn't have a direct object) and would be modified by the same adverbial prepositional phrase. (Although the [semantically] implied object of "help" would still be "you," since it isn't present in the sentence, from a grammatical perspective "help" wouldn't have an object at all.)