Why don't you ask him to come with your family on a camping trip?
There is no direct object to come in this sentence; come is an intransitive verb, meaning it takes no object. Both of the constituents which follow come are preposition phrases (PP): PP follow no set order (and may stack indefinitely), so both the original sentence and your rewrite are acceptable.
Ordinarily come is followed by at least one locative phrase, either a full PP or an intransitive preposition or a term like away or home† which can play the same role as a preposition phrase. This expresses the source, path or goal of the motion; it may be regarded as an 'obligatory' complement, though it may be omitted if its sense is recoverable from context.
A: I'm going down to the bar for a beer.
B: I'll come ∅, too.
In your sentence on a camping trip fills this complementary role.
The other preposition phrase, with your family is not obligatory. In the original sentence it must be understood as an adjunct modifying the VP headed by come, but when it's moved to the right of on a camping trip it becomes ambiguous: it may be understood to modify either the VP or a camping trip.
† CGEL in fact regards home as an intransitive preposition; I think this is stretching a point, and home should rather be regarded as a 'fused-head' PP, in which the head preposition has fused into its object.