This probably isn't strictly proper, but back up the road (e.g. "you need to go back up the road" or "it's back up the road") is a very common colloquialism where I grew up (Northwest US) which means exactly what you're asking. With this, one would not need to indicate the direction as WendiKidd's answer specifies, because it implies the direction is in the direction from which you previously came.
It's possible to use it unambiguously in conversation with someone who isn't, or hasn't recently been traveling, but the context of the previous part of the conversation would have set a direction in which use of this phrase would imply the direction is opposite of previously established direction.
Hey, you know where the Wilson Ranch is up Flat Creek right?
Yeah.
Well, just back up the road from there I saw a whole bunch of turkeys the other day.
In this example, the initial direction is established as the direction one would travel on Flat Creed Rd to arrive at the Wilson Ranch from wherever the conversation is taking place. "Just back up the road from there", would indicate the location being referenced is on this side of the Wilson Ranch, probably somewhere between a quarter of a mile to five miles from it.