I recommend your consideration of Jason Bassford's comment. There are a number of verbs that indicate mode of propulsion on travel in a river or stream. And of course your comment about Venice is absurd because Venice is on a lagoon of the sea so there is no upstream or downstream; there is only with the flood or with the ebb.
But regardless of the mode of propulsion or the occupancy of the vessel on a river, there are adverbs indicating direction such as "upstream" or "downstream," "with the current" or "against the current," "toward the source" or "toward the sea," "seaward" or "inland," and probably other contrasting pairs that I cannot now recollect. As for verbs, there are "go," "move," and "travel," which are neutral with regard to direction, mode of propulsion, and occupancy. There are others that imply relative direction, e.g., "slip," "flow," or "race," or, in the other direction, "struggle," "crab," "inch," or "beat." The reason that I say relative is that in tidal reaches what is relevant is not the direction to or from the sea, but the direction of the tide.