In this context, 'down' is "any direction, but I'll arrive where you are".
In short, it means there, where you are.
"I'll be at your establishment early tomorrow" is the full intent.
The rest is 'pseudo-random direction' depending on the height difference, geographical centre, any vague hint of directionality.
Natives use these all the time.
"I'll be [up/round/down/in/about/over]" is relative to where the speaker considers they currently are in relation to where they're going. Uphill or downhill, out to the outskirts of the city or into the centre, indoors or outdoors, upstairs or down.
To the listener, they indicate some element of 'travel' but the absolute direction is often actually irrelevant to the overall meaning.
I was assuming 'first thing' wasn't the issue.
If it is, then 'first thing' implies intent to be there at approximately opening time. It indicates the importance to the speaker of early arrival at that destination.