In the sentence: Is your brother coming with us? Why the verb come is in the gerund format?
Is your brother coming with us?
Here, coming is a present participle. It works with the auxiliary verb be (the present, third person is).
This is clearer if the question is answered:
Yes, my brother is coming (with us).
In your example, although coming is not adjacent to is, it is still working with it.
The present participle just happens to take the same ing form as a gerund.
Consider its use as a past participle:
Did your brother come with us?
It is still a participle, but, because of the tense, does not take the ing form.
A gerund is a noun, not a verb.
Coming can also act as a noun (arguably as part of a compound noun or as an adjective to a noun):
The young artist had a real coming out after people saw her paintings in the gallery.
For most theatres, Avengers: Infinity War is a major coming attraction.
The fact that a verb word ends in ing does not necessarily make it a gerund. You need to determine the context in which it's being used. Is it a present participle verb or is acting as a noun?
The way we present the uses of the present participle is confusing.
In general, in English, the present particple is the bare infinitive with the suffix "ing." The words "being," "doing," "shouting," "lying," etc. are examples are of that standard syntactical construction.
The present participle may be used as part of a verb phrase to indicate progressive aspect, e.g.,
I am driving to a movie
Right now, I am in the process of driving a car from somewhere to somewhere else in order to see a movie
In the immediate future, I shall drive a car from somewhere to somewhere else to see a movie.
A present participle may be used as an adjective
a driving rain
means a rain so forceful that it appears to drive people in a particular direction exactly as wolves drive a herd of deer to exhaustion.
A present participle may be used as a noun to describe the activity denoted by use of the verb
In America, driving a car is a skill usually learned during adolescence.
Linguists apparently find it useful to distinguish between gerunds and other types of derivative of the root of a verb when talking about languages generically (see gerund and gerundive in Latin for example), but distinguishing between one use and the other two uses by giving only one a distinct name is not helpful to those learning English. It is a typical example of forcing a distinction found helpful, perhaps even essential, in one field onto another field in which the distinction is merely confusing. It is a form of academic colonialism.