He was going at a rate of 50 miles per hour.
OR
He was going at the rate of 50 miles per hour.
Which one is the most appropriate grammatically speaking and why?
He was going at a rate of 50 miles per hour.
OR
He was going at the rate of 50 miles per hour.
Which one is the most appropriate grammatically speaking and why?
They are both correct. Generally speaking, which is appropriate depends on the context.
In many contexts, including those you quote, you can use either. The distinction is a subtle one.
For example, it sounds more natural to say:
This device works best at a rate of between 100 and 200 rpm.
Because it can be any rate within that range. But you could use either article in:
This device works best at a/the rate of 100 rpm.
The following example requires the because it refers to a specific rate and context:
The rate at which he was travelling posed a danger to other traffic.
In short, you simply have to decide which article fits your sentence better.