Neither sentence makes sense because both suggest that Nadia is one of her own friends.
"Of all of Nadia's friends..." or "among Nadia's friends" indicates a group of people that don't include Nadia because she is not one of her own friends. So both sentences are nonsense. They are equivalent to saying:
Among the women I know, Mark is the tallest. (Mark is a man)
Nadia is a member of her group of friends, so correct and natural ways of expressing the idea about Nadia's math skills would be:
- Nadia is the best at math among her group of friends
- Within Nadia's group of friends, she's the best at math