To my knowledge, as an American, First floor always refers to the ground-level in American English. Ground floor and first floor are synonymous.
Example of a simple floor plan; in this case, First floor will always be at ground level:
However... there are times when there are multiple floors at ground level, as when a building is built on a hillside, and this can lead to confusion. If such a building has the floors labeled (as in a stairwell or elevator), typically the lowest level of the building with an official/main entrance is called the First floor, and any floors above that are labeled second, third, etc. Any levels below that are typically called basement, and there may be multiple basement levels, commonly indicated as B1, B2, etc.
In some other cases, again, especially for buildings built on a hillside, with two or more "ground levels" (I'm thinking especially of hotels), there may be a floor between the basement level (or B1) and the first floor. In an elevator, you may see, for instance:
B, ?, 1, 2, 3
Where ? might be GL for Ground Level, R for Reception, M for Main, SL for Street Level, etc.
And naturally, it gets even more complicated if there are more than two floors between different ground-level entrances, requiring more naming creativity:
(Image borrowed without permission from here)