The Longman Dictionary seems to suggest that on the fringes can be used either literally or figuratively, while on the fringe refers to something actually physically away from the center.
on the fringes (of something)
a) not completely belonging to or accepted by a group of people who share the same job, activities etc
a small group on the fringes of the art world
b) (also on the fringe) at the part of something that is farthest from the centre SYN on the edge of something
Nina remained on the fringe of the crowd.
The Free Dictionary, on the other hand, seems to only have an entry for on the fringe and defines it both in literal and figurative usages.
on the fringe
- Lit. at the outer boundary or edge of something. He doesn't live in the city, just on the fringe.
- Fig. at the extremes of something, typically political thought. He is way out. His political ideas are really on the fringe.
Any differences? Or is it something that has not reached an orthographic consensus?