Most widespread languages show a variety of dialects: In Germany there is High and Low German, not to mention distinct Swiss and Austrian dialects. French has significant variation between the North and South; Brittany, Alsace and Quebec also have distinct regional dialects
English is no different in this regard. There are significant differences between the English spoken in London, in Manchester and in Glasgow, yet all could claim to be "British English" (d'ye ken). There are also class differences.
Similarly, in North America, there are many dialects: The language of the West coast, the South, New England are all clearly different. The dialects in Canada (there are several) are generally closer to other American dialects than to British dialects.
Australian English is split mostly by class, into a "broad", "general" and "cultivated" accent, though they tend to be closer to British English.
Books tend to be split into American and British because there are differences in spelling. They might make some small changes in vocabulary, but almost no changes in grammar are needed.
However, these difference are at the level of dialect. Speaker of southern English accents have no trouble understanding American speakers. As a learner, if you are a competent speaker of American English, you will have no difficulty being understood by Britons, and will not find it so difficult to adapt to a new accent if you need to.