Block as a measure of distance is largely an American phenomenon, a result of America's famous grid-planned cities. A city block in this context is a pre-planned unit of buildings, normally rectangular, surrounded by roads on every side.
This is the US city of Phoenix. It's built in the desert, so it has an extremely orderly structure. You can see here the arrangement of the city in square or rectangular blocks - it's units like these that are the source of the informal distance measure.
The size of a block will vary from city to city, or between parts of a single city. It often depends on what type of structures the block contains (a block containing individual houses will be narrower than one holding skyscrapers). An average distance for a block could be considered something like 100 metres or so.
Other parts of the English speaking world do not have any equivalent of these city blocks. For instance, this is Birmingham, a relatively new city in England:
As for whether a non-native should use informal terms like these, it certainly pays to be careful. Make sure you thoroughly understand what the word you're using means in the context and culture you're in. As you've seen, an American would understand what you meant by a "block", while a Brit would give you a strange look. But don't stress it, either - most people are pretty decent about giving non-native speakers the benefit of the doubt.