The thirsty children drank up all the water.
Is it all right to use define article the with water as in above sentence ? I feel it would have been better without it as water here is used in general sense.
Not only is it OK to use the article the, you must use it here. Why? Because we are discussing a specific quantity or amount of water that the children drank.
Saying "The children drank up all water" actually doesn't make very much sense in English, because it's unclear. All water what? All water in existence?
Yes, you can say both "The children drank water" and "The children drank the water", but there is a slight difference in meaning:
But in your example, if they drank up all of it, it must have been all of some specific, known quantity.