Is that any expression in English for not too sweet tea?
Here, we normally say it is as sweet as guava, which means the taste of tea is just half sweet, because the amount of sugar is reduced.
There's no exact expression, as the sweetness of tea or other drinks is not customarily expressed by analogy in English.
You may be asked how many lumps you take, referring to sugar lumps or sugar cubes, white sugar formed into cubes for mixing into drinks like coffee and black tea. In the U.S., a traditional sugar cube measures about ½ teaspoons (2.5 cubic centimeters) in volume, or about 0.074 ounces (2.1 grams) in mass, but Wikimedia Commons images suggest that they are somewhat larger in Europe:
The lump is not necessarily literal; a host may well be spooning the sugar as opposed to dropping individually formed chunks of sugar into your drink. It is simply an approximation of sweetness to add.
In the U.S., restaurants and cafés will often provide you with a tray of cream and sugar with your conventional coffee or tea so that you can add as much of each to your liking. (Cream and sugar is a fixed expression; you ask for cream and sugar even if you will load your drink with non-dairy lightener and sucralose instead). For non-conventional tea recipes that come pre-sweetened like Thai iced tea, Masala "chai tea", or bubble tea, you must ask the server or preparer to have it made less sweet; unfortunately, there is no universal way to specify the level of sweetness.
In establishments which serve conventional coffee and tea lightened and sweetened, you may be asked how many sugars you take as opposed to how many lumps; after all, sugar cubes are a rarity nowadays, having long since been replaced by individual paper packets containing roughly the same serving of sugar.
In parts of the U.S., furthermore, you must specify whether you want sweet tea or unsweetened tea. Sweet tea is not merely tea to which sugar has been added, but a particular way of preparing black tea that results in an an intensely sweet tea. If it is too sweet for your liking, you must ask for unsweetened tea, then add sugar to suit your taste. Sweet tea is the standard way of preparing tea in the Southern U.S., whereas unsweetened tea is the norm in the rest of the country.