If it's a database table naming thing, and your table set-up is so that there is one entry with one piece of the same type of information per entry (but allows multiple entries per country), I would probably use nothing... or By if i really wanted to use something.
Personally, I think per denotes that there is only one thing per category-thing.
By denotes that there are multiple things and can be sorted by each category-thing.
I also feel like "By Country" is a phrase that is used much more commonly than "Per Country" but that depends on the usage and context.
Usually I name my relational database tables without any prepositions/conjunctions like "per" or "by".
e.g.
SpecificInformationCountries
or I would probably have done
CountrySpecificInformation
(notice the singular form of "country" in this one)
Database naming has its own recommended conventions and you should not be looking for consulting in the "English language", but rather database naming conventions...