This is my sentence:
A study shows that item-item approach is less affective to shilling attack than User-User approach.
To understand that sentence, you'd need to know what shilling attack is. It is where people provide high ratings for their products, and lower ratings for the other products (usually competitors’ products).
I am trying to say that user-user approach is better than item-item to deal with shilling attack
Yes, "effective" would be correct, but the sentence needs some alterations, perhaps:
A study [reference] shows that an item-item approach is less effective in preventing a shilling attack than a user-user approach.
The word "User" should not be capitalised unless you are using it as a proper noun, which does not appear to be the case here.
I added the indefinite articles "an" and "a" to make it read more naturally. However, if you are referring to one particular item-item approach and one particular user-user approach then "the" would be the correct definite article to use instead:
A study [reference] shows that the item-item approach is less effective in preventing a shilling attack than the user-user approach.
It is probably a good idea to explicitly state that the intention is to prevent shilling attacks.
Finally, if you mention a study, you should give a reference to it so that readers can confirm its conclusion for themselves.
