You said the names wrong. But I appreciate you trying.
You said the names wrong. But I appreciate your trying.
I saw the first sentence in a TV-show. Is it grammatical? I thought sentence 2 was the correct one.
You are correct. In this sentence, "trying" is a gerund. It functions like a noun and serves as the direct object of "appreciate," and it should be preceded by the possessive pronoun "your." There's a detailed discussion of the use of the possessive case with gerunds here.
However, it's not at all uncommon to encounter gerunds preceded instead by a direct object pronoun in cases like this, especially in informal speech. Some native speakers hearing or saying the example sentence you've provided will misinterpret "you" as the object of "appreciate" and "trying" as a verb – even though that is not really what's going on in this sentence at all.
You are right. Sentence 2 is correct as taught in school.
But sentence 1 is used a lot. Many native speakers of English don't even realize that sentence 2 is what grammar books prescribe. "I appreciate you trying" is so common that I think many others on this site will say that it is correct.
And since the grammar of a language is dictated by its speakers, usually either 1 or 2 is OK.