In the sentence:
He was only a yard off me.
I believe 'only' is an adjective because it adds information to the noun 'yard'. 'Off' is a preposition because it relates 'me' to the rest of the sentence.
Am I thinking right?
You are half right (that would be the second half).
Only is used as an adverb in your sentence. Notice that, if it were used as an adjective to modify the word yard, the indefinite article would be in front of the whole phrase, like in this example:
He is an only child. (Meaning: he has no siblings.)
If you move the indefinite article, you get an entirely new meaning:
He is only a child. (Meaning: Don't yell at him, he is too young to know what he did was wrong)
and in the 2nd example only is a focus adverb. These adverbs precede the word or phrase they modify, to focus attention on it.
It can be argued whether only modifies the word yard or the whole phrase a yard off me in your example but one way or the other it is an adverb that emphasises that the distance was short, that is not more than a (or one) yard. For this meaning, dictionaries (such as LDOCE) confirm that only is an adverb.
As for off you are correct, it is a preposition as it shows relation in space between the two people 'he' and 'me'.
This post by F.E. discusses in detail how to determine whether a word is an adjective, an adverb or a preposition.