Can I write
I checked it, but nothing
I checked it but couldn't find anything
"But nothing" is too incomplete to be an independent clause, so "I checked it, but nothing" would not be a well formed sentence.
Very informally, one might possibly say:
I checked. Nothin'.
… but I would recommend against it for an English learner.
What could be an acceptable sentence would be:
I checked it, but found nothing.
I checked it, but nothing.
This not a sentence, even in the most radical of grammars. However, it would certainly be seen in reported speech, in journalism or fiction. There, it might be taken to mean:
I checked it, but nothing (was there/was happening/had happened/was new/&c.)
Your proposed replacement sentence is grammatical, but would be improved by the addition of a comma:
I checked it, but couldn't find anything.