These aren't the same at all.
The second one is what you want. It means that you saw a coin on the ground and picked it up.
In the first sentence, you're saying "pick" as in "choose".
When I see this sentence, what I see in my mind is three coins on the ground next to each other and you chose one of the three. It doesn't mean that you took it in your hand, or that you walked off with it.
Let's do it in terms of cards... "Find the lady" is a card game (or magic trick) with three cards on a table. One is the queen of hearts.
Image from here.
The cards are turned face down and shuffled around so that you don't know which is which. Now, I tell you,
Pick a card.
You point at the middle card. I flip it over. Wrong. Oh well, better luck next time.
So, "you picked a card on the table" but you didn't touch it. If you want to say that you actually touched the card and lifted it off the table, you need the word "up". "You picked up a card [that was] on the table."