The sentence has both meanings. The listener will pick which is most likely.
This kind of ambiguity is the basis for some humor, for example, the famous Groucho Marx quote:
One morning (while in Africa) I shot an elephant in my pajamas. How he got in my pajamas I'll never know.
Like "in New York" it's not clear whether the phrase "in my pajamas applies to "I" or "an elephant". We initially assume the more likely option, only to be told the other, improbable option is the truth.
In the same way you could make a (weak) joke poking fun at some undesirable location:
I was offered a job in New Jersey. Thank goodness the actual job was in California.
If you want to remove the ambiguity, you just need to add a word or two:
I was offered a job while in New York. (I was in New York)
I was offered a job located in New York. (The job is in New York)