According to the Cambridge Dictionary
It's down to me to find a suitable person for the job.
Is it's down to me commonly used in casual conversations?
Like:
It's down to me to find a suitable person for this job.
It's down to me to find an ideal vacation location.