I was facing the similar problem few years back.
My mother tongue is Hindi and despite being studying in English medium, I was too shy to speak English in public. Because of this, I always stayed away from attending any function or event. But, believe me, its all my loss by missing the moments that I would have enjoyed.
So, after remaining in depression and trying to fight my own issues, I finally decided to communicate in English with myself [by standing in front of mirror] and with my family [most supportive people ever] and continued this trend for 1 year.
After that, I started reading fictional novels and read "Word Power Made Easy". Reading helped me a lot in finding appropriate vocabulary.
Then came a time, for which I was waiting from almost 3 years. I was organising an event in my college and since it was a national level tech fest, students different colleges of India came.
Then there was a moment where I have to explain rules and regulations to a crowd of 400 people in ENGLISH. I remember I almost drank 1 litre water being nervous and went on the stage. Then slowly but steadily, I started speaking .. and then it was like miracle. Everything was like dream, I was speaking proper English smoothly without hesitating and when I was done, all I was listening is the sound of appreciation in the form of applauds.
So, don't stop learning, believe in yourself and don't care what others think.
Note: "Fear and Frustration are the two things that can make you succeed in life".