Time to start digging into your English books and reading tapes then. Trust me, when I'm stateside, my German gets really rusty and sucks bad unless I find some nice alte dame (german for "old lady") and have a chat with her in pure old country German. Of course after I go over to Germany for a couple of months and come back stateside, it takes me about a week before I can speak English straight again after having jabbered in pure German for 6-9 months. I can talk perfect German, but my english gets sloppy.
Same idea with learning any language. The more you use it, the better you get at it. The less you use it, the harder it is to speak fluently. That's what I'm planning on taking a long trip to Japan in a couple of years. Maybe crash out with The Small One for a couple of months just so I can clean up and perfect my Japanese. Of coure I'm sure The Small One wouldn't mind having a fellow Kraut with him for a couple of months. ^_^ (just kidding The Small One)
But seriously, before tackling a 3rd language, perfect your English. Here's a trick you can use. I do something similar to this when I'm going over to Germany for any period of time. Find somewhere that lots of people speak good English and often. Go hang out there and talk with them, listen to them, etc. It will do good to take the rough edges off what you know and make your english smoother. May not work for you, but I know it works for me, so it's worth a try for you.