I streamed all the time during Season 2 (most actually don't know you can stream livestream inside synchtube). Anyways, Synchtube has always had these problems (except for the playlist one, which popped up I think sometime mid last year), movies and ponies.
It seems that the more people you have watching one stream, the more probability you'll have for it glitching. (my friends and I use to call it Derpytube due to it always derping out) People with bad internet will be kicked from the chat all the time and if you're the host.. it may kick everyone out of the stream until you refresh. So it's not just the server on their end, it's also our end as the user. Combine those together and.. it can be bad. They do need better/more servers for sure though.
There's not that many people running the site (or at least last I used it), so problems will pop up. However, I do believe there is a twitter you can follow of theirs that'll tell you when maintenance will happen.
So as you said though.. you'll just have to deal with it.