Microkorg was my first synth (it took me forever to figure anything out on it when I first got it). A couple of volcas and a cheap bass amp, or no amp at all if you use headphones, would be a fun place to start. A microbrute or monoloue would also be excellent choices.
Where it's your first synth I'd start as basic as possible. If you get a Volca FM I feel like there's an excellent possibility that you wouldn't learn much about how a synthesizer works. Almost all the controls are hidden in sub-menus, like over a hundred controls. I would highly recommend avoiding any synth that you have to menu-dive with as a first synth.
O-Coast and Mother 32 are excellent starting points. I could see the patch-bay being a little intimidating if you're new to this, but you could always ignore the patchbay entirely on those until you feel like you're ready for it. I would've killed for synths like these when I was first getting into synths. You can play the Mother 32 with its little built in keyboard, but you would need something for the ocoast.
You can plug synths into anything, I use mine with a little 5w guitar tube amp. But you have to be a little more careful when using guitar amps (they aren't made to handle the lows synths can put out). You could use your amp, I would just shy away from cranking it up too much. Your turntable speakers would work if they're powered and you had the right kind of adapter cables? I'd recommend a bass amp, or headphones. I usually just use headphones
They use the same cables for the most part. The little "toy" synths like the volcas and pocket operators have a 1/8th inch trs output (I say "toy" because they're small and super fun). So, you just have to use the same kind of adapter you'd use for your phone/ipod/cd player/whatever (a 1/8" male trs to 1/4" male mono adapter). Most things use the same full size cables your guitar and pedals use.
) : 
