Where to start
As a minimum, assuming you've got some basic kit to wear (including a waterproof), I'd say you need something like:
Helmet (up to you I guess but I've seen enough people crack helmets in crashes to convince me that they're worth wearing) £50
Pump (preferably a track pump with a pressure gauge so you don't get lots of punctures from having tyres too soft) £20, plus £15 for a mini-pump or Co2 for the road
Multi tool £15
Lights - basic set £30ish?
puncture kit and spare tube - £15
Chain tool and 'powerlink' - £15.
Gloves £15
Cycling shorts £20
Lock if you're going to be leaving it anywhere unattended.
prices are rough guesses for decent budget kit. Good enough to start out with anyway.
I'd also add a lot more stuff, but you could probably cope with that. Might be worth asking someone who only cycles casually what stuff they definitely need
re: bike, check with the local bike shops (proper bike shops, not big chains). Should be able to get something decent for £300 to £350.
Look up Specialised Sirrus, Scott Sub 40 or Cervelo R5ca for a rough idea.
Mind you, if you live near some good trails you may ride a MTB more - hybrid is the jack of all trades but you need to have an idea of where you'd be riding.
Of course you could always just buy some trainers and start running
