Learning to drive the first couple of times is scary but you get used to it the more you drive. You basically have to reach a comfort level. The unknown is scary after all. What I think is most dangerous aside from other drivers is you being reckless, getting distracted, not paying attention, and basically being over-confident in your driving. :\
Oh, and whatever you do, do not move the steering wheel left and right very quickly one after the other. You're going to swerve like mad and if you're in a suv, there's a good chance you're going to roll over.