What to do when you "accidentally" accepted a job offer and are waiting to hear from another job?

Let's say that ABCo was the first to offer me an interview and the second that actually interview me. They offered me the job on the spot. I was pretty hesitant at first being that it was a minimal wage job but is somewhat relevant to my field. I pushed my self to agree, fearful that it would be my only choice, and found myself signing a handful of documents, one being the written job offer. This was the first job offer since I graduated with BS in December and I haven't gotten a chance to see how the real working world is like. Needless to say I didn't know what it meant to sign a job offer (my past employment only included the dollar store and catering which I don't remember signing a job offer for).

Not too long after I get the offer, I receive a call from 123Co, the first people I interviewed with, for a chance of a second interview. I was really excited for this because it was for an bachelor's level job, paid more, came with benefits, was an easier commute, and seemed like a better learning opportunity. I believe the second interview went great. The guy mentioned that he had to get approval from the higher ups before he could do anything else and that he WILL get back to me. I left hopeful that I would be able to get the job.

However, I ignored the fact that I officially signed the job offer for ABCo. I learned today that they want to start orientation Monday. How wrong would it be to quit this job if I receive an offer from the preferred job?