No! 23 is not too old to graduate from college. As you may know, many colleges and universities have both day and night divisions. Many of the night-students are well over 40 years of age, while some are in their 20s.
With the cost of a college education so high these days, it's best to be a very serious student and not waste your hard-earned cash fooling around.
After I received my PhD in chemistry, I remember my adviser saying to me, that now my real education begins. So it goes...Your education will/or should continue till you draw your last breath.
By the way, I was in my 20s, when I finished the PhD.