Accused of Falsifying Information!

When I initially applied for unemployment benefits I clearly wrote in the application that I had been working 8-hours a day, but I am now working only 2-hours. So when my checks started to come, I assumed they took into consideration that I am working part-time.

Then, my part-time job went from 2-hours per day to 3-hours. So I called to notify them of this increase.
At this time they told me that I should have been reporting my part-time job hours on a weekly basis!

Then they told me I've given them false information and I'm going to get penalized. They've setup a phone Interview next week.

If I had intended to defraud them, I obviously wouldn't be calling in to report the increase in my work hours.

Any advice? What do you think is going to happen to me?
I'm sure they're going to want back the over-paid amounts, but how big is the penalty? Will I face any other consequences.

This is what happens when everything is automated and there is no one to see face to face when submitting the application.