To give me their confidence is one thing; to give me of their confidence is another, sure. But I am not sure about the difference.
Context:
I should be destitute of feeling if I was not deeply affected by the strong proof which my fellow-citizens have given me
of their confidence in calling me to the high office whose functions I am about to assume. As the expression of their good opinion of my conduct in the public service, I derive from it a gratification which those who are conscious of having done all that they could to merit it can alone feel. My sensibility is increased by a just estimate of the importance of the trust and of the nature and extent of its duties, with the proper discharge of which the highest interests of a great and free people are intimately connected. Conscious of my own deficiency, I cannot enter on these duties without great anxiety for the result. From a just responsibility I will never shrink, calculating with confidence that in my best efforts to promote the public welfare my motives will always be duly appreciated and
More:
http://www.presidency.ucsb.edu/ws/index.php?pid=25807#axzz1q6RL9T2T