Does "the barbarians beyond the walls held back" refer to...?

Does "the barbarians beyond the walls held back" refer to"the barbarians beyond the walls of the White House held back"?

Context:

A week after the death of JFK, Jacqueline Kennedy summoned White to the Kennedy compound in Hyannis Port to rescue her husband's legacy. She proposed that White prepare an article for Life magazine drawing a parallel between her husband and his administration to King Arthur and the mythical Camelot. At the time, a play of that name was being performed on Broadway and Jackie focused on the ending lyrics of an Alan Jay Lerner song, "Don't let it be forgot, that once there was a spot, for one brief shining moment that was known as Camelot." White, who had known the Kennedys from his time as a classmate of the late President's brother, Joseph P. Kennedy, Jr., was happy to oblige. He heeded some of Jackie's suggestions while writing a 1,000 word essay that he dictated later that evening to his editors at Life. When they complained that the Camelot theme was overdone, Jackie objected to changes. By this telling, Kennedy's time in office was transformed into a modern day Camelot that represented, “a magic moment in American history, when gallant men danced with beautiful women, when great deeds were done, when artists, writers, and poets met at the White House, and the barbarians beyond the walls held back.” Thus was born one of the nation's most enduring, and inaccurate, myths. White later described his comparison of JFK to Camelot as the result of kindness to a distraught widow of a just-assassinated leader, and wrote that his essay was a "misreading of history. The magic Camelot of John F. Kennedy never existed."[3]

More:
https://en.wikipedia.org/wiki/Theodore_H._White#Career:_Making_of_the_President_series