Not sure I'm sold on the notion that reproduction of ultrasonic overtones is the reason that analog recording and reproduction sounds "warmer" than when digital is introduced into the mix. Just because analog is theoretically capable of reproducing frequencies over 20 kHz doesn't mean that we somehow perceive them directly. I would think it's at least equally possible that the quantization of digital recording -- chopping up the continuous stream of sound into very short bits -- affects the texture of the final product.
What I really suspect, though, is that there's just something about the "dirt" in the sound of an LP playing that we find inherently pleasing.
Without a doubt, though, ultrasonic frequencies affect the character of the sound that we are able to hear. When tones are produced by a stringed instrument or reproduced through an amplifier, all of the tones together combine to produce a single complex waveform.
This site, judging from a brief skim, looks to cover the effects of constructive and destructive interference between sound waves pretty well: http://www.physicsclassroom.com/class/sound/u11l3a.cfm
This image from the site illustrates how "pure" tones an octave apart (the higher note having a frequency exactly double that of the lower note) interact to create a waveform with periodically increasing and decreasing amplitude.
These amplitude shifts are called beats, and for complex combinations of notes -- or even a single note and multiple orders of overtones from an instrument -- beats created by interference caused by higher frequency tones are generated at considerably lower frequencies. So, it is very likely that tones with frequencies outside the range our ears actually can detect are affecting the texture of the overall tone in ways that are perceptible, if subtle.
Maybe, though, analog recording and reproduction are better able to reproduce these subtleties than digital recording. Even within the analog realm, tube amplification produces what's generally accepted to be a warmer tone than solid-state amplification, though I've never seen a scientific explanation of how the different signal paths produce quantitatively different signals.
Come to think of it, I've never looked for one. I think I'll do that for a bit now...
(Keeping in mind that the tube vs. solid state argument is generally to do with the quality of the distortion -- but since distortion always emphasizes and generates certain high frequency overtones or additive noise, I think the question is relevant to the question of perception of sound...)