He was born and raised in Manhattan, New York City and studied with drummers Max Roach and Elvin Jones before joining the Clark Terry Big Band in 1967. During his tour in the Vietnam War, he performed as part of the US Navy's Drum and Bugle Corps. He met guitarist Carlos Alomar when they were both playing with Roy Ayers. Davis was hired by David Bowie in 1974 with Alomar and bassist George Murray. Davis formed the rhythm section which performed on a number of Bowie's albums released in the 1970s. The snare sound used on Bowie's Low album is considered one of the most influential musical recording aspects in popular music.
In the early 2000s, Davis played percussion on David Bowie's live tours, including Bowie's last tour, A Reality Tour, in 2003. The drummer for that band was Sterling Campbell, who was previously a student of Davis.
Davis died on April 6, 2016 after a battle with cancer.