Fleck always had diverse musical interests, and his own style was influenced by Tony Trischka, Earl Scruggs, Chick Corea, Charlie Parker, John Coltrane, the Allman Brothers, Aretha Franklin, the Byrds, and Little Feat. After graduation, he joined the Tasty Licks, a group from Boston. They recorded two albums and dissolved in 1979. Afterwards, Fleck joined the Kentucky band Spectrum. That year, only five years after he took up the instrument, he made his solo recording debut with Crossing the Tracks, which the Readers' Poll in Frets magazine named Best Overall Album. In 1982, he joined New Grass Revival and stayed with them until the end of the decade. During this time, his reputation continued to grow and in 1990, Frets magazine added his name to their Hall of Greats. In 1988, one of his compositions, "Drive" (from the album New Grass Revival), was nominated for a Grammy.

Fleck, mandolin player Sam Bush, fiddler Mark O'Connor, bassist Edgar Meyer, and Dobro player Jerry Douglas teamed up in 1989 to form Strength in Numbers and record The Telluride Sessions. Late that year, Fleck was asked by PBS television to play on the upcoming Lonesome Pine Special; in response he gathered together a veritable "dream team" of musicians to form the Flecktones. The original members included Howard Levy, who played piano, harmonica, and ocarina, among other instruments; bass guitarist Victor Lemonte Wooten, and his brother Roy "Future Man" Wooten on the drumitar, an electronic drum shaped like a guitar. Though the special wasn't aired until 1992, the Flecktones recorded their eponymous debut album in 1990 and followed it up with Flight of the Cosmic Hippo (1991).

In 1993, they released their third album, UFO Tofu, which featured music blending different genres ranging from bluegrass to R&B to worldbeat. In 1995, they released Tales from the Acoustic Planet; Left of Cool followed in 1998, and Tales from the Acoustic Planet 2: The Bluegrass Sessions was released a year later. Outbound followed in mid-2000. Busy and prolific, Fleck released an album of classical pieces, Perpetual Motion, in late 2001, followed by Live at the Quick in 2002, the ambitious double-disc Little Worlds (and its truncated single-disc version, Ten from Little Worlds) in 2003, and Music for Two (with bassist Edgar Meyer) in 2004. Fleck appeared on Abigail Washburn's first album, Song of the Traveling Daughter, in 2005. Hidden Land, another album with the Flecktones, appeared on Columbia Records in 2006. The band released its first holiday collection in 2008, appropriately titled Jingle All the Way. The Melody of Rhythm: Triple Concerto & Music for Trio appeared in 2009 from Koch Records, which teamed Fleck with cellist/bassist Edgar Meyer and the Indian percussionist Zakir Hussain along with the Detroit Symphony Orchestra directed by Leonard Slatkin.

Fleck was also part of Washburn's Sparrow Quartet (with cellist Ben Sollee and fiddle player Casey Driessen), which, sponsored by the U.S. government, toured China and released Abigail Washburn & the Sparrow Quartet in 2008. Also that year, Fleck went to Africa to take part in a documentary film directed by his half brother Sascha Paladino and collaborated with over 40 of the continent's finest musicians, including D'Gary, Baaba Maal, Vusi Mahlasela, Toumani Diabaté, Bassekou Kouyate, and Oumou Sangare in Tanzania, Gambia, Mali, and Uganda. In 2009, Throw Down Your Heart, Tales from the Acoustic Planet, Vol. 3: Africa Sessions -- both a film and recording -- was released to widespread critical acclaim and commercial success. Fleck reunited the original Flecktones for the spring 2011 release Rocket Science, and toured with the band as well as the trio with Hussain and Meyer.