The group was initiated by Jon Langford (also of the Waco Brothers and The Mekons) as a covers group, with a constantly shifting repertory and cast of backing members. The name was first used for Langford's 1995 album of Johnny Cash cover songs. In 1998, Langford enlisted the help of a number of prominent alt-country musicians (including Neko Case, Alejandro Escovedo, and Robbie Fulks) for a full-length tribute album to Bob Wills. The PVC served next as the backing band for Kelly Hogan's second album, Beneath the Country Underdog; Hogan had provided vocals on the Wills tribute. Their next LP, The Executioner's Last Songs, was released in 2002 and is a collection of songs about death. Following this record's success, the group recorded two more albums of songs about death as benefits for the Illinois Coalition Against the Death Penalty. These albums featured such guests as Steve Earle and Mark Eitzel. The group rarely performs live, though it occasionally does special performances in the Chicago area. In 2007 the group appeared as backing band on Chumbawamba member Danbert Nobacon's solo album The Library Book of the World, as well as playing live with Nobacon.

The PVC appeared at the Old Town School of Folk Music in Chicago, Illinois on October 10, 2009. The band was originally scheduled to back Australian Aboriginal singer Roger Knox, but Knox's US visa was denied. Instead, the band played a number of Knox's songs accompanied by singer Sally Timms. A second set had the band supporting a performance of Rosie Flores. Read more on Last.fm. User-contributed text is available under the Creative Commons By-SA License; additional terms may apply.