The courthouse was first established in 1762, on land donated by Dr. Thomas Walker. William Cabell was the first builder of the original frame building, which was torn down and replaced in 1803 by the rear wing of the existing building. The grounds of the courthouse once held a whipping post, pillory and stocks, as was typical of the time. In addition to serving as the courthouse and sole place to vote in Albemarle County, religious services for Episcopal, Presbyterian, Methodist and Baptist worshippers were held in this space, which Jefferson called "The Common Temple.” Jefferson, Madison and Monroe were frequent visitors here, especially after Jefferson retired in 1809. In 1859 a front addition was constructed in the Gothic Revival style, with octagonal stair towers flanking the entrance. The portico was added in the 1870's.