Williamsburg Cemetery, Groveland

Williamsburg Cemetery has seen better days. The last century in particular has taken a tremendous toll on the resting place of some of the earliest and most influential settlers in western New York. Broken stones abound, but this burial ground, located on the site of the once thriving settlement, remains one of the most significant historic sites in Livingston County.

Numerous references can be found to the history of Williamsburg, the first white settlement in western New York. Similarly notable are the southern aristocrats Major Charles Carroll (1767-1823) and Col. William Frisby Fitzhugh (1761-1839), who along with Nathaniel Rochester (1752-1831) heavily invested in land development in the Genesee Valley and went on to found the City of Rochester. Their financial investments, political involvement, and philanthropy helped form the foundation of Livingston County and the Genesee Valley. 

Around 1815, both Carroll and Fitzhugh brought their large families and several enslaved people from Maryland to reside on the lands along the Genesee flats in the town of Groveland. William Fitzhugh built Hampton, a mansion on the Williamsburg site and Charles Carroll built the Hermitage estate a short distance south. The marriage between William Fitzhugh’s son William and Charles Carroll’s daughter Ann brought the two families even closer. 

Several of the Fitzhugh/Carroll men went on to distinguished careers in New York and Michigan; many of the women from this family, however, were notable as well. Of William Fitzhugh’s 12 children, two daughters in particular, Elizabeth Potts Fitzhugh Birney (1802-1869), and Ann Carroll “Nancy” Fitzhugh Smith (1805-1875), rose in national prominence. The sisters were most known for their anti-slavery activities and philanthropic contributions to the advancement of African Americans. Ann’s daughter, Elizabeth Smith Miller (1822-1911), who was born at the Hampton homestead, is well known as the designer of the original Bloomer costume. She and her daughter, Ann Fitzhugh Miller (1856-1912), were suffragists and carried on the family tradition of philanthropy and social reform.

Of all these women, Elizabeth Fitzhugh Birney and her husband and children are buried at Williamsburg Cemetery alongside her parents and various other members of the Fitzhugh and Carroll families.


Elizabeth Potts Fitzhugh Birney (1805-1869), was the oldest daughter of Col. William Frisby Fitzhugh and Ann Hughes Fitzhugh. In 1841, she married James Gillespie Birney, a slaveholder turned abolitionist and a prominent national political figure (see also: https://www.loc.gov/item/mm79012799/). They met in Cazenovia, New York, at her sister Ann’s home. Her brother Daniel was the connection, being a close acquaintance of Birney and Ann’s husband, the renowned abolitionist, Gerrit Smith

After living at Hampton for a couple of months, the Birneys moved west to the pioneer community of Lower Saginaw (now Bay City), Michigan to settle on lands owned by James Birney. Birney invested his time and financial resources in establishing Bay City and the abolitionist Liberty Party. He ran on the party ticket for U.S. President in 1840 and 1844. While James was busy running for office, Elizabeth had two children and helped establish the Trinity Episcopal Church.

James Birney had previously fathered several children with his first wife, Agatha McDowell, who died in 1838. James and Elizabeth’s two children tragically both died at young ages. Their daughter, Ann Hughes Birney, died in 1846 at two years old. Their first child, Major Fitzhugh Birney, the first to be baptized at the Trinity Church, died of wounds received in battle during the Civil War in 1864. 

In 1853, James and Elizabeth moved to New Jersey after James fell and was partially paralyzed. Abolitionist and suffragist, Sarah Grimke, a friend of the Birneys, writes in her diary that Elizabeth was “quite a sunbeam” to the community. James Birney died in 1857, and after settling his massive estate in Michigan, Elizabeth returned to New York and lived in Geneva near her extended family until her death in 1869.

Elizabeth Fitzhugh Birney had arranged for her husband, her two children, and her son’s wife and child to all be buried in the Fitzhugh plot in Williamsburg Cemetery. Sadly, the Hermitage mansion, where she and James Birney were married in 1841 and now owned by her brother, was destroyed by fire while he was in Geneva to return Elizabeth’s remains to the Hampton estate for burial beside her loved ones at Williamsburg Cemetery.

Elizabeth’s obituary in the Genesee Valley Herald, dated January 20, 1869, said she was a “noble and Christian lady, with a mind stored with useful knowledge, and a heart that prompted good deeds of charity.” Indeed, in her will Elizabeth bequeathed thousands of dollars to organizations and people she had helped support while she was alive, including the Rochester Orphan asylum, Frederick Douglass and others as trustees for the education of African-American children in Rochester, Elizabeth Blackwell and others as trustees for the Women’s Hospital in New York City, the Trinity Church in Bay City, and a life annuity to Harriet Tubman, the “Moses of her people,” who risked her life helping dozens of enslaved people escape to find freedom.