Dillard grew up in an upper-middle-class home in Pittsburgh, the daughter of an oil business executive. Ann Haven Morgan’s The Field Book of Ponds and Streams, which she compared to the Book of Common Prayer, was one of her favorite books; it detailed the study and gathering of flora and insects in great detail. She earned both a bachelor’s and a master’s degree from Hollins College in Roanoke County, Virginia (1968). Richard Henry Wilde Dillard, a poet and creative writing professor at Hollins, was her mentor there, and she married him in 1965. Richard, she would later say, taught her everything she knew about writing.
Walden Pond and Thoreau was the title of her master’s thesis, which looked at the pond as a structuring technique in Henry David Thoreau’s Walden. Although reviewers have pointed out many distinctions between their two works, Dillard’s familiarity of Thoreau’s works was a clear inspiration. However, in the text, Dillard mentions that she named her goldfish Ellery Channing after one of Thoreau’s closest friends, as a nod to his influence.