In 1865, Joseph Hopkins Twitchell assumed the pastorate of the Asylum Hill Congregational Church and settled on nearby Woodland Street. He had volunteered for the Army in 1861 and became the chaplain of the roughest regiment he could find, the New York Zouaves, because he had observed that men less "rough and wicked" were well attended by chaplains already and because he thought himself well-suited to influence this kind of man.

His sermons stayed away from politics and focused on brotherhood and the loving relationship of man to God. His daughter, Harmony, married the composer Charles Ives.