This identification is speculative:

• Newton St. runs along the right side of the house which is on the south side of the street at roughly today’s #508
• The house faces a small lane that proceeds to the left toward another Goodnough property on South St.
• There is a separate outbuilding to the left of the house

Phineas Goodnough died in 1860. Several sons and grandsons lived at the nearby South St. property and they could be among the men pictured in the photo.

[Courtesy of the Digital Commonwealth (CC BY-NC-ND). From the Brookline Photograph Collection published by the Public Library of Brookline]