By 1931, just before the building was donated to the town, its condition had deteriorated to the point where it had been condemned as unfit for human habitation. At that time it still had an earth toilet at the bottom of the back yard.

The Society's first task was to set up a trust to raise funds for renovation and then to see that through to completion. The cottages had to be virtually rebuilt. Most of the original roof timbers were retained and five oak tie beams were fixed across the interior to strengthen the whole structure. The end result was a single-storey, light, airy room.

The museum was opened by the Mayor just a year after it had been acquired on June 8th 1933 under the curatorship of Dr Dudley Collings.

The main and original part of the museum is a Grade II listed building.

An extension was added in 1959 and dedicated to the memory of Dr. Collings. A further small extension was built in 1962 and, in 1967, the Society purchased a two-storey carpenter's workshop, which had been built in the 1950s at the bottom of the back yard. This was needed to provide basic storage space for objects which there was no room to display permanently and to house the museum's growing archive of books, photographs and postcards. It also made possible the construction of a museum office.

In 1999 the museum underwent a programme of essential refurbishment and was equipped with a new lighting system and carpeting. However this was only a very partial response to what was becoming an increasingly pressing need - to bring the museum up to today's conservation and equal-access standards and to present the story of Southwold in a more coherent, accessible and attractive way.

Between 2006 and 2008 Southwold Museum closed for a £375,000 rebuild and redesign programme supported by the Heritage Lottery Fund with local match funding. In 2009 the museum was highly commended in the Museums and Heritage Awards for Excellence. The judges commented that this was a "small museum that just gets it absolutely right".

A further major programme of refurbishment was undertaken during 2014-15, with the installation of low-energy lighting and climate-control systems. Several new displays have been added and there is a greater emphasis on exhibiting objects, hundreds of which have languished in storage for decades.