South of Albert Park Beach was Middle Park Beach. In the 1900s when men and women were not permitted to swim together at the beach, there were public baths along the shoreline to cater to this law.

The building extended right into sea so that during the gusty winters, sections of the baths required annual repairs. With an end to this law, the baths were torn down.