Boydtown owes its existence to the early Australian whaling industry, which was already flourishing in Twofold Bay in the early 1830’s. In 1843 a flamboyant entrepreneur named Benjamin Boyd decided to establish a base in Boydtown for his whaling and pastoral operations. Boyd was one of the most colourful and important characters in the Nation’s early history and Twofold Bay became the centre of his operations.

Boyd had a great deal of financial backing, which was evident in the grand style in which Boydtown was built. The best evidence of this grandeur is the historic Seahorse Inn, built in 1843 using convict labour. Boyd also built brick houses, a store, a woolshed and a jetty where Boyd’s vessels could be berthed. Nearby Boyd constructed a lookout, known as Boyd's Tower, which was built for the whalers to spot their potential catch. A church was also begun, but never completed. The tower and church ruins can still be seen today.

The Depression of the 1840's hit Benjamin Boyd hard and brought an end to many of his operations. From a prosperous commercial centre with 200 inhabitants, Boydtown soon became a deserted ruin, with buildings abandoned and deteriorating, and part of the population moving to Eden. Boyd left Australia for the Californian Gold Fields, but disappeared while hunting at Guadalcanal.

The Seahorse Inn remained deserted, reduced to a mere shell due to vandalism and natural deterioration. In 1936 it was puchased by the Whiter brothers, who renovated, restored and added a second storey.

In 1973 the Seahorse Inn was purchased by the Lyon Group with a vision to restoring it to its former glory. The Seahorse Inn has now been completely renovated to provide first class accommodation and conference facilities for the local and international tourist market. Renovations were completed in 2006.