The fort is named after King James II of England. Work on the fort started in 1706, and most of the buildings were built in 1739. In 1773, the fort was armed with 36 guns, and had a barracks which accommodated 75 men. In the 19th century, a gun fired salutes at visiting warships, as well as signals at sunrise and sunset.