Peter Bayne, along with his wife Catherine (nee Anderson) and four sons and twin daughters named Christina and Isabella sailed to Dunedin New Zealand on the barque "Sir Edward Padget" in 1856, eight years after the founding of the Otago colony and settled in a small valley just below the present Methodist Eventide Home at Company Bay, on the Otago Peninsula. The road is now called McTaggart Road, and a clump of bluegums marks the homestead place.

The Bayne's section of land adjoined that of the land belonging to Catherine's brother John Anderson, and to the Macandrew Bay side, Christie's lay. Each section of land was of forty acres and heavily wooded. Christie's established a sawmill and together the three families felled the bush and cleared the land for farming. Christie put in a wooden tram rail down to the water edge and the logs were pulled along the rails and formed into rafts to be taken up to Dunedin for sale. The three families in effect formed themselves into a Company to dispose of the timber and thus gave the bay the name "Company Bay".

The Bayne land extended up the hill towards the spot where eventually Larnach built his castle and on this section a home was made for the family, which was increased by further children born in New Zealand ( possible eleven in all) although there were only three girls, first the twins and later Ruby, who married a Mr. Brownlie, of Fore. Christina married a McTaggart who had a farm close by, a farm which remained in the McTaggard family until recently and Isabella married Kenneth Williamson.

• Married: 25th June, 1848 in Blackford Near Auchterarder,Perth,Scotland
• Note: _STATMARRIED

1. Isabella Anderson BAYNE b: 1st May,1852 in Forteviot,Perth,Scotland
2. Christina BAYNE b: 1st May,1852 in Forteviot,Perth,Scotland
3. James BAYNE b: 2nd September, 1848 in Auchterarder,Perth,Scotland
4. Peter BAYNE b: 1st October, 1850 in Muthill,Perth,Scotland
5. John BAYNE b: 16th April, 1854 in Aberfeldy,Perthshire,Scotland
6. George BAYNE b: 26th December, 1855 in Lanark,Glasgow,Scotland
7. David Ferguson BAYNE b: 25th August 1857 in N. E. Harbour,Dunedin New Zealand
8. Catherine BAYNE b: 10th Sept. 1858 in N. E. Harbour,Dunedin New Zealand
9. Robina BAYNE b: 4th Dec. 1859 in N. E. Harbour,Dunedin New Zealand
10. William BAYNE b: 10th March, 1861 in N. E. Harbour,Dunedin New Zealand
11. Andrew BAYNE b: 24th March, 1862 in N. E. Harbour,Dunedin New Zealand
12. Margaret BAYNE b: 17th August 1863 in N. E. Harbour,Dunedin New Zealand
13. Kenneth BAYNE b: 4th April 1866 in N. E. Harbour,Dunedin New Zealand
14. Joseph Samuel BAYNE b: 3rd October, 1867 in N. E. Harbour,Dunedin New Zealand