Btw, I have them just growing over the fence and lead them with little hooks

The most common backyard variety, bred to withstand cooler temperatures, is the Nellie Kelly. This variety grafts popular passionfruit vines onto a hardier rootstock vine making them less susceptible to disease, pests and frosts. You can find tips on how to grow it here http://www.nelliekelly.com.au/passionfruit.html.
Commercial growers recommend purple varieties for temperate regions such as Northern NSW and Panamas for more tropical regions such as Central and Far North QLD.

I would go and speak with someone at your local nursery.

I would go and have a chat to your local nursery, I’m not too fimaliar with the climate in Sydney but I imagine there would be a strain of passion fruit perfect for your conditions! They love to climb and don’t really need much to get started, a fence is great and maybe just a roll of plastic lattice to get you started.