A good base would be Manly. It has a wide range of accommodation and services within a compact area and is a ferry ride from central Sydney.
By bus from there you can get to several birding sites on the Northern Beaches, including the significant Long Reef.
A good birding location nearer Central Sydney is Bicentennial Park (also known as Olympic Park or Homebush).
It has several habitats and hides.
You can get there by train or bus and either walk around or hire a bike by the hour.
The government has a website and app called Opal which is useful for planning the feasibility and cost of getting between locations on foot or public transport.
For more information there are three bookshops in central Sydney with a good selection of bird books including guides to sites, as well as the identification field guides.
They are Abbeys, Dymocks and Kinokuniya and are all close together near Town Hall Station.