Dunno the Islamic ruling on this. However, the most sensible option would be to politely ask your neighbour.
We had a neighbour with a cherry tree they told us to pick the fruit that grew on the overhang in our garden and keep it....we never did.
I have a pear tree, I really dont mind if people pick the fruit and eat it, just need to ask me first as it is actually compeletly in my garden, one year I found a strange man in my garden helping himself which ticked me off!!!