There is a nut which presses down on the packing and pushes it against the shaft. The grease keeps the joint lubricated and also helps with the sealing effect, they were never supposed to be 100% watertight as I understand it.
You should be able to buy more packing from a decent chandler or marine engineering place.
I would recommend changing those small cap type lubricators though and replacing with one with a decent reservoir. (these look a bit like an old stirrup pump)
I've read the info on the link posted above and it certainly seems worth reading.