When you send a message, you encrypt it with their public key, so that only they can decrypt it. Additionally, you sign the message with your private key, so the person who receives the message can verify that you signed it, by using your public key.

This is the general idea of how public/private key crypto works. The actual Keybase implementation is a bit more complicated because a person doesn't have exactly one public/private keypair, but rather keys for each device.