No, the 13B model outperforms GPT-3. Judging from the metrics published in the paper, it does look like the 7B model is not far off from GPT-3 however.

Supposedly double the model size so 14gb. RTX 4090 might be able to handle it. You can use lambdalabs to rent a server gpu for one of the larger models.

I don't know if it matters but the 7B parameter checkpoint is 13.5GB in size. Someone with 24GB VRAM struggled to run it:

https://github.com/facebookresearch/llama/issues/55

7B would require at least 14GB VRAM in 8 bit precision. 28GB in 16 bit precision.