GP is not completely wrong. Many state constitutions do indeed create the office of Sheriff. The federal one does not, of course, because there is no federal sheriff.
Each state Constitution will have him listed. He is an officer of the state but sworn to uphold and defend both the state Constitution and the federal Constitution. Answerable only to the people of his county.
His job is to protect the law AND the rights of the people from federal law that would strip their rights away.
1. If you use the phrase "the Constitution" without first referencing a specific one, the overwhelming majority of Americans will assume you're referring to the US Constitution.