I'm in a similar boat to Confusion on this one. I don't care about elegance; if I make a comment and then someone shows up with 2 sentences that basically say "You're an idiot. Obviously the answer is X." then context is important. Without knowing what the score is, I'll probably disregard the comment because I obviously disagree. However, if my comment has 1 point and the responder's has 50 points, then I know there's a pretty good chance I'm wrong, and know to go investigate.
Perhaps I'm in the minority; I'd downvote that even if it was right because that's a shitty attitude for somebody to take.
Meanwhile, I think if somebody replies to you with something like that, it's worth investigating regardless of if they're the top answer or not. I'll draw a parallel to Reddit's "What's your favorite album?" threads. If you were to go by comment points alone, you'd think Radiohead's OK Computer was the best album of all time and that NIN took the four spots under that; meanwhile, albums out of the hive mind don't get as many upvotes, even if the commenter is more passionately in love with them.
Similarly, just because a lot of people think one thing on HN doesn't mean that one thing is the best answer. Looking up multiple solutions may take more time, but that leads to learning and growth, which I think should be the point of any community.
I don't know what kind of time you have available in an average day, but I can tell you that I don't have the time to go research what every yahoo with an opinion has. I don't mind being wrong, but if I say something, I am saying it because I think I'm right. Just because someone disagrees with me doesn't make me think I should go spend the time trying to prove their position.
You're awake for sixteen hours a day and you can't spend five minutes looking up what somebody's talking about? Research nowadays doesn't take a long time.
It's not a matter of doing it "part-time". The original example given was that if somebody said "this way is better" and they had lots of upvotes, then it would be worth looking into. Thing is, that's a rare case in a conversation. Most talks here are opinionated, and opinions don't require research. In the off-chance that there is a comment that has, say, five responses suggesting alternative methods, then that's obviously a subject that warrants research on the part of the original commenter, if he cares at all about the subject.
I trust both my own and the community's judgment. In general, I don't value one above the other. However, when they are at odds, that is an indication that I need to pay special attention, either because I am wrong, because many people are somehow wrong (which I think is quite relevant knowledge), because the situation isn't as clearcut as I may have thought, etc. In specific cases, I may come to value one judgment above the other, which does not necessarily mean dismissing the other judgment. These things are quite gray and I feel comment scores help in distinguishing the shades of gray.
To me, the points are a way for other readers to signal agreement without needing to clutter the thread with dozens of "me too" comments. A score of 50 gives a comment the force of many voices, but it compacts those voices into two efficient bytes.
Sure, it's a noisy signal because scores don't just indicate agreement, but the context usually makes that clear. If a factual claim is up- or downvoted, it usually indicates that people believe the claim is true or false. For analysis or opinion, it may have to more with whether the comment is insightful or thought-provoking.
If somebody refutes you elegantly, then you should look into their refutation regardless of whether or not the refutation has many points.