The best way to let a guy down is to be kind, but just like RozeMaiden said, you HAVE to be frank. Guys are very black and white. If you say something sugar-coated and soft, they might think they still have a chance. You need to tell them the truth, and make sure they understand it. But don't be rude, and don't be mean unless it's warranted.
I've had to do this. It was...hm, maybe 2 years ago now? I thought I liked him, but realized I was just flattered by the attention and wanted to have a guy that liked me. I came to this realization pretty suddenly, and the moment I did, I told him exactly what my real feelings were. It went kinda rough. I'm not a mean person, but I can be pretty harsh. And I had to be harsh, because he just wouldn't accept the fact that we weren't in love. In the end I had to be brutal, but I didn't let myself feel bad - he needed to understand that I didn't love him! I actually had to go as far as to say "Stop being such a sap!"
We didn't talk for a LONG time after that. But a few months ago, he contacted me and told me that he'd changed, and that he was sorry for the way he treated me, and he was thankful that I tried to help him but didn't let him use me, and broke it off before things got serious. He has a girlfriend and they're very happy, and we've become good friends again.
So don't worry. Letting a guy down, no matter how horrible it goes...it's not the end of his world. He ought to be thanking you, in the end. My friend is proof!