Belt Tensioner

I thought that the belt on my '02 TJ 6 cyl. Sport was the problem so I tightened it up a little but that did not seem to fix the problem. Do you think that it is the belt tensioner? Is it hard to replace? How much should I expect to spend on the part? Thanks, Scooby