I have been building backlinks to this site and others lately, mainly through commenting on blogs and forum posts. It can be very frustrating, because the search engines index these links in their own good time. So you can add a couple of dozen links and you don't see any benefit at all for weeks, if not months.
It's the same slow process when you add unique content to a blog or website. But at least you can see it there. So you do have a bit of sense of achievement about it.
Off site optimization through link building is very important, though. So you should do as much as possible - without letting it make you go crazy with frustration!