If you are wondering, I got a bit of my source from a 4chan post on /b/ (although I think this guy is nuts):
It's not that I would ever entertain the thought of doing anything with children. The biggest thrill of looking at lolicon stuff for me is that it reminds me of sexual fantasies I had when I was young. I mean, seriously, are you saying you never had "nasty" thoughts about the kids around you when you were young?
Isn't it natural to think of little girls as cute, delightful, pretty, etc? Don't those qualities carry a sexual nature about them? Isn't it possible to have sexual thoughts that include children without ever having an urge to actually do something sexual with kids? Does looking at pictures of adult women automatically make someone want to go out and rape a woman against her will?
I've had several relationships (including my current one) with women of legal age. I have no need to do anything sexual with a child and know that it wouldn't be very satisfying. Looking at drawn lolicon pictures and the fantasies involved seem to me like a nice temporary escape into my own childhood sexuality... reliving thoughts I had back then and adding upon them.
Do we automatically have to think of everyone who looks at lolicon the same way as we would a child rapist or pedophile? Can't someone play with fantasies without desiring to actualize them?
And now I'll go wait for the gooberment to ban computer monitors with built-in power cords.