Perhaps This Is Part of the Problem?

I was reading the current issue of Money magazine this weekend, and the following stat blew my mind- 46.4% of U.S. taxpayers pay NO federal income tax. What? Is this some kind of joke? Now I understand that certain people have incomes too low to pay (according to the article, 15% of this group makes less then $20,000). But that still leaves a lot of other people who aren't paying in. To put it another way, each person paying federal taxes like me is essentially paying for him or herself and someone else. Does anyone think this mayhaps be part of our deficit problem?