Reihan Salam rights about some of the social origins of political polarization in the fact that different kinds of people live in different kinds of places. To this I would add the baseline fact that extreme non-polarization in the 20th century was an artifact of Jim Crow. Members of congress’ views on economic policy didn’t correlate strongly with their views on highly salient race-related issues. And to that I would add Alan Abramowitz’ point that the modern electorate is much better educated and ideologically aware than was the electorate of the Gilded Age era of polarization.
All of this makes it quite difficult to achieve a “grand bargain” on tax and budget issues. Which is one reason I think it’s unfortunate that both our political institutions and the larger political culture in which they’re embedded places such a premium on grand bargains. You can easily imagine a healthier approach based on bipartisanship by alternation. The way this would look is that we would have no fiscal consolidation in 2011 or 2012, since none is necessary. But then if Barack Obama was still Prime Minister in 2013, you would do the left-wing version of deficit reduction—higher taxes, technocratic health care cost controls, and less defense spending. The volume of higher taxes the Democrats would want to implement wouldn’t suffice to close the long-term gap, but it would reduce it enough to be manageable for a while. Then if at some point the Democrats’ approach stopped working and led to an economic slowdown, Republicans would come into office and implement the right-wing version of deficit reduction—benefit cuts in safety net programs and austerity for public workers. Some of the stuff Democrats did would be rolled back, but the most politically and economically sustainable elements wouldn’t be. Then at some point Democrats would come into office, and roll back some of the stuff the GOP did, but the most politically and economically sustainable stuff would stick. Then they would do some new stuff and eventually they’d lose.
In many places, I think it’s taken for granted that this is how progress is made. In the UK, Labour never wins a definitive victory against the Conservatives and vice versa. Nor is there ever a “grand bargain.” But over the long term, public policy reflects both conservative and progressive ideas because people inspired by those ideas take turns trying to govern the country in a way that pleases the voters. America, by contrast, is stuck between people with apocalyptic visions of final victory and unrealistic visions of a “deal” that takes issues “off the table” via bipartisan consensus.