The Republican contest to avoid being House speaker, one of the most storied and powerful positions in American governance, should tell us that something in our politics has changed profoundly. Kevin McCarthy, Paul Ryan and Trey Gowdy are ambitious men of exceptional talents and highly developed conservative convictions. They have already sacrificed family time to get where they are. Individual circumstances cannot explain why they all wish to stay off center stage.
The disarray within Republican ranks isn’t an adequate explanation. Party fractiousness is nothing new. It didn’t dissuade Henry Clay, Thomas Reed, Joe Cannon or Sam Rayburn. The 40-odd Freedom Caucus Republicans are radicals, but on the surface their radicalism is tactical. They are not more strongly opposed than the Republican leaders to deficit spending, ObamaCare, Internet regulation and executive imperiousness. They may be more impatient, and more enraged over the manifold corruptions of Washington politics. But the exigent questions are the tactics of opposition. Hard tactical choice is what leadership is for, in the legislature as on the battlefield.
The answer lies elsewhere, in deep changes in the incentive structure of American politics. These changes, at once cultural and economic, have transformed everything from the parties to the presidency to the Congress in recent decades. The circumstance of conservative congressional majorities facing a determined leftist president has brought them to the surface in a vivid new form.
The key is to recognize that the members of the Freedom Caucus are not stupid or deranged; they are smart and calculating. They do not actually imagine that closing down the government over the debt ceiling or Planned Parenthood or the particulars of another continuing resolution will extract policy concessions from President Obama. Instead they are avid for Promethean drama and opportunities for personal display—even if, or especially if, they are crushed in the process and the Washington establishment emerges stronger than ever.
Some of them may think they can shoot the moon, with a string of losses eventually producing an overarching victory (their Senate soul mate Ted Cruz sees this as his route to the White House). But for most, the spectacle of unbending opposition is not a means. It is an end in itself.
That they find such a course attractive, and can pursue it against the obvious interests of their party, is the latest instance of the “atomization” of American culture and politics. The term was first applied in the 1970s by the sociologist Robert Nisbet and the political scientists Anthony King and James Q. Wilson. Cultural individualism has worn down one mediating institution after another, from family to church to local community. Rising affluence and advanced information and communications technologies have done the same to network news, labor unions and the hierarchal corporation. Together they have disestablished political parties and Congress.
Today’s atomized politics is entrepreneurial on the supply side and issue-specific on the demand side. It is structured around the networked affinity group. Some groups are devoted to general ideas and principles—progressivism, libertarianism, constitutional restoration. Others are devoted to discrete issues and causes—the endless list would include women’s sports, persecution of Christians in the Middle East, Medicare reimbursement rates for cardiac surgeons, school-lunch requirements, mortgage preferences for veterans, and supporting or opposing fracking or the Export-Import Bank. As candidates and legislators, modern politicians work through the party system but depend primarily on affinity groups for support, funding and guidance, and integrate them into their personal PACs.
This form of politics has transformed Congress. Single-member activism has replaced the committee hierarchies and autocratic chairmen of times past. Members do not need to bow to the leadership or patiently master the arts of legislative negotiation and coalition building. They advance their careers by demonstrating fidelity to the principles of general affinity groups and coaxing the executive agencies on behalf of discrete affinity groups. They are in turn closely monitored by these groups, which greatly limits their latitude for the results-oriented compromises that are the lifeblood of the representative legislature. And they devote an enormous amount of time to personal fundraising.
This is not to say that party loyalty and group cohesion are no longer important. Republicans and Democrats are united by common world views, conservative and progressive, to a much greater degree than in earlier eras. And although Congress’s internal power structure has atrophied, a new mechanism of party discipline has arisen—the almighty executive state with its fingers in every pie.
When the president’s party holds one or both houses of Congress, his party is loyal, delivering majority votes the president needs in exchange for executive actions individual members need. Group cohesion comes naturally to the Democrats, whose party has become a coalition of government-dependent interest groups. Each of them tends to favor almost everything the government is doing and simply wants more: Logrolling is easy among members who want more of this in exchange for more of that.
The Republican majority lacks these methods for galvanizing the conservative worldview of its members. The Democratic president can block their highest legislative priorities and is hardly going to use executive discretion to help them assemble majority votes on narrower bills. They are left with incremental changes of their own devising that have little appeal to the Freedom Caucus. Its members—younger and from relatively safe districts—are tightly networked with ideological affinity groups, whose membership interests and organizational needs are opposed to incrementalism. To them, logrolling and compromise are forms of corruption—the very things that have led over time to today’s overreaching, badly indebted, freedom-destroying federal leviathan.
The problem is captured by the recent experience of a staunchly conservative congressman of the old school, who had negotiated a highly favorable legislative compromise on a favorite issue. With several Democrats on board but needing a few more Republican votes, he approached a friend who is one of the most intelligent, attractive members of the Freedom Caucus. His friend saw the merits but averted his eyes and said softly, “I can’t . . . I just can’t.”
These political dynamics are well known to every member and are the reason that many ambitious Republicans fear the speakership. Whoever winds up in the post will need to convince the Freedom Caucus of two things: First, that a Congress of solo practitioners has become a powerful engine of executive-led government growth. They will have no prospect whatever of restoring limited government without first restoring a Congress that knows how to exercise its constitutional powers. Second, that two congressional majorities, acting cohesively under strong committee leadership, would have a good prospect of using incremental compromise for limiting rather than growing government.
Imagine, for example, if the Republicans had enacted a new deficit ceiling, reinstated regular budgeting and made other procedural reforms earlier this year. They would now be in a position to exercise the power of the purse by attaching riders to the appropriations of individual agencies. Debate would then focus on policy issues of their choice, without being upstaged by the risk of a government shutdown or financial default that play into the hands of the president and congressional Democrats.
In this context, Republicans could surely round up enough Democratic support to close down the Labor Department’s rule-making to convert broker-dealers in retirement funds into legal fiduciaries of their customers—a cherished and highly perverse Obama goal. They might even restrict or defund Planned Parenthood at one or two agencies. Facing the prospect of undistracted national attention to that organization’s sideline in harvesting fetal parts, the Democratic leadership might deem it prudent, with sincerest regret, to cut Planned Parenthood loose.
If compromise-averse conservatives want a macho motto to justify their reformed ways, they might try a paraphrase of Gen. George S. Patton: The aim of legislative politics is not to compromise your principles for your country—it’s to get the other guy to compromise his principles for his country.
Mr. DeMuth is a distinguished fellow at the Hudson Institute. He was president of the American Enterprise Institute from 1986 to 2008.