To which political forces are executive agencies responsive and why?

The Federal bureaucracy is a remarkable entity in many ways; as a collective layer of government, it employs over three million people across the country (and the world), and consists of bodies that play a part in the regulation of countless activities engaged in by American businesses and individuals. It also attracts enormous hostility from many sections of American society, not least Presidential nominees (it was George Wallace who coined the phrase "pointy-headed bureaucrats"). This is perhaps not surprising given that it is essentially a fourth branch of government not provided for in the constitution. However, the lack of a framework such as those that exist for the other three branches has meant that neither the scope nor the means of control of the bureaucracy are agreed upon by those they are nominally responsible to. Combined with the reactionary and ad hoc manner in which many agencies have been set up in response to unforeseen crises, particularly around the time of the New Deal and W.W.II, this has produced a legacy of struggle between legislative and executive branches over the lines of responsibility that the executive agencies should follow. Even the judiciary can influence the degree to which an agency can carry out its function. Political scientists have produced many theories specifying to whom the different sections of the bureaucracy should be accountable to; our interest for this discussion lies in more positive terms of what the real-world case actually is, and why it is like that.
The American experience of bureaucracy differs greatly from that of Europe in one key historical fact; its development came after the establishment of political democracy. Thus it was deliberately founded to serve the purposes of elected officials, and did not grow out of a monarch's personal group of administrators (although, ironically, some would say that that is precisely the sort of person that the modern President surrounds himself with). The period of most sustained growth was in the 1930s and 1940s. The emergence of the US as a global military, economic, and political superpower after the war prompted an influx of experts into government ranks for three main reasons: firstly, to give expert advice on the dominant political themes of the post-war world (i.e. the Cold War), secondly, to develop the new weapons and technology that this necessitated, and thirdly to help apply the Keynesian theories that became the accepted economic norm in this period. Thus a growth in the number of bureaucrats was experienced in common with most other developed countries. Another reason for the concern this prompted for accountability was the large difference in lengths of tenure between the bureaucrats and their political masters. The development of a merit-based civil service following the Pendleton Act of 1893 was intended to remove positions in the civil service from being distributed through the "spoils" system of rewarding loyal supporters by a successful President following his election. This created a class of administrators who were present throughout several different executive administrations, and were thus, not swayed by short-term political goals. The same theory applies with regard to Congress, where the tenure of Representatives is even shorter at two years. The downside to this is the danger that the bureaucracy becomes complacent as to implementing the wishes of its controllers, and takes an attitude that if it does not like what it is being told to do, it can stall until such time as a more amenable master is installed (although given the low turnover in Congress, this is less applicable to the legislative branch). Regardless, the threat of usurpation of power by the bureaucracy has always been taken seriously enough by both executive and legislative branches to move them to implement measures to constrain and control the activities of the bureaucracy. Further, the necessity to delegate results the problems of informational uncertainty and moral hazard arising - essentially, the principal has to come up with measures that will ensure that the agent (in this case, the bureaucrat) will carry out his/her wishes, and not shirk or defy orders given to them.
According to McCubbins and Page, there are several ways in which Congress can achieve this, either in the legislation itself, in which the scope of an agency's powers and the instruments they are permitted to use in achieving their mandate are specified, or in threats that Congress can make to the agency, such as threatening to cut off its funding or not renewing its authorisation. They predict that the varying degrees of uncertainty present in Congress and divergences in conflicts of interest between executive agencies over various topics impact on the amount of delegation given to bureaucrats, and the scope to pursue those goals as restricted by the means already mentioned. Their model predicts that acts dealing with environmental and health issues are delegated with greater scope, but more procedural restrictions, to bureaucrats in comparison with economic legislation. Given the pressure that legislators come under from groups that would like as few restrictions as possible on business activity with respect to health and environmental concerns, this offers further theoretical support for this model. However, it fails to take into account the growth of Congress' own bureaucracy, which has had the desired effect of closing the expertise gap with the executive branch, which in turn has improved the general level of information of the legislators as to the work of the various agencies. Thus Aberbach suggests that Congressional oversight can be seen in terms of a more continuous process rather than a series of discrete pushes towards regulation and intervention by Congress, usually prompted by some mishap. To this extent uncertainty is reduced, and there may in the future be a convergence as to the amount of delegation and restrictions placed on agency procedures regardless of the area of activity engaged in.
Successive Presidential administrations have also been wary of the bureaucracy, despite the fact that most of the agencies are within the executive branch and exist for the purpose of helping the President to develop policy initiatives and then to implement them. As evidence of this irrational paranoia that exists in the White House, Rourke can offer only two examples of when presidential plans have been substituted for those of independent actors - the first is General Douglas MacArthur's actions in Korea following the Inchon landing, and the illegal bombing raids carried out by the Air Force in Vietnam. It is obvious that these are extreme circumstances, and are not directly the actions of the bureaucracy as it would normally be perceived, so if these are the best examples that can be thought up, it seems that in reality the President has nothing to worry about from the bureaucracy. Indeed, many of the scandals that have dogged recent administrations, such as Watergate and Iran-Contra, have not been bureaucrats following their own agenda, but members of the White House staff. Nonetheless, a concentration of power among individuals in the White House staff has undoubtedly occurred, the intention being to maintain the hegemony of the President in the executive branch in the face of rising numbers of bureaucrats. To go so far as to claim as Rourke does that the power of the executive branch now resides among a collegial or collective Presidency seems too extreme. Undoubtedly the President's closest advisors wield a great deal of influence over other members of the executive branch, but they cannot act entirely independently of him in formulating, agreeing, and implementing policy - at some stage, the President must be involved in at least authorising these activities (even if only implicitly, as was the case with Reagan and Iran-Contra). Thus it would perhaps be more accurate to maintain that many of the activities of delegating policy proposals to the appropriate authority or agency are carried out on behalf of the President once he has decided upon the appropriate course of action; unless the issue is entirely clear and/or very mundane, then the advisors cannot act totally without the knowledge or sanction of the President. Regardless, the suspicion of the bureaucracy's loyalty is based on the tension between the short-term political term of the President and the careerist mindset of the bureaucracy. This prompts the White House staff to draw up major policy initiatives themselves, as they do not believe that the executive agencies will give the relevant Cabinet secretaries to whom they are directly responsible impartial advice and information. Again, this supposition appears somewhat flawed: theories regarding the weakness of the Cabinet stress that Cabinet secretaries are often presented with proposals from their departments that run contrary to Presidential prerogatives. This means that the bureaucrats are merely supplying the Cabinet secretaries with accurate information as regards necessary expenditure to meet campaign objectives in advancing the Presidential agenda; it is not the bureaucrats fault if it has not been made clear to them through their heads in the Cabinet that the policies have to be drawn up against the background of fiscal restraint (for example). However, the ultimate conclusion of Rourke's argument is that the more involved a President is with policy initiatives, the inevitability of the ascendancy of the White House staff with respect to the bureaucracy over national policy decisions. This is because the channels of communication between the President and his aides are much stronger than the indirect route between the bureaucracy and the President through the Cabinet. Thus if any institution is in need overhaul it would seem that it is the Cabinet; if lines of communication are made clearer and more direct between the departments that contain the expertise and the President, then given that the fear of substitution of policy is misplaced, this should enable a more accountable and focused bureaucracy to emerge.
This would not solve the underlying schizophrenia that we have identified as being at the heart of the problems of control over the bureaucracy. In essence, the executive agencies find themselves under scrutiny from all three other branches, and may report to both Congressional oversight committees and heads of executive branches. The bureaucracy faces the same problem when dealing with both of these branches - that of political uncertainty, and the need to maintain continuity between administrations. To that extent, it is inevitable that a certain "smoothing" of the implementation of directives from executive and the legislature will occur; the bureaucracy has to moderate sudden changes in policy that may follow elections to provide continuity in government. On the other hand, without a clearly defined structure of control either within a single branch or sharing power between branches is bound to result in the bureaucracy making up its own mind about what its long-term objectives are. Strengthening their isolation and independence from the administrators by either or both of the President and Congress will only push the bureaucracy to determine its own fate. Clarifying the situation would be incredibly complicated; it is a little late to write the structure of the fourth branch into the constitution. However, if the bureaucracy remains at its current size, or grows in the future, there may be no alternative in order to ensure that it isn't pushed through ignorance into increasing its power for what it sees as the purpose of keeping government moving.