Rumors with regard to General Butler's ability to make exchanges continued to reach the Secretary of War, and upon being sent for myself I found the Secretary in conference with General Halleck upon the subject, and heard the statement made by the Secretary that General butler was not only of the opinion himself that he could effect exchanges if empowered so to do, but that is was also the opinion of several members of Congress, upon which I expressed the opinion that he ought to be allowed to try; and I was thereupon directed to proceed to Fort Monroe to communicate the authority of the Secretary of War to General Butler to make exchange of man for officer, according to grade, enjoining upon General Butler that he was on no account to compromise or jeopardize the claims of our colored troops to the protection of the Government, according to the laws of war.
I proceeded to Fort Monroe, and after communicating with General Butler, and stating the restrictions under which the Secretary had placed him, I signed his orders, which were drawn up by himself, giving him the authority he desired, and returned to this city.
It is proper that I should state here that General Butler is my senior in rank, and that he immediately assumed the designation of commissioner of exchange, and has acted ever since in entire independence of myself, making no reports to me of his proceedings in that character, which I mention simply as a fact, but not as taking exception to it. From that time until within a few days I have had no practical control of the subject of exchanges.
Soon after receiving the proper authority to make exchanges General Butler sent for exchange several hundred rebel prisoners by the flag - of - truce boat, offering them in exchange for a like number of Union prisoners. When this became known to the rebel authorities - as I feel justified in saying from the information I have received - there was some sort of conference held by the chief rebel officers in Richmond, in which it was determined that inasmuch as a number of their men had been sent for exchange, and were then within what they considered their boundaries, they should not be turned back into captivity, but that a corresponding number of Union prisoners should be delivered in exchange for them; but it was decided at the same time by the authorities in Richmond that, under a proclamation of Jefferson Davis, General Butler was an outlaw, and that no business whatever should be done with him, and, as I am credibly informed, they declared that the flag of truce even should nor protect him. This put an and to any further proceedings for some length of time in the business of exchange, and until an experiment was made at my suggestion, though not after the manner suggested by me. I had recommended to the Secretary of War that 300 of 400 rebel officers should be sent for exchange under a flag of truce, which I knew would not be accompanied by General Butler himself, and I was in hopes that public opinion in Richmond would constrain the authorities to accept that class of prisoners and return a like number for them, after which I thought they could not refuse to receive a boatload to their men. This suggestion was approved by the Secretary of War, and General Butler was directed through General (then Colonel) Canby to make the trial, but General Butler assumed to deviate from his orders, and sent a boatload of officers and men instead of officers alone. The enemy thereupon decided to return, not a like number of officers and men, but a number proportionate to the number of Union prisoners held be them as against the number of rebel prisoners held by us.