When the interest and principal, or interest only, of any registered bonds are payable in New York City, or elsewhere out of the state, payment shall be then made at the place so designated in such bond or coupon, or at the financial agency of the state for such purposes. In order that the funds may not be misapplied, the county treasurer shall procure a draft for the amount, to be transmitted by drawing his check on some bank in this state, and both check and draft shall be so endorsed as to show upon what bond or bonds the funds shall be applied; or at the request of the party holding or owning such bonds, payment may be made at the office of said treasurer.