How to handle Authorization with @custom queries

Is there any way to correctly format or forward errors when the authentication at the custom endpoint fails? Now you get:

{
  "errors": [
    {
      "message": "Evaluation of custom field failed because external request returned an error: unexpected status code: 401 for field: getCustomQuery within type: Query.",
      ...
      ]
    }
  ],
  ...
}