Terraform's external data source: STDOUT syntax unclear - json

Question

I would like to use terraform's external data source to identify certain AWS EC2 instances:
data "external" "monitoring_instances" {
program = ["bash", "${path.module}/../bash/tf_datasource_monitoring.sh"]
query = {
env = var.env_stage
}
}
The bash script is using AWS CLI to return a list of instance IDs.
But I keep receiving this Error: command "bash" produced invalid JSON: json: cannot unmarshal array into Go value of type string
I don't understand what the expected syntax of my script's STDOUT would be for terraform to understand the result.
So let's assume the script is supposed to return 3 instance IDs i-1, i-2 and i-3.
What would be the correct JSON syntax to be returned to terraform?
Examples, that do NOT work:
{
"instances": [
"i-1",
"i-2",
"i-3"
]
}
[
"i-1",
"i-2",
"i-3"
]

Answer 1

It is a known issue in Terraform for provider-external: https://github.com/hashicorp/terraform-provider-external/issues/2. It was opened a while ago, unfortunately is still present for latest version (Terraform v1.011).
You may want to avoid returning JSON objects which contain arrays.

Answer 2

I had the same issue, while executing a python script to generate a dynamic bigquery schema, which is per definition an array of JSONs.
I solved it, by implementing a wrapper JSON with the schema as a string value (see dummy code below).
# get_dynamic_bigquery_schema.py
import json
bigquery_schema = [
{"name": "int_field", "type": "INTEGER", "mode": "NULLABLE", "description": "int_field"},
{"name": "int_field_repeated", "type": "INTEGER", "mode": "REPEATED", "description": "int_field_repeated"}
]
wrapper_json = {'actual_output': str(json.dumps(bigquery_schema))}
print(json.dumps(wrapper_json))
which I can then access within terraform
# main.tf
data "external" "bigquery_schema" {
program = ["python", "${path.module}/get_dynamic_bigquery_schema.py"]
}
locals {
bigquery_schema= data.external.bigquery_schema.result['actual_output']
}