AWS SageMaker SparkML Schema Eroor: member.environment' failed to satisfy constraint - json

Question

I am deploying a model onto AWS via Sagemaker:
I set up my JSON schema as follow:
import json
schema = {
"input": [
{
"name": "V1",
"type": "double"
},
{
"name": "V2",
"type": "double"
},
{
"name": "V3",
"type": "double"
},
{
"name": "V4",
"type": "double"
},
{
"name": "V5",
"type": "double"
},
{
"name": "V6",
"type": "double"
},
{
"name": "V7",
"type": "double"
},
{
"name": "V8",
"type": "double"
},
{
"name": "V9",
"type": "double"
},
{
"name": "V10",
"type": "double"
},
{
"name": "V11",
"type": "double"
},
{
"name": "V12",
"type": "double"
},
{
"name": "V13",
"type": "double"
},
{
"name": "V14",
"type": "double"
},
{
"name": "V15",
"type": "double"
},
{
"name": "V16",
"type": "double"
},
{
"name": "V17",
"type": "double"
},
{
"name": "V18",
"type": "double"
},
{
"name": "V19",
"type": "double"
},
{
"name": "V20",
"type": "double"
},
{
"name": "V21",
"type": "double"
},
{
"name": "V22",
"type": "double"
},
{
"name": "V23",
"type": "double"
},
{
"name": "V24",
"type": "double"
},
{
"name": "V25",
"type": "double"
},
{
"name": "V26",
"type": "double"
},
{
"name": "V27",
"type": "double"
},
{
"name": "V28",
"type": "double"
},
{
"name": "Amount",
"type": "double"
},
],
"output":
{
"name": "features",
"type": "double",
"struct": "vector"
}
}
schema_json = json.dumps(schema)
print(schema_json)
And deployed as:
from sagemaker.model import Model
from sagemaker.pipeline import PipelineModel
from sagemaker.sparkml.model import SparkMLModel
sparkml_data = 's3://{}/{}/{}'.format(s3_model_bucket, s3_model_key_prefix, 'model.tar.gz')
# passing the schema defined above by using an environment variable that sagemaker-sparkml-serving understands
sparkml_model = SparkMLModel(model_data=sparkml_data, env={'SAGEMAKER_SPARKML_SCHEMA' : schema_json})
xgb_model = Model(model_data=xgb_model.model_data, image=training_image)
model_name = 'inference-pipeline-' + timestamp_prefix
sm_model = PipelineModel(name=model_name, role=role, models=[sparkml_model, xgb_model])
endpoint_name = 'inference-pipeline-ep-' + timestamp_prefix
sm_model.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge', endpoint_name=endpoint_name)
I got the error as below:
ClientError: An error occurred (ValidationException) when calling the CreateModel operation: 1 validation error detected: Value '{SAGEMAKER_SPARKML_SCHEMA={"input": [{"type": "double", "name": "V1"}, {"type": "double", "name": "V2"}, {"type": "double", "name": "V3"}, {"type": "double", "name": "V4"}, {"type": "double", "name": "V5"}, {"type": "double", "name": "V6"}, {"type": "double", "name": "V7"}, {"type": "double", "name": "V8"}, {"type": "double", "name": "V9"}, {"type": "double", "name": "V10"}, {"type": "double", "name": "V11"}, {"type": "double", "name": "V12"}, {"type": "double", "name": "V13"}, {"type": "double", "name": "V14"}, {"type": "double", "name": "V15"}, {"type": "double", "name": "V16"}, {"type": "double", "name": "V17"}, {"type": "double", "name": "V18"}, {"type": "double", "name": "V19"}, {"type": "double", "name": "V20"}, {"type": "double", "name": "V21"}, {"type": "double", "name": "V22"}, {"type": "double", "name": "V23"}, {"type": "double", "name": "V24"}, {"type": "double", "name": "V25"}, {"type": "double", "name": "V26"}, {"type": "double", "name": "V27"}, {"type": "double", "name": "V28"}, {"type": "double", "name": "Amount"}], "output": {"type": "double", "name": "features", "struct": "vector"}}}' at 'containers.1**.member.environment' failed to satisfy constraint: Map value must satisfy constraint: [Member must have length less than or equal to 1024,** Member must have length greater than or equal to 0, Member must satisfy regular expression pattern: [\S\s]*]
I try to reduce my features to 20 and it able to deploy. Just wondering how can I Pass the schema with 29 attributes?

Answer 1

I do not think the environment length of 1024 limit will be increased in a short time. To work around this, you could try to rebuild the spark ml container with the SAGEMAKER_SPARKML_SCHEMA env var:
https://github.com/aws/sagemaker-sparkml-serving-container/blob/master/README.md#running-the-image-locally