Related
I've been working on a chatbot and i'm using the Microsoft Adaptive Cards for my bot ui.
Currently i am im trying to create a a row that contains 2 textblocks and 2 input.text.
But when entering a value into these input.text they don't seem to set the value to the given id.
This does work however when the input.text are not in a coulmn set but instead are stacked on top of each other.I think its the Azure BotService not handling the JSON right.
I've tried using diffrent channels. Web-Chat, Teams-Chat. It does work on the adaptive card designer but not with any other service. Anyone else got this problem?
{
"type": "AdaptiveCard",
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"version": "1.3",
"body": [
{
"type": "ColumnSet",
"columns": [
{
"type": "Column",
"width": "stretch",
"items": [
{
"type": "TextBlock",
"text": "New TextBlock",
"wrap": true
},
{
"type": "Input.Text",
"id": "inputOne",
"placeholder": "Placeholder text"
}
]
},
{
"type": "Column",
"width": "stretch",
"items": [
{
"type": "TextBlock",
"text": "New TextBlock",
"wrap": true
},
{
"type": "Input.Text",
"id": "inputTwo",
"placeholder": "Placeholder text"
}
]
}
]
}
]
}
I tried to reproduce the issue and the pattern which was mentioned in the code block is having some blockage due to some input fields calling structure problem. Try to check the below code block for sample and replace with the current requirements.
Data JSON:
{
"title": "Tell us about yourself",
"body": "We just need a few more details to get you booked for the trip of a lifetime!",
"disclaimer": "Don't worry, we'll never share or sell your information.",
"properties": [
{
"id": "myName",
"label": "Your name (Last, First)",
"validation": "^[A-Z][a-z]+, [A-Z][a-z]+$",
"error": "Please enter your name in the specified format"
},
{
"id": "myEmail",
"label": "Your email",
"validation": "^[A-Za-z0-9._%+-]+#[A-Za-z0-9.-]+[.][A-Za-z0-9-]{2,4}$",
"error": "Please enter a valid email address"
},
{
"id": "myTel",
"label": "Phone Number (xxx-xxx-xxxx)",
"validation": "^[0-9]{3}-[0-9]{3}-[0-9]{4}$",
"error": "Invalid phone number. Use the specified format: 3 numbers, hyphen, 3 numbers, hyphen and 4 numbers"
}
],
"thumbnailUrl": "https://upload.wikimedia.org/wikipedia/commons/b/b2/Diver_Silhouette%2C_Great_Barrier_Reef.jpg",
"thumbnailAlt": "Diver in the Great Barrier Reef"
}
Template JSON:
{
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"type": "AdaptiveCard",
"version": "1.0",
"body": [
{
"type": "ColumnSet",
"columns": [
{
"type": "Column",
"width": 2,
"items": [
{
"type": "TextBlock",
"text": "${title}",
"weight": "bolder",
"size": "medium",
"style": "heading"
},
{
"type": "TextBlock",
"text": "${body}",
"isSubtle": true,
"wrap": true
},
{
"type": "TextBlock",
"text": "${disclaimer}",
"isSubtle": true,
"wrap": true,
"size": "small"
},
{
"type": "Container",
"$data": "${properties}",
"items": [
{
"type": "Input.Text",
"label": "${label}",
"id": "${id}",
"regex": "${validation}",
"errorMessage": "${error}",
"isRequired": true
}
]
}
]
},
{
"type": "Column",
"width": 1,
"items": [
{
"type": "Image",
"url": "${thumbnailUrl}",
"altText": "${thumbnailAlt}",
"size": "auto"
}
]
}
]
}
],
"actions": [
{
"type": "Action.Submit",
"title": "Submit"
}
]
}
The result adaptive card looks like below screen as the sample.
I have a JSON schema and a CSV file. The CSV file has 2,511 rows and one header row (2,512 rows total). Each row has 43 columns. I was able to convert the CSV to a JSON using one of the myriad of online converters, but the result is what I believed is termed a 'flat JSON file'.
Here is the CSV header row:
F1,F2,F3.1.F1,F3.1.F2,F3.1.F3,F3.1.F4,...F3.10.F1,F3.10.F2,F3.10.F3,F3.10.F4,F4
Here is my JSON schema:
{
"$schema": "http://json-schema.org/schema#",
"$id": "./.schema.json",
"title": "",
"description": "",
"type": "object",
"properties": {
"F1": {
"description": "",
"type": "string"
},
"F2": {
"description": "",
"type": "string"
},
"F3": {
"description": "",
"type": "array",
"items": {
"description": "",
"type": "object",
"properties": {
"F3.F1": {
"description": "",
"type": "string"
},
"F3.F2": {
"description": "",
"type": "string"
},
"F3.F3": {
"description": "",
"type": "string"
},
"F3.F4": {
"description": "",
"type": "string"
}
},
"required": [
"F3.F1",
"F3.F2",
"F3.F3",
"F3.F4"
]
},
"numItems": 10,
"unique": false
},
"F4": {
"description": "",
"type": "string"
}
},
"required": [
"F1",
"F2",
"F3",
"F4"
],
"additionalProperties": false
}
From the CSV->JSON conversion, my JSON file looks like:
[
{
"F1": 2429546524130460000,
"F2": 2429519276857919500,
"F3.1.F1": 2428316170619109000,
"F3.1.F2": 0.0690932185744956,
"F3.1.F3": 2.6355498567408557,
"F3.1.F4": 0.4369495787854096,
...
"F3.10.F1": 2429415922764859400,
"F3.10.F2": 0.15328371980044203,
"F3.10.F3": 2.677944208300451,
"F3.10.F4": 0.31036472544281585,
"F4": 0.16889514829995647
},
... //repeated 2,509 times
{
"F1": 1143081876266241000,
"F2": 1143588785487818100,
"F3.1.F1": 1141377392726037800,
"F3.1.F2": 1.332366799133926,
"F3.1.F3": 0.24878185970548322,
"F3.1.F4": 1.560443994684636,
...
"F3.10.F1": "XXX",
"F3.10.F2": "XXX",
"F3.10.F3": "XXX",
"F3.10.F4": "XXX",
"F4": 2.2916768389567497
}
]
Clearly, making the necessary changes 2,511 times is impractical, so I am hoping there is a method to make the changes automatically. I can code, but I could not find any specific solutions anywhere to go from a CSV to a JSON with the JSON output matching a specific JSON schema. Preferably, I would like a solution that is not restricted to just converting this one set of data to this one specific format, i.e., a general solution that could be used with a different CSV and different JSON schema.
i want to create sub-categories from the existing data frame
data frame column consists of (sample table) my changes required at the columns level not any changes in the data like a set of columns are the and column names 3 different suffixes (few with similar column names and other column names)
example like
|payer_id|payer_name|halo_payer_name|delta_payer_name|halo_desc|delta_desc|halo_operations|delta_notes|halo_processed_data|delta_processed_data|extra|insurance_company|
I want it to be grouped in this halo group halo_payer_name|halo_desc|halo_operations|halo_processed_data|
I want it to be grouped in this delta group delta_payer_name|delta_desc|delta_notes|delta_processed_data|
and the remaining columns as one group
so when converted to JSON it would come in this layout
{
"schema": {
"fields": [{
"payer_details": [{
"name": "payer_id",
"type": "string"
},
{
"name": "payer_name",
"type": "string"
},
{
"name": "extra",
"type": "string"
},
{
"name": "insurance_company",
"type": "string"
}
]
},
{
"halo": [{
"name": "halo_payer_name",
"type": "string"
},
{
"name": "halo_desc",
"type": "string"
},
{
"name": "halo_operstions",
"type": "string"
},
{
"name": "halo_processed_data",
"type": "string"
}
]
}, {
"delta": [{
"name": "delta_payer_name",
"type": "string"
},
{
"name": "delta_desc",
"type": "string"
},
{
"name": "delta_notes",
"type": "string"
},
{
"name": "delta_processed_data",
"type": "string"
}
]
}
],
"pandas_version": "1.4.0"
},
"masterdata": [{
"payer_details": [{
"payer_id": "",
"payer_name": "",
"extra": "",
"insurance_company": ""
}],
"halo": [{
"halo_payer_name": "",
"halo_desc": "",
"halo_operations": "",
"halo_processed_data": "",
}],
"delta":[{
"delta_payer_name": "",
"delta_desc": "",
"delta_notes": "",
"delta_processed_data": "",
}]
}]
}
for this type of situation i couldn't find a solution as it is a column based grouping instead of data-based grouping
so came across this post today it helped with my situation (adding data from a data frame and using it to create looped data and insert it into a dict and then convert the whole into a JSON file)
the ref that was helpful to me is link
so the solution for this question goes like this
schema={
"schema": {
"fields": [{
"payer_details": [{
"name": "payer_id",
"type": "string"
},
{
"name": "payer_name",
"type": "string"
},
{
"name": "extra",
"type": "string"
},
{
"name": "insurance_company",
"type": "string"
}
]
},
{
"halo": [{
"name": "halo_payer_name",
"type": "string"
},
{
"name": "halo_desc",
"type": "string"
},
{
"name": "halo_operstions",
"type": "string"
},
{
"name": "halo_processed_data",
"type": "string"
}
]
}, {
"delta": [{
"name": "delta_payer_name",
"type": "string"
},
{
"name": "delta_desc",
"type": "string"
},
{
"name": "delta_notes",
"type": "string"
},
{
"name": "delta_processed_data",
"type": "string"
}
]
}
],
"pandas_version": "1.4.0"
},
"masterdata": []
}
derived the schema above as i have desired
payer_list=[]
for i in df.index:
case={
"payer_details": [{
"payer_id": "{}".format(df['payer_id'][i]),
"payer_name": "{}".format(df['payer_name'][i]),
"extra": "{}".format(df['extra'][i]),
"insurance_company": "{}".format(df['insurance_company'][i])
}],
"halo": [{
"halo_payer_name": "{}".format(df['halo_payer_name'][i]),
"halo_desc": "{}".format(df['halo_desc'][i]),
"halo_operations": "{}".format(df['halo_operations'][i]),
"halo_processed_data": "{}".format(df['halo_processed_data'][i]),
}],
"delta":[{
"delta_payer_name": "{}".format(df['delta_payer_name'][i]),
"delta_desc": "{}".format(df['delta_desc'][i]),
"delta_notes": "{}".format(df['delta_notes'][i]),
"delta_processed_data": "{}".format(df['delta_processed_data'][i]),
}]
}
payer_list.append(case)
schema["masterdata"] = payer_list
created and empty list and run the loop and included in the empty list and joined or linked to the schema
I created a task that talks to a REST API to retreive the values for 2 picklists.
Filling the first dropdown box works fine, when using just the jsonpath.
Based on the first picklist I'd like to retreive values of the second list.
I've tried some variations and I'm trying something like this:
The json which I receive in the first rest call is similar to:
{
"id": "45",
"href": "https://selfservice/api/configurations/45/",
"name": "Type",
"description": "",
"version": "0.0.4",
"attributes": [
{
"value": {
"sdk-object": {
"type": "ConfigurationElement",
"id": "56"
}
},
"type": "ConfigurationElement",
"name": "win"
},
{
"value": {
"sdk-object": {
"type": "ConfigurationElement",
"id": "57"
}
},
"type": "ConfigurationElement",
"name": "lin"
}
]
}
I try to show the attributes name in the list and need the id of the attribute in the second picklist.
I created the following datasourcebindings in the task.json. Of course, the targets exist in the task.
task.json:
{
"id": "GUID",
"name": "Spinup",
"friendlyName": "Create environment",
"description": "Starts Workflow to create an environment. ___ The following placeholders are created deplending on the template and can be used in the rest of the release:**XLDEnvironment** and **testAgentHostname**",
"category": "Deploy",
"author": "***",
"version": {
"Major": 0,
"Minor": 0,
"Patch": 33
},
"minimumAgentVersion": "1.95.3",
"inputs": [
{
"label": "Endpoint",
"name": "connectedServiceName",
"required": true,
"type": "connectedService:server",
"helpMarkDown": "endpoint to connect to."
},
{
"name": "stage",
"type": "string",
"label": "Stage",
"defaultValue": "$(Release.EnvironmentName)",
"required": true,
"helpMarkDown": "Stage of the release, default value is based on the pipeline."
},
{
"name": "releaseName",
"type": "string",
"label": "Environment name",
"defaultValue": "$(Release.ReleaseName)",
"required": true,
"helpMarkDown": "Name of the environment that will be created."
},
{
"name": "owner",
"type": "string",
"label": "Owner of the environment",
"defaultValue": "***",
"required": true,
"helpMarkDown": "It is common to use the initials of the product owner in this field. There has to be a valid AD user owner of each environment. This can also be set by using a variable through the pipeline."
},
{
"name": "group",
"type": "string",
"label": "Group of the owner",
"defaultValue": "",
"required": true,
"helpMarkDown": "group that owns the environment. groups can be requested by the infrastructure department. Example: ****"
},
{
"name": "platform",
"type": "pickList",
"label": "Platform (OS)",
"defaultValue": "",
"required": true,
"helpMarkDown": "Choose the type of the target platform."
},
{
"name": "size",
"type": "pickList",
"label": "Environment Template",
"defaultValue": "",
"required": true,
"helpMarkDown": "Size of the environment to create."
}
],
"dataSourceBindings": [
{
"dataSourceName": "GetCEPlatformType",
"endpointId": "$(connectedServiceName)",
"target": "platform",
"selector": "jsonpath:$.attributes[*].name",
"keySelector": "jsonpath:$.attributes[*].value.sdk-object.id"
},
{
"dataSourceName": "GetCEPlatformSize",
"endpointId": "$(connectedServiceName)",
"target": "size",
"parameters": {
"platformTypeId": "$(platform)"
}
}
],
"instanceNameFormat": "Spin up $(size) $(platform) environment",
"execution": {
"PowerShell3": {
"target": "$(currentDirectory)\\task.ps1",
"argumentFormat": "",
"workingDirectory": "$(currentDirectory)"
}
}
}
vss-extension.json:
{
"manifestVersion": 1,
"id": "*****",
"name": "Release Tasks",
"version": "1.0.1",
"publisher": "***",
"targets": [
{
"id": "Microsoft.VisualStudio.Services"
}
],
"description": "Tools to contact ***. Includes a task to spin up a platform and a task to remove the platform.",
"categories": [
"Build and release"
],
"icons": {
"default": "images/extension-icon.png"
},
"files": [
{
"path": "RemoveEnvironment"
},
{
"path": "SpinUpEnvironment"
}
],
"contributions": [
{
"id": "******",
"type": "ms.vss-distributed-task.task",
"targets": [
"ms.vss-distributed-task.tasks"
],
"properties": {
"name": "RemoveEnvironment"
}
},
{
"id": "*********",
"type": "ms.vss-distributed-task.task",
"targets": [
"ms.vss-distributed-task.tasks"
],
"properties": {
"name": "SpinUpEnvironment"
}
},
{
"description": "Service Endpoint type for all connections",
"id": "endpoint-type",
"properties": {
"authenticationSchemes": [
{
"inputDescriptors": [
{
"description": "Username",
"id": "username",
"inputMode": "textbox",
"name": "Username",
"validation": {
"dataType": "string",
"isRequired": true
}
},
{
"description": "Password",
"id": "password",
"inputMode": "passwordbox",
"isConfidential": true,
"name": "Password",
"validation": {
"dataType": "string",
"isRequired": false
}
}
],
"type": "ms.vss-endpoint.endpoint-auth-scheme-basic"
}
],
"dataSources": [
{
"endpointUrl": "api/configurations/*****/",
"name": "GetCEPlatformType",
"resultSelector": "jsonpath:$.attributes[*].name"
},
{
"endpointUrl": "api/configurations/$(platformTypeId)/",
"name": "GetCEPlatformSize",
"resultSelector": "jsonpath:$.attributes[*].name"
}
],
"displayName": "*****",
"helpMarkDown": "Enter the url and credentials to connect to the endpoint.",
"name": "server",
"url": {
"displayName": "Server URL",
"helpText": "Url for the server to connect to."
}
},
"targets": [
"ms.vss-endpoint.endpoint-types"
],
"type": "ms.vss-endpoint.service-endpoint-type"
}
]
}
Sadly I can't get this to work. The datasources exist in the endpoint, but I'm not sure what to do with the resultSelector.
Does anyone have an idea on how to get this to work? The documentation on this isn't too good.
I'm not familiar with mustache, but in the mustache test tool this seems to work.
Thoughts are appreciated!
Try with this in the first data source:
{
"dataSourceName": "dsList1",
"endpointId": "$(connectedServiceName)",
"target": "list1",
"selector": "jsonpath:$.attributes[*].name",
"keySelector": "jsonpath:$.attributes[*].value.sdk-object.id"
}
Update:
You were trying to use "platformTypeId" in vss-extension.json file while it was defined in task.json, this cause the error message you mentioned in the comment.
By the way, I just noticed that you are using "dataSourceBindings", then you cannot use "selector" and "KeySelector" to parse the result since they are used for "sourceDefinitions".
To achieve the feature you want with "dataSourceBindings", you can definition the endpoint url in the task.json directly instead of defining datasource in endpoint contribution.
So you can move the datasources section in the vss-extension.json file first, and then in the task.json file, change to following:
"dataSourceBindings": [
{
"endpointId": "$(connectedServiceName)",
"endpointURL": "api/configurations/*****/",
"target": "platform",
"resultselector": "jsonpath:$.attributes[*]",
"resultTemplate": "{ \"Value\" : \"{{{value.id}}}\", \"DisplayValue\" : \"{{{name}}}\" }"
},
{
"endpointId": "$(connectedServiceName)",
"endpointURL": "api/configurations/$(platformTypeId)/",
"target": "size"
}
],
In our data we have json fields that include repeated sections, as well as infinite nesting possibilities (the samples I have so far are quite simplistic). After seeing BQ repeated fields and records, I decided to try restructuring the data into repeated record fields, as our use case is related to analytics and then wanted to test out different use cases for the data to see which approach is more efficient (time/cost/difficulty) for the analysis we intend to do on it. I have created a sample json record that I want to upload to BQ, that uses all the features that I think we would need (I have validated is using http://jsonlint.com/):
{
"aid": "6dQcrgMVS0",
"hour": "2016042723",
"unixTimestamp": "1461814784",
"browserId": "BdHOHp2aL9REz9dXVeKDaxdvefE3Bgn6NHZcDQKeuC67vuQ7PBIXXJda3SOu",
"experienceId": "EXJYULQOXQ05",
"experienceVersion": "1.0",
"pageRule": "V1XJW61TPI99UWR",
"userSegmentRule": "67S3YVMB7EMQ6LP",
"branch": [{
"branchId": "1",
"branchType": "userSegments",
"itemId": "userSegment67S3YVMB7EMQ6LP",
"headerId": "null",
"itemMethod": "null"
}, {
"branchId": "1",
"branchType": "userSegments",
"itemId": "userSegment67S3YVMB7EMQ6LP",
"headerId": "null",
"itemMethod": "null"
}],
"event": [{
"eventId": "546",
"eventName": "testEvent",
"eventDetails": [{
"key": "a",
"value": "1"
}, {
"key": "b",
"value": "2"
}, {
"key": "c",
"value": "3"
}]
}, {
"eventId": "547",
"eventName": "testEvent2",
"eventDetails": [{
"key": "d",
"value": "4"
}, {
"key": "e",
"value": "5"
}, {
"key": "f",
"value": "6"
}]
}]
}
I am using BQ interface, to upload this json into a table with the following structure:
[
{
"name": "aid",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "hour",
"type": "INTEGER",
"mode": "NULLABLE"
},
{
"name": "unixTimestamp",
"type": "INTEGER",
"mode": "NULLABLE"
},
{
"name": "browserId",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "experienceId",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "experienceVersion",
"type": "FLOAT",
"mode": "NULLABLE"
},
{
"name": "pageRule",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "userSegmentRule",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "branch",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "branchId",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "branchType",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "itemId",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "headerId",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "itemMethod",
"type": "STRING",
"mode": "NULLABLE"
}
]
},
{
"name": "event",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "evenId",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "eventName",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "eventDetails",
"type": "RECORD",
"mode": "REPEATED",
"fields": [
{
"name": "key",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "value",
"type": "STRING",
"mode": "NULLABLE"
}
]
}
]
}
]
My jobs fail with a
JSON parsing error in row starting at position 0 in <file_id>. Expected key (error code: invalid)
It is possible I can't have multiple nesting in a table, but the error seems more as if there was an issue with parsing the JSON itself. I was able to generate and successfully import a json with a simple repeated record (see example below):
{
"eventId": "546",
"eventName": "testEvent",
"eventDetails": [{
"key": "a",
"value": "1"
}, {
"key": "b",
"value": "2"
}, {
"key": "c",
"value": "3"
}]
}
Any advice is appreciated.
There doesn't seem to be anything problematic with your schema, so BigQuery should be able to load your data with your schema.
First, make sure you are uploading newline-delimited JSON to BigQuery. Your example row has many newline characters in the middle of your JSON row, and the parser is trying to interpret each line as a separate JSON row.
Second, it looks like your schema has the key "evenId" in the "event" record, but your example row has the key "eventId".