I'm having a hard time understanding how context providers work in the Orion Context Broker.
I followed the examples in the step-by-step guide written by Jason Fox. However, I still do not exactly get what happens in the background and how the context broker exactly creates the POST from the registration. Here is what I am trying to do:
I do have a WeatherStation that provides sensor data for a neighborhood.
{
"id": "urn:ngsi-ld:WeatherStation:001",
"type": "Device:WeatherStation",
"temperature": {
"type": "Number",
"value": 20.5,
"metadata": {}
},
"windspeed": {
"type": "Number",
"value": 60.0,
"metadata": {}
}
}
Now I like the WeatherStation to be a context provider for all buildings.
{
"id": "urn:ngsi-ld:building:001",
"type": "Building"
}
Here is the registration that I try to use.
{
"id": null,
"description": "Random Weather Conditions",
"provider": {
"http": {
"url": "http://localhost:1026/v2"
},
"supportedForwardingMode": "all"
},
"dataProvided": {
"entities": [
{
"id": "null",
"idPattern": ".*",
"type": "Building",
"typePattern": null
}
],
"attrs": [
"temperature",
"windspeed"
],
"expression": null
},
"status": "active",
"expires": null,
"forwardingInformation": null
}
The context broker accepts both entities and the registration without any error.
Since I have a multi-tenant setup I use one fiware_service for the complete neighborhood but every building would later have a seperate fiware_servicepath. Hence, the weatherstation has a different servicepath than the building. Although I also tried to put them both on the same path.
For now I used the same headers for all entities.
{
"fiware-service": "filip",
"fiware-servicepath": "/testing"
}
Here is the log of the context broker (version: 3.1.0):
INFO#2021-09-23T19:17:17.944Z logTracing.cpp[212]: Request forwarded (regId: 614cd2b511c25270060d873a): POST http://localhost:1026/v2/op/query, request payload (87 bytes): {"entities":[{"idPattern":".*","type":"Building"}],"attrs":["temperature","windspeed"]}, response payload (2 bytes): [], response code: 200
INFO#2021-09-23T19:17:17.944Z logTracing.cpp[130]: Request received: POST /v2/op/query?options=normalized%2Ccount&limit=1000, request payload (55 bytes): {"entities": [{"idPattern": ".*", "type": "Building"}]}, response code: 200
The log says that it receives the request and forwards it as expected. However, as I understand it this would simply point to the same building entity again. Hence, it is somehow a circular forwarding. I also cannot tell anything about the headers of the request.
I do not understand how the forwarded request from the building can actually query the weather station for information. When I query my building I still only receive the entity with no own properties:
{
"id": "urn:ngsi-ld:building:001",
"type": "Building"
}
I also tried to vary the url of the registration but with no success.
Is this scenario actually possible with the current implementation? It would be very useful
Is there any example for this including also the headers?
I know that I could simply use reference but that would put more work on the user.
Thanks for any help on this.
It is messy, but you could achieve this via a subscription. Hold the weather station as a separate entity in the context broker and poll or push updates into the entity. The subscription would fire whenever the data changes and make two NGSI requests:
Find all entities which have a Relationship servicedBy=WeatherStationX
Run an upsert on all entities to add a Property to each entity:
{
"temperature" : {
"type" : "Property",
"value" : 7,
"unitCode": "CEL",
"observedAt": "XXXXX",
"providedBy": "WeatherStation1"
}
}
Where observedAt comes either from the payload of the weather station or the notification timestamp.
Within the existing IoT Agents, provisioning the link attribute allows a device to propagate measures to a second entity (e.g. this Thermometer entity is measuring temperature for an associated Building entity)
{
"entity_type": "Device",
"resource": "/iot/d",
"protocol": "PDI-IoTA-UltraLight",
..etc
"attributes": [
{"object_id": "l", "name": "temperature", "type":"Float",
"metadata":{
"unitCode":{"type": "Text", "value" :"CEL"}
}
}
],
"static_attributes": [
{
"name": "controlledAsset",
"type": "Relationship",
"value": "urn:ngsi-ld:Building:001",
"link": {
"attributes": ["temperature"],
"name": "providedBy",
"type": "Building"
}
}
]
}
At the moment the logic just links direct one-to-one, but it would be possible to raise a PR to check for an Array and update multiple entities in an upsert - the relevant section of code is here
I'm getting the
"API Version 2: Failed to parse JSON response string with 'INVALID_ARGUMENT' error: \": Cannot find field.\"."
and so am hoping you can help me, as I cannot get Implicit Invoking working for my app.
I've attached screenshots of the Welcome Intent setup on DialogFLow as well as pasted the Google Cloud fail logs here:
Google Cloud FAIL LOGS:
{
"request": {
"conversationToken": "",
"debugLevel": 1,
"inputType": "KEYBOARD",
"locale": "en-US",
"mockLocation": {
"city": "Mountain View",
"coordinates": {
"latitude": 37.421980615353675,
"longitude": -122.08419799804688
},
"formattedAddress": "Googleplex, Mountain View, CA 94043, United States",
"zipCode": "94043"
},
"query": "ask follow-up to log a call",
"surface": "GOOGLE_HOME"
},
"response": {
"conversationToken": "GidzaW11bG...",
"response": "Follow up isn't responding right now. Try again soon.",
"visualResponse": {
"visualElements": []
}
},
"debug": {
"agentToAssistantDebug": {
"agentToAssistantJson": "{\"message\":\"Failed to parse Dialogflow response into AppResponse, exception thrown with message: Empty speech response\",\"apiResponse\":{\"id\":\"875b123f-bcb2-4f45-b2f4-10193a6132c3\",\"timestamp\":\"2018-01-30T06:33:52.773Z\",\"lang\":\"en-us\",\"result\":{},\"status\":{\"code\":200,\"errorType\":\"success\"},\"sessionId\":\"1517294032097\"}}"
},
"assistantToAgentDebug": {
"assistantToAgentJson": "{\"user\":{\"userId\":\"ABwppHHDarg-rpyAaSt0hm8TZaycr30xhUpQcRKfchRbXriPUKmmzi_BqQrpXInBGyGmgfF4yIEiMX0jInJ8rQ\",\"accessToken\":\"{\\\"access_token\\\":\\\"00D3600000uHY5!AQoAQAcyjXEI.J.5EnB4.R.EdNXBKlymGOI4I6PPJVb465uyQLxnbQDyjPHtD0uE0W1RMdhnhgXLEpr8qPIMOTcnvsfKH0j\\\",\\\"signature\\\":\\\"etNbI3erh1iYmsTqCRicfKKJknRtGnCb1esvufdg7g=\\\",\\\"scope\\\":\\\"refresh_token web api\\\",\\\"instance_url\\\":\\\"https://follow-up-ed.my.salesforce.com\\\",\\\"id\\\":\\\"https://login.salesforce.com/id/00D3600000uHY5EAM/0053600000L9ePAAS\\\",\\\"token_type\\\":\\\"Bearer\\\",\\\"issued_at\\\":\\\"1517293582207\\\"}\",\"locale\":\"en-US\",\"lastSeen\":\"2018-01-30T06:26:14Z\"},\"conversation\":{\"conversationId\":\"151729432097\",\"type\":\"NEW\"},\"inputs\":[{\"intent\":\"Log Call by Business\",\"rawInputs\":[{\"inputType\":\"VOICE\",\"query\":\"ask follow-up to log a call\"}],\"arguments\":[{\"name\":\"trigger_query\",\"rawText\":\"log a call\",\"textValue\":\"log a call\"},{\"name\":\"Type\",\"rawText\":\"call\",\"textValue\":\"call\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.SCREEN_OUTPUT\"}]}]}",
"curlCommand": "curl -v 'https://api.api.ai/api/integrations/google?token=1ee421e5c9504f5b995ce9df62f7d275' -H 'Content-Type: application/json;charset=UTF-8' -H 'Google-Actions-API-Version: 2' -H 'Authorization: eyJhbGciOiJSUzI1NiIsImtpZCI6IjI2YzAxOGIyMzNmZTJlZWY0N2ZlZGJiZGQ5Mzk4MTcwZmM5YjI5ZDgifQ.eyJhdWQiOiJzYWxlc2ZvcmNlLWEwOWVkIiwiYXpwIjoiMTE4NDUyMTUyMjE5LW1zZ2VldXBkaGU5YWp0MzZpNnJxbHJtdTExZGQ1Y2gyLmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwiZXhwIjoxNTE3Mjk0MTUyLCJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJqdGkiOiI5ZTJjOGI1OGQzZjRiNTc3MmQ0MDc2ZjIwNWIyMGEyYTdmYTQ3MGY3IiwiaWF0IjoxNTE3Mjk0MDMyLCJuYmYiOjE1MTcyOTM3MzJ9.WnIVnDFUESVaLYOITacgDzS_4qPCat8YEMTHgsSzvHUzJNNeZ3oaDqUZ5lwECI0jfp2qHpW7Il5Tv1iDyPScOeggvm2cZZa4OXdr7PLr362eT5wyOZnsFWlrU8n4KlNZsuKl_uMSxhftP0qD2eUuwAKQ0bbAPurApTF_iY8Gzh2V0QYpn1Ol06bJLJo8B9z4lXmFuGfzldjWXojQ1eA794nItQKtt2X7tiSfBOoXOL2fpT8omy293vcMI-dWtf5FY1nH9_bN1GKHCbOQ4LiJTB__r7PVHOcq-SNb9_CtKaXvrLo9EW3CyfnHlc0SGv1UUo9akbvDZEwZG9B3gH7K1g' -A 'Mozilla/5.0 (compatible; Google-Cloud-Functions/2.1; +http://www.google.com/bot.html)' -X POST -d '{\"user\":{\"userId\":\"ABwppHHDarg-rpyAaSt0hm8TZaycr30xhUpQcRKfchRbXriPUKmmzi_BqQrpXInBGyGmgfF4yIEiMX0jInJ8rQ\",\"accessToken\":\"{\\\"access_token\\\":\\\"00D36000000uHY5!AQoAQAcyjXEI.J.5EnB4.R.EdNXBKlymGOI4I6PPJVb465uyQLxnbQDyjPHtD0uE0W1RMdhnhgXL8Epr8qPIMOTcnvsfKH0j\\\",\\\"signature\\\":\\\"etNbI3erh1iYmsTqCRicfKKJknRtGnC4b1esvufdg7g=\\\",\\\"scope\\\":\\\"refresh_token web api\\\",\\\"instance_url\\\":\\\"https://follow-up-dev-ed.my.salesforce.com\\\",\\\"id\\\":\\\"https://login.salesforce.com/id/00D36000000uHY5EAM/00536000000L9ePAAS\\\",\\\"token_type\\\":\\\"Bearer\\\",\\\"issued_at\\\":\\\"1517293582207\\\"}\",\"locale\":\"en-US\",\"lastSeen\":\"2018-01-30T06:26:14Z\"},\"conversation\":{\"conversationId\":\"1517294032097\",\"type\":\"NEW\"},\"inputs\":[{\"intent\":\"Log Call by Business\",\"rawInputs\":[{\"inputType\":\"VOICE\",\"query\":\"ask follow-up to log a call\"}],\"arguments\":[{\"name\":\"trigger_query\",\"rawText\":\"log a call\",\"textValue\":\"log a call\"},{\"name\":\"Type\",\"rawText\":\"call\",\"textValue\":\"call\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.SCREEN_OUTPUT\"}]}]}'"
},
"sharedDebugInfo": [
{
"name": "ResponseValidation",
"subDebugEntry": [
{
"debugInfo": "API Version 2: Failed to parse JSON response string with 'INVALID_ARGUMENT' error: \": Cannot find field.\".",
"name": "UnparseableJsonResponse"
}
]
}
]
},
"errors": [
[
{
"debugInfo": "API Version 2: Failed to parse JSON response string with 'INVALID_ARGUMENT' error: \": Cannot find field.\".",
"name": "UnparseableJsonResponse"
}
]
]
}
Google Cloud SUCCESS LOGS:
{
"request": {
"conversationToken": "",
"debugLevel": 1,
"inputType": "KEYBOARD",
"locale": "en-US",
"mockLocation": {
"city": "Mountain View",
"coordinates": {
"latitude": 37.421980615353675,
"longitude": -122.08419799804688
},
"formattedAddress": "Googleplex, Mountain View, CA 94043, United States",
"zipCode": "94043"
},
"query": "Talk to Follow-Up",
"surface": "GOOGLE_HOME"
},
"response": {
"conversationToken": "CiZDIzVhNm...",
"expectUserResponse": true,
"response": "Follow-Up is Online. Would you like to log a call, add a note or create a reminder?",
"visualResponse": {
"visualElements": []
}
},
"debug": {
"agentToAssistantDebug": {
"agentToAssistantJson": "{\"conversationToken\":\"[\\\"defaultwelcomeintent-followup\\\",\\\"sessiondata\\\"]\",\"expectUserResponse\":true,\"expectedInputs\":[{\"inputPrompt\":{\"richInitialPrompt\":{\"items\":[{\"simpleResponse\":{\"textToSpeech\":\"Follow-Up is Online. Would you like to log a call, add a note or create a reminder?\",\"displayText\":\"Follow-Up is Online. Would you like to log a call, add a note or create a reminder in Salesforce?\"}}],\"suggestions\":[{\"title\":\"Log a call\"},{\"title\":\"Add a note\"},{\"title\":\"Create a reminder\"},{\"title\":\"Exit\"}]}},\"possibleIntents\":[{\"intent\":\"assistant.intent.action.TEXT\"},{\"intent\":\"907faabf-33a2-49ff-a368-263d01e812fc\"},{\"intent\":\"a4bdf329-5430-4833-827d-620ed00e4288\"},{\"intent\":\"68d2be51-880d-44dd-9939-1c09089b5fbf\"},{\"intent\":\"d9f56992-363e-410d-a00e-1e9a59ed613d\"},{\"intent\":\"66de3f32-c11a-4e36-80b1-64582bc1ef69\"},{\"intent\":\"befff469-3348-49d7-b9d6-5bbe7eef2aa6\"},{\"intent\":\"a813eeba-2a35-4f20-8fdb-09f8e9b08b7c\"},{\"intent\":\"e9add01c-1067-4c14-a48c-979f4934e192\"},{\"intent\":\"1dbe2f1a-a12e-4f22-9092-11dafce0cf26\"},{\"intent\":\"94c36f7f-8fff-4c55-b6f4-f5556fa83d8a\"},{\"intent\":\"d503d957-6dea-4d40-b161-adb779df2f66\"},{\"intent\":\"040b1388-4aaa-4e3b-8af9-67c111bd9cc7\"},{\"intent\":\"494afd87-d03a-49a6-a5da-340061c9121a\"}],\"speechBiasingHints\":[\"$Classification\",\"$Type\",\"$BusinessName\",\"$Task\",\"$FollowUp\",\"$Calendar\"]}],\"responseMetadata\":{\"status\":{},\"queryMatchInfo\":{\"queryMatched\":true,\"intent\":\"6bb1ee8e-8a54-4422-b20e-de50839c40bc\"}}}"
},
"assistantToAgentDebug": {
"assistantToAgentJson": "{\"user\":{\"userId\":\"ABwppHHDarg-rpyAaSt0hm8TZaycr30xhUpQcRKfchRbXriPUKmmzi_BqQrpXInBGyGmgfF4yIEiMX0jInJ8rQ\",\"accessToken\":\"{\\\"access_token\\\":\\\"00D36000000uHY5!AQoAQAcyjXEI.J.5EnB4.R.EdNXBKlymGOI4I6PPJVb465uyQLxnbQDyjPHtD0uE0W1RMdhnhgXL8Epr8qPIMOTcnvsfKH0j\\\",\\\"signature\\\":\\\"etNbI3erh1iYmsTqCRicfKKJknRtGnC4b1esvufdg7g=\\\",\\\"scope\\\":\\\"refresh_token web api\\\",\\\"instance_url\\\":\\\"https://follow-up-dev-ed.my.salesforce.com\\\",\\\"id\\\":\\\"https://login.salesforce.com/id/00D36000000uHY5EAM/00536000000L9ePAAS\\\",\\\"token_type\\\":\\\"Bearer\\\",\\\"issued_at\\\":\\\"1517293582207\\\"}\",\"locale\":\"en-US\",\"lastSeen\":\"2018-01-30T06:33:52Z\"},\"conversation\":{\"conversationId\":\"1517294128019\",\"type\":\"NEW\"},\"inputs\":[{\"intent\":\"actions.intent.MAIN\",\"rawInputs\":[{\"inputType\":\"VOICE\",\"query\":\"Talk to Follow-Up\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"}]}]}",
"curlCommand": "curl -v 'https://api.api.ai/api/integrations/google?token=1ee421e5c9504f5b995ce9df62f7d275' -H 'Content-Type: application/json;charset=UTF-8' -H 'Google-Actions-API-Version: 2' -H 'Authorization: eyJhbGciOiJSUzI1NiIsImtpZCI6IjI2YzAxOGIyMzNmZTJlZWY0N2ZlZGJiZGQ5Mzk4MTcwZmM5YjI5ZDgifQ.eyJhdWQiOiJzYWxlc2ZvcmNlLWEwOWVkIiwiYXpwIjoiMTE4NDUyMTUyMjE5LW1zZ2VldXBkaGU5YWp0MzZpNnJxbHJtdTExZGQ1Y2gyLmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwiZXhwIjoxNTE3Mjk0MjQ4LCJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJqdGkiOiIxY2M5ODYxOGVlNzI5NzQ4YjdiZTczZmI3NjY2ZDM3YzllYjk0ZDQ4IiwiaWF0IjoxNTE3Mjk0MTI4LCJuYmYiOjE1MTcyOTM4Mjh9.S28YQqlVGJEK0xXfgZ-rxgwdXlfBMBoV3UXBIsvN6SeKK3PYvkvShqpKB5Icr89TSvzS7riq2H9YBwzrvPCRQscrH_3tVRyQL2EsCQhnpGQhnvVdO7rwE2b1-xnoAj7dy9D8EOuNskOK7V2Qek7u-_ZdU9r7w4W_saVwhyFtGMjfXFjgPgRurZq3Ei3-fnZ9GJ-3RqlgGU8FiSSFXheBgSvwWq9Ai7QeiVDaGcjxovX1qZXhkhu9W5lPPdTE1tUYVZ3CZcJfE5YqQiPJfQj6OJwguRg5Qb3aKMHlBV50Pb5Ux302ZWT_L49lVxk-cFM0-oTeNo5YLuFzQIwCgpcaoA' -A 'Mozilla/5.0 (compatible; Google-Cloud-Functions/2.1; +http://www.google.com/bot.html)' -X POST -d '{\"user\":{\"userId\":\"ABwppHHDarg-rpyAaSt0hm8TZaycr30xhUpQcRKfchRbXriPUKmmzi_BqQrpXInBGyGmgfF4yIEiMX0jInJ8rQ\",\"accessToken\":\"{\\\"access_token\\\":\\\"00D36000000uHY5!AQoAQAcyjXEI.J.5EnB4.R.EdNXBKlymGOI4I6PPJVb465uyQLxnbQDyjPHtD0uE0W1RMdhnhgXL8Epr8qPIMOTcnvsfKH0j\\\",\\\"signature\\\":\\\"etNbI3erh1iYmsTqCRicfKKJknRtGnC4b1esvufdg7g=\\\",\\\"scope\\\":\\\"refresh_token web api\\\",\\\"instance_url\\\":\\\"https://follow-up-dev-ed.my.salesforce.com\\\",\\\"id\\\":\\\"https://login.salesforce.com/id/00D36000000uHY5EAM/00536000000L9ePAAS\\\",\\\"token_type\\\":\\\"Bearer\\\",\\\"issued_at\\\":\\\"1517293582207\\\"}\",\"locale\":\"en-US\",\"lastSeen\":\"2018-01-30T06:33:52Z\"},\"conversation\":{\"conversationId\":\"1517294128019\",\"type\":\"NEW\"},\"inputs\":[{\"intent\":\"actions.intent.MAIN\",\"rawInputs\":[{\"inputType\":\"VOICE\",\"query\":\"Talk to Follow-Up\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"}]}]}'"
}
},
"errors": []
}
And the DialogFLow JSON blob here:
FAIL DIALOGFLOW
{
"id": "60acafcf-ceb2-485c-b3f6-663407832e1c",
"timestamp": "2018-01-30T06:39:56.632Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "ask follow up to log a call",
"action": "input.welcome",
"actionIncomplete": false,
"parameters": {},
"contexts": [
{
"name": "defaultwelcomeintent-followup",
"parameters": {},
"lifespan": 2
},
{
"name": "sessiondata",
"parameters": {},
"lifespan": 1
}
],
"metadata": {
"intentId": "6bb1ee8e-8a54-4422-b20e-de50839c40bc",
"webhookUsed": "true",
"webhookForSlotFillingUsed": "false",
"webhookResponseTime": 340,
"intentName": "Default Welcome Intent"
},
"fulfillment": {
"messages": [
{
"type": "suggestion_chips",
"platform": "google",
"suggestions": [
{
"title": "\"Log a call\""
},
{
"title": "\"Add a note\""
},
{
"title": "\"Create a reminder\""
}
]
},
{
"type": 0,
"speech": "Follow-Up isn't responding right now. Please try again later."
}
]
},
"score": 0.5
},
"status": {
"code": 200,
"errorType": "success",
"webhookTimedOut": false
},
"sessionId": "24069c06-d6c0-4723-a0d9-fa284884d023"
}
SUCCESS DIALOGFLOW
{
"id": "210514e2-6702-4d30-9689-eb3279fdde6d",
"timestamp": "2018-01-30T06:41:19.848Z",
"lang": "en",
"result": {
"source": "agent",
"resolvedQuery": "talk to follow up",
"action": "input.welcome",
"actionIncomplete": false,
"parameters": {},
"contexts": [
{
"name": "defaultwelcomeintent-followup",
"parameters": {},
"lifespan": 2
},
{
"name": "sessiondata",
"parameters": {},
"lifespan": 1
}
],
"metadata": {
"intentId": "6bb1ee8e-8a54-4422-b20e-de50839c40bc",
"webhookUsed": "true",
"webhookForSlotFillingUsed": "false",
"webhookResponseTime": 237,
"intentName": "Default Welcome Intent"
},
"fulfillment": {
"messages": [
{
"type": "suggestion_chips",
"platform": "google",
"suggestions": [
{
"title": "\"Log a call\""
},
{
"title": "\"Add a note\""
},
{
"title": "\"Create a reminder\""
}
]
},
{
"type": 0,
"speech": "Follow-Up isn't responding right now. Please try again later."
}
]
},
"score": 1
},
"status": {
"code": 200,
"errorType": "success",
"webhookTimedOut": false
},
"sessionId": "24069c06-d6c0-4723-a0d9-fa284884d023"
}
Google Actions Testing Logs FAIL
{
"request": {
"conversationToken": "",
"debugLevel": 1,
"inputType": "KEYBOARD",
"locale": "en-US",
"mockLocation": {
"city": "Mountain View",
"coordinates": {
"latitude": 37.421980615353675,
"longitude": -122.08419799804688
},
"formattedAddress": "Googleplex, Mountain View, CA 94043, United States",
"zipCode": "94043"
},
"query": "ask follow up to log a call",
"surface": "GOOGLE_HOME"
},
"response": {
"conversationToken": "GidzaW11bG...",
"response": "Follow up isn't responding right now. Try again soon.",
"visualResponse": {
"visualElements": []
}
},
"debug": {
"agentToAssistantDebug": {
"agentToAssistantJson": "{\"message\":\"Failed to parse Dialogflow response into AppResponse, exception thrown with message: Empty speech response\",\"apiResponse\":{\"id\":\"49c16313-09e6-4431-9765-37095a19e3bb\",\"timestamp\":\"2018-01-31T01:45:41.734Z\",\"lang\":\"en-us\",\"result\":{},\"status\":{\"code\":200,\"errorType\":\"success\"},\"sessionId\":\"1517363140452\"}}"
},
"assistantToAgentDebug": {
"assistantToAgentJson": "{\"user\":{\"userId\":\"ABwppHGKOxCmVg53MPiRI_5NnIt0vUjDf0Hqwxgm9pTNnH8vOquUymEX8T2OtFC1NA48-X4JiKBTk0an2wTYVw\",\"accessToken\":\"{\\\"access_token\\\":\\\"00D36000000uHY5!AQoAQFHFHGAYHInuT1.FtcUSN7k81w1tgkEh.ijyqq1Pw3UqtlCM6SGi_qTrvFDAvPBG673Lgr119bpIUEUNuOnC4XF2d7o2\\\",\\\"refresh_token\\\":\\\"5Aep861QbHyftz0nI9mDOXbILtyhnTRY2lNmFwvaIHwc6w_JBasCpmEoOoWUo5W9asHeibIB9HbomiclZ2P_6pk\\\",\\\"signature\\\":\\\"nUZJxL8SFUVOScLjP6c5ydpjwL8iLRyHZ+xOyehfLhc=\\\",\\\"scope\\\":\\\"refresh_token web api\\\",\\\"instance_url\\\":\\\"https://follow-up-dev-ed.my.salesforce.com\\\",\\\"id\\\":\\\"https://login.salesforce.com/id/00D36000000uHY5EAM/00536000000L9ePAAS\\\",\\\"token_type\\\":\\\"Bearer\\\",\\\"issued_at\\\":\\\"1517363133651\\\"}\",\"locale\":\"en-US\",\"lastSeen\":\"2018-01-31T01:44:48Z\"},\"conversation\":{\"conversationId\":\"1517363140452\",\"type\":\"NEW\"},\"inputs\":[{\"intent\":\"Log Call by Name\",\"rawInputs\":[{\"inputType\":\"VOICE\",\"query\":\"ask follow up to log a call\"}],\"arguments\":[{\"name\":\"trigger_query\",\"rawText\":\"log a call\",\"textValue\":\"log a call\"},{\"name\":\"Type\",\"rawText\":\"call\",\"textValue\":\"call\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"}]}]}",
"curlCommand": "curl -v 'https://api.api.ai/api/integrations/google?token=1ee421e5c9504f5b995ce9df62f7d275' -H 'Content-Type: application/json;charset=UTF-8' -H 'Google-Actions-API-Version: 2' -H 'Authorization: eyJhbGciOiJSUzI1NiIsImtpZCI6IjI2YzAxOGIyMzNmZTJlZWY0N2ZlZGJiZGQ5Mzk4MTcwZmM5YjI5ZDgifQ.eyJhdWQiOiJzYWxlc2ZvcmNlLWEwOWVkIiwiYXpwIjoiMTE4NDUyMTUyMjE5LW1zZ2VldXBkaGU5YWp0MzZpNnJxbHJtdTExZGQ1Y2gyLmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwiZXhwIjoxNTE3MzYzMjYwLCJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJqdGkiOiI5NDQxOTU4YzVkYWY3ZjgyODkzOTdhYWRkNjgzMzNmNDQ1ZjY2NDFkIiwiaWF0IjoxNTE3MzYzMTQwLCJuYmYiOjE1MTczNjI4NDB9.Fc5SWQakdTwfhW3x1NGgbwzT4pjec5GW2fOVc0Vh9aZ8XseAOeMWDRdiVK5Q4ApRMfJWC239P2LYvXGzW2tqZBPjxS_LKdD_6GbRJaNAYMs2SyiTuJmLs0G9RqFzXja-0mz8q_lejpWSuLgjzO7H136lHjR4d6bFiNS0ec3UIzIrgx7oaGCtRFuEtj4af82llyztaKkDAtwpankG01CBWm_sX_FmFD9svLUk7u22NA3KXsCM23fasLcmietBEj6LktnfiR6Tk85mk4n5FYi4VJ7KHzsfgIPC2zkmJVaAc1OczLvV_qtLb_9hoM_3k8jLMXK0n1oypQHZSpCvElOzwQ' -A 'Mozilla/5.0 (compatible; Google-Cloud-Functions/2.1; +http://www.google.com/bot.html)' -X POST -d '{\"user\":{\"userId\":\"ABwppHGKOxCmVg53MPiRI_5NnIt0vUjDf0Hqwxgm9pTNnH8vOquUymEX8T2OtFC1NA48-X4JiKBTk0an2wTYVw\",\"accessToken\":\"{\\\"access_token\\\":\\\"00D36000000uHY5!AQoAQFHFHGAYHInuT1.FtcUSN7k81w1tgkEh.ijyqq1Pw3UqtlCM6SGi_qTrvFDAvPBG673Lgr119bpIUEUNuOnC4XF2d7o2\\\",\\\"refresh_token\\\":\\\"5Aep861QbHyftz0nI9mDOXbILtyhnTRY2lNmFwvaIHwc6w_JBasCpmEoOoWUo5W9asHeibIB9HbomiclZ2P_6pk\\\",\\\"signature\\\":\\\"nUZJxL8SFUVOScLjP6c5ydpjwL8iLRyHZ+xOyehfLhc=\\\",\\\"scope\\\":\\\"refresh_token web api\\\",\\\"instance_url\\\":\\\"https://follow-up-dev-ed.my.salesforce.com\\\",\\\"id\\\":\\\"https://login.salesforce.com/id/00D36000000uHY5EAM/00536000000L9ePAAS\\\",\\\"token_type\\\":\\\"Bearer\\\",\\\"issued_at\\\":\\\"1517363133651\\\"}\",\"locale\":\"en-US\",\"lastSeen\":\"2018-01-31T01:44:48Z\"},\"conversation\":{\"conversationId\":\"1517363140452\",\"type\":\"NEW\"},\"inputs\":[{\"intent\":\"Log Call by Name\",\"rawInputs\":[{\"inputType\":\"VOICE\",\"query\":\"ask follow up to log a call\"}],\"arguments\":[{\"name\":\"trigger_query\",\"rawText\":\"log a call\",\"textValue\":\"log a call\"},{\"name\":\"Type\",\"rawText\":\"call\",\"textValue\":\"call\"}]}],\"surface\":{\"capabilities\":[{\"name\":\"actions.capability.AUDIO_OUTPUT\"},{\"name\":\"actions.capability.MEDIA_RESPONSE_AUDIO\"}]},\"isInSandbox\":true,\"availableSurfaces\":[{\"capabilities\":[{\"name\":\"actions.capability.SCREEN_OUTPUT\"},{\"name\":\"actions.capability.AUDIO_OUTPUT\"}]}]}'"
},
"sharedDebugInfo": [
{
"name": "ResponseValidation",
"subDebugEntry": [
{
"debugInfo": "API Version 2: Failed to parse JSON response string with 'INVALID_ARGUMENT' error: \": Cannot find field.\".",
"name": "UnparseableJsonResponse"
}
]
}
]
},
"errors": [
[
{
"debugInfo": "API Version 2: Failed to parse JSON response string with 'INVALID_ARGUMENT' error: \": Cannot find field.\".",
"name": "UnparseableJsonResponse"
}
]
]
}
Google Cloud Logs SUCCESS Post
{
insertId: "epw74xfuhr98a"
labels: {
channel: "preview"
source: "JSON_RESPONSE_VALIDATION"
}
logName: "projects/salesforce-a09ed/logs/actions.googleapis.com%2Factions"
receiveTimestamp: "2018-01-31T02:07:08.026897050Z"
resource: {
labels: {
action_id: ""
project_id: "salesforce-a09ed"
version_id: ""
}
type: "assistant_action"
}
severity: "DEBUG"
textPayload: "Sending request with post data: {"user":{"userId":"ABwppHGKOxCmVg53MPiRI_5NnIt0vUjDf0Hqwxgm9pTNnH8vOquUymEX8T2OtFC1NA48-X4JiKBTk0an2wTYVw","accessToken":"{\"access_token\":\"00D36000000uHY5!AQoAQPlRWyuv4mA0oNyUUcUWBr1PRzsaQB0NGFR3f9CD6j4Z_vHGSHCcRtGyOet5F_jEdvo.ykj1es.d2y.d7lFwanc1x1en\",\"refresh_token\":\"5Aep861QbHyftz0nI9mDOXbILtyhnTRY2lNmFwvaIHwc6w_JBZrL0KU0BZ3nLp.5Q1bBdSVn.zL53m3QsG0ZW1J\",\"signature\":\"UncpON0wsMm9OfoudbJw4liWBdWktyCat6lxArAO3iU=\",\"scope\":\"refresh_token web api\",\"instance_url\":\"https://follow-up-dev-ed.my.salesforce.com\",\"id\":\"https://login.salesforce.com/id/00D36000000uHY5EAM/00536000000L9ePAAS\",\"token_type\":\"Bearer\",\"issued_at\":\"1517364271309\"}","locale":"en-US","lastSeen":"2018-01-31T02:05:39Z"},"conversation":{"conversationId":"1517364426179","type":"NEW"},
"inputs":[{"intent":"actions.intent.MAIN","rawInputs":[{"inputType":"VOICE","query":"open follow-up"}]}],
"surface":{"capabilities":[{"name":"actions.capability.MEDIA_RESPONSE_AUDIO"},{"name":"actions.capability.AUDIO_OUTPUT"}]},"isInSandbox":true,"availableSurfaces":[{"capabilities":[{"name":"actions.capability.SCREEN_OUTPUT"},{"name":"actions.capability.AUDIO_OUTPUT"}]}]}."
timestamp: "2018-01-31T02:07:06.435444389Z"
trace: "projects/118452152219/traces/1517364426179"
}
Google Cloud Logs FAIL Post
{
insertId: "5kahhnfa51fzb"
labels: {
channel: "preview"
source: "JSON_RESPONSE_VALIDATION"
}
logName: "projects/salesforce-a09ed/logs/actions.googleapis.com%2Factions"
receiveTimestamp: "2018-01-31T02:08:18.510485438Z"
resource: {
labels: {
action_id: ""
project_id: "salesforce-a09ed"
version_id: ""
}
type: "assistant_action"
}
severity: "DEBUG"
textPayload: "Sending request with post data: {"user":{"userId":"ABwppHGKOxCmVg53MPiRI_5NnIt0vUjDf0Hqwxgm9pTNnH8vOquUymEX8T2OtFC1NA48-X4JiKBTk0an2wTYVw","accessToken":"{\"access_token\":\"00D36000000uHY5!AQoAQPlRWyuv4mA0oNyUUcUWBr1PRzsaQB0NGFR3f9CD6j4Z_vHGSHCcRtGyOet5F_jEdvo.ykj1es.d2y.d7lFwanc1x1en\",\"refresh_token\":\"5Aep861QbHyftz0nI9mDOXbILtyhnTRY2lNmFwvaIHwc6w_JBZrL0KU0BZ3nLp.5Q1bBdSVn.zL53m3QsG0ZW1J\",\"signature\":\"UncpON0wsMm9OfoudbJw4liWBdWktyCat6lxArAO3iU=\",\"scope\":\"refresh_token web api\",\"instance_url\":\"https://follow-up-dev-ed.my.salesforce.com\",\"id\":\"https://login.salesforce.com/id/00D36000000uHY5EAM/00536000000L9ePAAS\",\"token_type\":\"Bearer\",\"issued_at\":\"1517364271309\"}","locale":"en-US","lastSeen":"2018-01-31T02:08:07Z"},"conversation":{"conversationId":"1517364498025","type":"NEW"},
"inputs":[{"intent":"ReminderIntent","rawInputs":[{"inputType":"VOICE","query":"ask follow up to create a reminder"}],"arguments":[{"name":"trigger_query","rawText":"create a reminder","textValue":"create a reminder"},{"name":"Task","rawText":"creates","textValue":"create"},{"name":"Task","rawText":"reminders","textValue":"reminder"}]}],
"surface":{"capabilities":[{"name":"actions.capability.AUDIO_OUTPUT"},{"name":"actions.capability.MEDIA_RESPONSE_AUDIO"}]},"isInSandbox":true,"availableSurfaces":[{"capabilities":[{"name":"actions.capability.AUDIO_OUTPUT"},{"name":"actions.capability.SCREEN_OUTPUT"}]}]}."
timestamp: "2018-01-31T02:08:18.282416932Z"
trace: "projects/118452152219/traces/1517364498025"
}
Screen shots:
DialogFlow Welcome Intent
DialogFlow LogCall Intent
DialogFlow / Assistant Integration Tab
Google Actions Simulator error
Google Cloud Error Logs
The issue is because the intent you are trying to match "Log call by name" which expects an input context "sessionData", will never be matched because invoking your app using a deep-link "ask follow up to log a call" doesn't send that context in the request. That explains why your default response (Text response) for that intent isn't returned.
To fix it you need to:
Remove the input contexts from the intents you intend to use as deep-links OR duplicate those intents and remove the input contexts from the duplicates.
Best practice 1: Have a fallback intent that has no input contexts, which will catch any unmatched in-dialog queries.
Best practice 2: Have a fallback intent that specifically handles unmatched deep-links at invocation time. (See image below)
The agentToAssistantJson field has an encoded JSON entry for result which typically suggests that your webhook isn't returning anything. Normally, Dialogflow returns the static messages you've defined in the Intent in this case, but it sounds like that isn't happening here. Verify that the URL you're using for fulfillment is correct and that it is returning valid JSON.
In this case, I think there are a few things that could be causing the problem.
The phrase "Ask follow up to log a call" should be triggering the Log Call by Name Intent it looks like. But this Intent isn't listed as one of the implicit invocation Intents. The Assistant might be passing this off to Dialogflow, and Dialogflow, finding no match, returns nothing.
But even if it did match the phrasing for the Log Call by Name Intent, and that Intent was an implicit invocation Intent, there are two other elements of the Intent that seem strange.
The first is that the Intent is expecting an input context of sessionData. But since this is meant to be used as an initial Intent, there can be no input context. Dialogflow may be told by the Assistant that this is the matching Intent, and then reject it because the input context doesn't match.
Similarly, the second oddity is that you're looking for an event called CALL_BY_NAME. Events generally override any phrases that may be spoken - they're meant to capture non-textual activities (the WELCOME intent, for example, or an option being selected, or the user saying nothing). Unless you're triggering the event (which you can do), you probably wouldn't want it. As above, I'm wondering if the Assistant is telling Dialogflow this is the Intent to use, but Dialogflow isn't getting the event, so rejects it and sends back nothing.
tl;dr
There are three possible things to look at and fix:
Make sure the Intent is listed as an implicit invocation Intent in the Assistant Integration.
Remove the incoming context.
Remove the event.