I'm in the process of learning the Forge platform. I'm currently using an example (Jigsawify) written by Kean Walmsley because it most accurately describes my goals. I'm running into an issue of getting my file to download from an Azure Storage Account to Forge. The error I receive is "The value for one of the HTTP headers is not in the correct format." My question is how does someone go about troubleshooting HTTP protocol when writing, in this case, a workitem in code? I can put in a breakpoint to view the workitem, but I'm not versed enough to understand where the flaw is in the HTTP header, or even where to find it. Is there a specific property of the workitem I should be looking at? If I could find the HTTP statement, I could test it, but I don't where I should find it.
Or am I just completely off base?
Anyway here's the code. It's a modified version of what Kean wrote:
static void SubmitWorkItem(Activity activity)
{
Console.WriteLine("Submitting workitem...");
CloudStorageAccount storageAccount =
CloudStorageAccount.Parse(Microsoft.Azure.CloudConfigurationManager.GetSetting("StorageConnectionString"));
StorageCredentials crd = storageAccount.Credentials;
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
CloudFileShare ShareRef = fileClient.GetShareReference("000scrub");
CloudFileDirectory rootDir = ShareRef.GetRootDirectoryReference();
CloudFile Fileshare = rootDir.GetFileReference("3359fort.dwg");
// Create a workitem
var wi = new WorkItem()
{
Id = "", // Must be set to empty
Arguments = new Arguments(),
ActivityId = activity.Id
};
if (Fileshare.Exists())
{
wi.Arguments.InputArguments.Add(new Argument()
{
Name = "HostDwg", // Must match the input parameter in activity
Resource = Fileshare.Uri.ToString(),
StorageProvider = StorageProvider.Generic // Generic HTTP download (vs A360)
});
}
wi.Arguments.OutputArguments.Add(new Argument()
{
Name = "Results", // Must match the output parameter in activity
StorageProvider = StorageProvider.Generic, // Generic HTTP upload (vs A360)
HttpVerb = HttpVerbType.POST, // Use HTTP POST when delivering result
Resource = null, // Use storage provided by AutoCAD.IO
ResourceKind = ResourceKind.ZipPackage // Upload as zip to output dir
});
container.AddToWorkItems(wi);
container.SaveChanges();
// Polling loop
do
{
Console.WriteLine("Sleeping for 2 sec...");
System.Threading.Thread.Sleep(2000);
container.LoadProperty(wi, "Status"); // HTTP request is made here
Console.WriteLine("WorkItem status: {0}", wi.Status);
}
while (
wi.Status == ExecutionStatus.Pending ||
wi.Status == ExecutionStatus.InProgress
);
// Re-query the service so that we can look at the details provided
// by the service
container.MergeOption =
Microsoft.OData.Client.MergeOption.OverwriteChanges;
wi = container.WorkItems.ByKey(wi.Id).GetValue();
// Resource property of the output argument "Results" will have
// the output url
var url =
wi.Arguments.OutputArguments.First(
a => a.Name == "Results"
).Resource;
if (url != null)
DownloadToDocs(url, "SGA.zip");
// Download the status report
url = wi.StatusDetails.Report;
if (url != null)
DownloadToDocs(url, "SGA-Report.txt");
}
Any help is appreciated,
Chuck
Azure requires that you specify the x-ms-blob-type header when you upload to a presigned URL. See https://github.com/Autodesk-Forge/design.automation-.net-input.output.sample/blob/master/Program.cs#L167
So, I was able to figure out how to download my file from Azure to Forge using Albert's suggestion of moving to a blob service. Here's the code:
static void SubmitWorkItem(Activity activity)
{
Console.WriteLine("Submitting workitem...");
CloudStorageAccount storageAccount =
CloudStorageAccount.Parse(Microsoft.Azure.CloudConfigurationManager.GetSetting("StorageConnectionString"));
CloudBlobClient BlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = BlobClient.GetContainerReference("000scrub");
CloudBlockBlob blockBlob = cloudBlobContainer.GetBlockBlobReference("3359fort.dwg");
// Create a workitem
var wi = new WorkItem()
{
Id = "", // Must be set to empty
Arguments = new Arguments(),
ActivityId = activity.Id
};
if (blockBlob.Exists())
{
wi.Arguments.InputArguments.Add(new Argument()
{
Name = "HostDwg", // Must match the input parameter in activity
Resource = blockBlob.Uri.ToString(),
StorageProvider = StorageProvider.Generic, // Generic HTTP download (vs A360)
Headers = new System.Collections.ObjectModel.ObservableCollection<Header>()
{
new Header() { Name = "x-ms-blob-type", Value = "BlockBlob" } // This is required for Azure.
}
});
}
wi.Arguments.OutputArguments.Add(new Argument()
{
Name = "Results", // Must match the output parameter in activity
StorageProvider = StorageProvider.Generic, // Generic HTTP upload (vs A360)
HttpVerb = HttpVerbType.POST, // Use HTTP POST when delivering result
Resource = null, // Use storage provided by AutoCAD.IO
ResourceKind = ResourceKind.ZipPackage, // Upload as zip to output dir
});
container.AddToWorkItems(wi);
container.SaveChanges();
// Polling loop
do
{
Console.WriteLine("Sleeping for 2 sec...");
System.Threading.Thread.Sleep(2000);
container.LoadProperty(wi, "Status"); // HTTP request is made here
Console.WriteLine("WorkItem status: {0}", wi.Status);
}
while (
wi.Status == ExecutionStatus.Pending ||
wi.Status == ExecutionStatus.InProgress
);
// Re-query the service so that we can look at the details provided
// by the service
container.MergeOption =
Microsoft.OData.Client.MergeOption.OverwriteChanges;
wi = container.WorkItems.ByKey(wi.Id).GetValue();
// Resource property of the output argument "Results" will have
// the output url
var url =
wi.Arguments.OutputArguments.First(
a => a.Name == "Results"
).Resource;
if (url != null)
DownloadToDocs(url, "SGA.zip");
// Download the status report
url = wi.StatusDetails.Report;
if (url != null)
DownloadToDocs(url, "SGA-Report.txt");
}
What isn't complete is the result section. The ZIP has nothing in it, but hey, baby steps right?
Thanks Albert.
-Chuck
Related
Step 1:
Created GraphServiceClient using Microsoft.Graph 4.9.0 and Microsoft.Graph.Core 2.0.5 SDK
var scopes = new[] { "https://graph.microsoft.com/.default" };
ClientSecretCredential clientSecretCredential = new ClientSecretCredential(tenantId, clientId, clientSecret, new ClientSecretCredentialOptions()
{
AuthorityHost = AzureAuthorityHosts.AzurePublicCloud
});`
GraphServiceClient graphServiceClient = new GraphServiceClient(clientSecretCredential, scopes);
Step 2:
And created a custom schema extension like below.
SchemaExtension schemaExtension = new SchemaExtension()
{
Id = "data1",
Description = "creating test schema extn",
TargetTypes = new List<string>()
{
"User"
},
Properties = new List<ExtensionSchemaProperty>()
{
new ExtensionSchemaProperty()
{
Name ="prop1",
Type ="String"
}
}
};
Step 3:
Updated the Schema extension status to "Available"
var updatedExtn = await graphServiceClient
.SchemaExtensions[schemaExtension.Id].Request()
.UpdateAsync(new SchemaExtension()
{
Status = "Available"
});
Step 4:
Create Class for extension data
public class data1
{
// You must serialize your property names to camelCase if your SchemaExtension describes as such.
[JsonProperty(NullValueHandling = NullValueHandling.Ignore, PropertyName = "prop1", Required = Newtonsoft.Json.Required.Default)]
public string Prop1 { get; set; }
}
Step 5:
Find the User and add the created schema extension to the user
IDictionary<string, object> extensionInstance = new Dictionary<string, object>();
// The below line is not working. but doesn't throw error
extensionInstance.Add(schemaExtension.Id, new data1 { prop1 = "testing" });
var usrCollection = await graphServiceClient.Users
.Request()
.Filter($"userPrincipalNames eq '{adelev_Mail}'")
.GetAsync();
var usr = usrCollection.FirstOrDefault();
if(usr != null)
{
usr.AdditionalData.Add(extensionInstance);
var updatedUser = await graphServiceClient.Users[usr.Id]
.Request()
.UpdateAsync(usr);
}
Step 6:
When you try to retrieve the extension the value is NULL.
User updatedUser = await graphServiceClient.Users[usr.Id].Request()
.Select($"id, {schemaExtension.Id}")
.GetAsync();
But it works with API using Graph Explorer.
PATCH https://graph.microsoft.com/v1.0/users/{userId}
{
"extXXXXXXXX_data1":
{
"prop1" : "testing"
}
}
Please let me know if I'm missing anything here. Any help here is much appreciated.
You should accessing the data on AdditionalData property. Try looking at user.AdditionalData in your result. Here is a screenshot with my example.
Getting User with Schema extension from Graph explorer.
While using the SDK, i access my custom data in user.AdditionalData
Check this thread - Graph SDK and SchemaExtensions for details.
I am trying to create a group with the following dot.net code:
var groupDef = new Group()
{
DisplayName = name,
MailNickname = name + " " + GetTimestamp(),
Description = "Group/Team created for testing purposes",
Visibility = "Private",
GroupTypes = new string[] { "Unified" }, // same for all teams
MailEnabled = true, // same for all teams
SecurityEnabled = false, // same for all teams
AdditionalData = new Dictionary<string, object>()
{
["owners#odata.bind"] = owners.Select(o => $"{graphV1Endpoint}/users/{o.Id}").ToArray(),
["members#odata.bind"] = members.Select(o => $"{graphV1Endpoint}/users/{o.Id}").ToArray(),
}
};
// Create the modern group for the team
Group group = await graph.Groups.Request().AddAsync(groupDef);
I am getting a "Method not allowed." error thrown on the last line shown (Group group = await ...).
The scope parameter for the auth provider contains "Group.Read.All Group.ReadWrite.All".
If I add Group.Create to the scope I get an error stating the scope is invalid. Reducing the scope to just "Group.Create" also gives an error.
It certainly appears that I cannot create a group without Group.Create in the scope, but that throws an error at sign in.
Microsoft.Graph is version 3.19.0
Microsoft.Graph.Core is version 1.22.0
I ended up serializing the object and making the Http call with my own code. Basically, something like this:
string json = JsonConvert.SerializeObject(groupDef, jsonSettings);
Group group = HttpPost<Group>("/groups", json);
No permissions were changed.
I want to display players stats in listview for which I am consuming this api: https://cricapi.com/api/playerStats?apikey=apikey&pid=pid
Output of above api is:
{
"pid": xxxx,
"profile": "profile description",
"imageURL": "https://www.cricapi.com/playerpic/xxxx.jpg",
pid for each player is retrieved from another api:
https://cricapi.com/api/playerFinder?apikey=apikey&name=playerName
Output of above api is:
{
"data": [
{
"pid": xxxx,
"fullName": "Firstname Lastname",
Currently, I am passing hardcoded pid in first api to display player's stats and code for it is:
FetchJson() async {
var response = await http.get(
'https://cricapi.com/api/playerStats?apikey=apikey&pid=1111');
if (response.statusCode == 200) {
String responseBody = response.body;
var responseJson = jsonDecode(responseBody);
pid = responseJson['pid'];
name = responseJson['name'];
playingRole = responseJson['playingRole'];
battingStyle = responseJson['battingStyle'];
country = responseJson['country'];
imageURL = responseJson['imageURL'];
data = responseJson;
var stats = data['data']['batting'];
var testStats = stats['tests'];
var odiStats = stats['ODIs'];
var tStats = stats['T20Is'];
// T20 Stats
matches_t = tStats['Mat'];
runs_t = tStats['Runs'];
half_t = tStats['50'];
century_t = tStats['100'];
highest_t = tStats['HS'];
avg_t = tStats['Ave'];
And I am calling FetchJson() inside initState().
I tried solution given on my similar / earlier question How to fetch api data by passing variables (parameters)?, but that led me to a different path. I cannot implement that solution, since there's no way for me to return pid through first api that will be received by FetchJson().
My question is:
How to retrieve pid from second api (playerFinder) and feed it to first api (playerStats) and how to make use of that pid so that instead of passing hardcoded pid, I can pass pid as variable and can display multiple players stats in UI?
Required code is here : https://pastebin.com/iU8x9U8z
I want to show players stats in UI but currently I am passing hardcoded playerid which is showing me only one player's stats, but I would like to show different players stats.
**********UPDATE *************
As an alternate solution, I am now using list of pids and parsed those using map and passing them to FetchJson() inside for loop, as below:
var playerIds = [{"pid":35320},{"pid":28114},{"pid":28779},{"pid":28763},{"pid":30176},{"pid":7133},{"pid":5390}]
#override
void initState() {
var intIds = playerIds.map<int>((m) => m['pid'] as int).toList();
for (int i = 0; i < intIds.length; i++) {
FetchJson(intIds[i]);
}
}
FetchJson(int ids) async {
print(ids);
var response = await http.get(
'https://cricapi.com/api/playerStats?apikey=apikey&pid=$ids');
....
}
The issue I am now facing with this approach is, its taking last pid from the list and displaying its data in UI repeatedly. The expected output I want to see is: players data for all pids in UI and I am not sure how to achieve this.
Complete referenced code here: https://pastebin.com/kFYBfHuf
One answer is to create Maps from both sets of api's down to desirable player data then use a switch statement as written below similar to a where clause in order to identify matching data.
The big problem is that you need to identify matching data items in both api's. In my example I've assumed it may be a players name or it could be their team and team number, but there has to be something that validates you are looking at differing data points for the same player.
switch(variable_expression) {
case name = full_name: {
// statements;
}
break;
case constant_expr2: {
//statements;
}
break;
default: {
//statements;
}
break;
}
How do a format my json data and/or change my function so that it gets stored as columns in Azure table storage?
I am sending a json string to the IoT hub:
{"ts":"2017-03-31T02:14:36.426Z","timeToConnect":"78","batLevel":"83.52","vbat":"3.94"}
I run the sample function (in the Azure Function App module) to transfer the data from the IoT hub into my storage account:
'use strict';
// This function is triggered each time a message is revieved in the IoTHub.
// The message payload is persisted in an Azure Storage Table
var moment = require('moment');
module.exports = function (context, iotHubMessage) {
context.log('Message received: ' + JSON.stringify(iotHubMessage));
context.bindings.deviceData = {
"partitionKey": moment.utc().format('YYYYMMDD'),
"rowKey": moment.utc().format('hhmmss') + process.hrtime()[1] + '',
"message": JSON.stringify(iotHubMessage)
};
context.done();
};
But in my storage table, it shows up as a single string rather than getting split into columns (as seen in the storage explorer.
How do I get it into columns for ts, timeToConnect, batLevel, and vbat?
In case anyone is looking for a solution in c#:
private static async Task ProcessMessage(string message, DateTime enqueuedTime)
{
var deviceData = JsonConvert.DeserializeObject<JObject>(message);
var dynamicTableEntity = new DynamicTableEntity();
dynamicTableEntity.RowKey = enqueuedTime.ToString("yyyy-MM-dd HH:mm:ss.fff");
foreach (KeyValuePair<string, JToken> keyValuePair in deviceData)
{
if (keyValuePair.Key.Equals("MyPartitionKey"))
{
dynamicTableEntity.PartitionKey = keyValuePair.Value.ToString();
}
else if (keyValuePair.Key.Equals("Timestamp")) // if you are using a parameter "Timestamp" it has to be stored in a column named differently because the column "Timestamp" will automatically be filled when adding a line to table storage
{
dynamicTableEntity.Properties.Add("MyTimestamp", EntityProperty.CreateEntityPropertyFromObject(keyValuePair.Value));
}
else
{
dynamicTableEntity.Properties.Add(keyValuePair.Key, EntityProperty.CreateEntityPropertyFromObject(keyValuePair.Value));
}
}
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("myStorageConnectionString");
CloudTableClient tableClient = storageAccount.CreateCloudTableClient();
CloudTable table = tableClient.GetTableReference("myTableName");
table.CreateIfNotExists();
var tableOperation = TableOperation.Insert(dynamicTableEntity);
await table.ExecuteAsync(tableOperation);
}
How do I get it into columns for ts, timeToConnect, batLevel, and
vbat?
To get these attributes as separate columns in table, you would need to defalte the object and store them separately (currently you are just converting the entire object into string and storing that string).
Please try the following code:
module.exports = function (context, iotHubMessage) {
context.log('Message received: ' + JSON.stringify(iotHubMessage));
var deviceData = {
"partitionKey": moment.utc().format('YYYYMMDD'),
"rowKey": moment.utc().format('hhmmss') + process.hrtime()[1] + '',
};
Object.keys(iotHubMessage).forEach(function(key) {
deviceData[key] = iotHubMessage[key];
});
context.bindings.deviceData = deviceData;
context.done();
};
Please note that I have not tried to execute this code so it may contain some errors.
I have a requirement wherein I have get the document from couchbase.
Following in the Map function that I am using for the same -
function (doc, meta) {
if (meta.type == "json" && doc!=null) {
emit(doc);
}
}
There is no reduce function. Also following is my java code to get the document -
List<URI> hosts = Arrays.asList(
new URI("http://<some DNS with port>/pools")
);
// Name of the Bucket to connect to
String bucket = "Test-Sessions";
// Password of the bucket (empty) string if none
String password = "";
//System.setProperty("viewmode", "development");
// Connect to the Cluster
CouchbaseClient client = new CouchbaseClient(hosts, bucket, password);
String designDoc = "sessions";
String viewName = "by_test";
View view = client.getView(designDoc, viewName);
Query query = new Query();
query.setIncludeDocs(true);
query.setKey(String.valueOf(122));
ViewResponse result = client.query(view, query);
Object object = null;
for(ViewRow row : result) {
if(null != row) {
object = row.getDocument();
}// deal with the document/data
}
System.out.println("Object" + object);
And the data that I have in couchbase is key - "122" and value - "true". But for some reason , I do not get any rows in the ViewResponse. What is going wrong can anyone help?
I don't understand what you are trying to achieve here, you are using a view to get a document by it's key? Key == 122? Why can't you just do client.get(122) ?
If you just need a list of all the keys in your bucket (of which you can use to pull back all documents via include docs) then make your function like so:
function (doc, meta) {
if (meta.type == "json") {
emit();
}
}
The key of the document is always emitted as an ID (viewRow.getId()). You don't need to emit the document, try to emit as little data as possible to keep view sizes small.
If you are needing to manipulate all the documents in your bucket be careful as the size grows, perhaps you'd need to look at pagination to cycle through the results. http://tugdualgrall.blogspot.com.es/
Also once you have the ViewResponse loop over it like so:
for(ViewRow row : result) {
row.getDocument(); // deal with the document/data
}
You don't need to be doing checks for null on the rows.