I'm using the "phpsoda" library and trying to query the city of Seattle's permits dataset through the Socrata API (SODA). Data: https://data.seattle.gov/Permitting/Land-Use-Permits/ht3q-kdvx
They have several columns such as the ones I'm selecting below, but in the "AppliedDate" and "IssuedDatae" columns, there may or may not be data in that column.
So when I try to look at the array and arrange this into an HTML table, I'm getting some arrays (rows) that have fewer elements (columns) than other rows. This turns out to make it difficult to display since I don't know which columns are missing in the array (row).
I'm wondering if when I make the query, that those empty fields will look they seem in the visualized table on their site or when I export a CSV. Those columns in the query will return into the array element an empty string ("") instead so my rows and columns will come out all filled with values.
$soql->select("PermitNum", "AppliedDate", "IssuedDate", "Description", "OriginalAddress1")
->where("PermitClass = 'Multifamily' OR PermitClass = 'Commercial')
->limit(20);
$results = $ds->getDataset($soql);
Data would look something like...
print_r($results);
Array[0] -> [Description]=>"XXXXXXX", [PermitNum]=>"123456"
Array[1] -> [Description]=>"XXXXXXX", [PermitNum]=>"234567", [AppliedDate]=>"XX/XX/XXXX"
So the first row is missing the "AppliedDate" column just because it's not in the data.
Will I need to just go through this manually in the results array using a loop and checking column names and inserting an empty string if the loop doesn't find a column?
Following my own advice, I was able to just check for each row key if a specific key was missing (array_key_exists function), then I'd fill it in with "" if it returned false.
This seemed to work.
I have same problem. Socrata API will skip empty field or null value. Means, if field is null or empty, the result will not show field-name:'', instead, the result will just missing this, that cause your shorter row.
This is annoying bug, I have to fix it by my own. If I found it missing field, I will have to add field-name:'' to the result json, that will fix your shorter row problem. Make equal length row.
Related
I want to query the value for JSON field role, which will always be inside the last JSON element in the column ip in the table named employeeRole. The whole JSON data for that column would be like below,
[{"submitterNumber":null,"name":{"fullName":"*****","parts":null},"role":"developer:56987t63-y876-3678","department":"****","Code":null,"pin":null,"Nnis":null},{"submitterNumber":null,"name":{"fullName":"******","parts":null},"role":"approver:38675r40-i456-0934","department":"****","Code":null,"ipn":null,"Nnis":null},{"submitterNumber":null,"name":{"fullName":"*****","parts":null},"role":"tester:76590w23-o895-3045","department":null,""Code":"****","ipn":null,"isni":null}]
I want to query only the guid(i.e the 56987t63-y876-3678) part of this field "role":"developer:56987t63-y876-3678" from the above JSON element.
Each row for the column IP would have a very similar JSON element as above. But the problem is not all row would have the final JSON field {"submitterNumber":null,"name":{"fullName":"*****","parts":null},"role":"tester:76590w23-o895-3045","department":null,""Code":"****","ipn":null,"isni":null} in that same position. So employeeRole.ip -> 3 -> role would not work always in all scenarios, as something else would be in the position 3 in other rows. But it will always be the final JSON field.
So, I expect an answer that I will be able to use the result of this query to check with a subquery, whether the result of this query(i.e the guid value) is in the result on the subquery. If it does that's what will be queried. The subquery part is done, but I am not sure how to the query the above JSON field and join in with the subquery.
Any help is highly appreciated. Thanks.
The -> operator you're using also supports relative indexing:
Extracts n'th element of JSON array (array elements are indexed from zero, but negative integers count from the end).
So to get the role of the last object in the array, just write
employeeRole.ip -> -1 ->> role
Lets say I have a search form on my site which generates a query string to filter results, eg. mysite/search?field1=value1&field2=value2&field3=value3
The user enters the following into the search fields:
Field 1 = Cat
Field 2 = Black
Field 3 = Stray
Given the nature of a query string, all three field values would have to be present in an item being searched, in order for it to be recognised as a match.. right?
Is there a way to either make the '&' an 'OR', so that any matching field will return a result.
Or
Is there a way to match the nearest result? Ie. Remove string pairs until a match is found, or in some way, find the next closest result.
For example. If the user enters 'Cat', 'Black' and 'Stray' and there is an item that includes all three values, it returns that result (standard response). If there isn't an item that includes all three values, let's say there's only an item that has 'Cat' and 'Stray', it recognizes there are no items containing all three values, so it looks for two field value matches instead?
Happy to consider any ideas to prevent "no items found" and at least render something rather than nothing.
It depends on what you're using for filtering but general idea is that after you filter the data you check if it is empty and if it is you remove one filter do the filtering again. Something like this for example:
result = filter(data,filters);
while(!result || filters.length > 0){
filters.pop();
result = filter(data, filters);
}
Not working code, just general idea.
I have a Coldfusion Component which returns my search results from a query in JSON using serializeJSON(myquery).
The results returned dont have the ROWCOUNT before the COLUMNS, if i do return the results with the ROWCOUNT then it messes everything up.
What is the best way to display the recordcount without using the ROWCOUNT value?
Unless there is something you are not telling us, you do not need to include an extra row count. It can be derived from the result.
By default serializeJSON(queryObject) returns a structure with two keys: DATA and COLUMNS (both arrays). DATA represents the rows in the query. So to obtain the query row count, simply check the length of the DATA array.
If the serialized object is a query object, you should be able to deserialize the JSON and get the array length of records from the result structure's data key.
So,
arrayLen(deserializeJSON(result).data)
I'm trying to create an expression with a missing column. as in the column is not returned from the query but does exist in the list of fields.
The problem I'm having is that every time I do an expression and one of the parameters is from a field that doesn't have a return from query the query fails silently. the following example is looking at a field "test" that, as I said, exists in the list of fields but is not returned from the query, how can I have this is statement send "alert"??
=IIF(Fields!test.IsMissing,"alert",Fields!test.Value)
The reason that I don't return the field is that the columns are dependent on the parameters I enter in the procedure (so they could be used or not depending on what the user is asking)
Thank you
From reading your question I think, you have dynamic columns which will be returns conditionally
So what you should do is,
1) Create the parameter in the parameter list and set it as the internal and assign the field value to that parameter, suppose the parameter is dyanmicfiledvalue
2) Change your expression as,
=IIF(IsNothing(Parameters!dyanmicfiledvalue.value),"alert", Parameters!dyanmicfiledvalue.value )
that should do it. Let me know in case of issues.
Or if you want to check null or empty value for that column just change as
=IIF(IsNothing(Fields!test.value),"alert",Fields!test.value)
I have a text file with more than one table's data in it (different column counts). I import the whole row as one column. Based on a conditional split, the rows are dispersed to their correct flow. I use a script component to split the single column values (row) into the correct columns for that table and give it as output columns. All of this is working fine, and data looks fine.
My problem comes in with some numeric fields. When a numeric field has no values in it, it ends up in the table with another column's numeric value.
I have put data viewers everywhere, in not one of them there is data for the column that should be empty. When I look in the table itself, there it is... data from another column.
It is not the mappings, I checked it a dozen times.
It is not the names that are the same or something like that.
There is no data according to dataviewers anywhere in the load process.
There is no hidden code anywhere.
I droped and recreated the table.
I displayed a messagebox with the column's (that is supposed to be empty) assigned "column value", and no data, like expected.
I used a derived column, same result, no data in dataviewers, but eventualy data in the table.
I also created another test table with those numeric fields as varchar. When I do this, the column is empty (like expected). When I change it to numeric, the field is populated again. (If it was the other way around I could understand).
What can be te reason for this? It is driving me insane.
EDIT
Script code:
//C#
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
ASCIIEncoding enc = new System.Text.ASCIIEncoding();
char[] seperator = { '|' };
Byte[] ByteBlob;
String[] ColumnValue;
ByteBlob = Row.Column0.GetBlobData(0, (int)(Row.Column0.Length));
ColumnValue = enc.GetString(ByteBlob).Split(seperator);
Row.OutputColumn0 = ColumnValue[0];
Row.OutputColumn1 = ColumnValue[1];
///etc
Just to give an example of what it does, this is what a row would look like in a sence.
Column names:
Source|Tablename|Value1|Value2|Description|Value3|Description2|Value4
Actual Data:
ABC|Revenue|123,456|729,537|MisterX||None|
Data in Table:
ABC|Revenue|123,456|729,537|MisterX|729,537|None|729,537
try using Row.ColumnX_IsNull , for example if (Row.Column0_IsNull) {youroutputcolumn=null} else {...}