How to remove extra column value from jqgrid json data - json

Free Jqgrid has actions column. colmodel:
{"hidden":false,"label":"","name":"_actions","width":72
,"align":"left","template":"actions","fixed":false,"resizable":true,
"formatoptions":{"editbutton":true,"delbutton":true,"delOptions":{"url":"Delete" }}},
{"label":"Nimetus","name":"Nimi","index":"Nimi","editoptions":{"maxlength":80,"size":80 }
It is populated from remote json data like
{"total":1,
"page":1,
"rows":[{"id":"2ARVELDUSARV", "cell":[null,"2ARVELDUSARV"]},
{"id":"ACME","cell":[null,"ACME"]},
{"id":"KAKSKOERA","cell":[null,"KAKSKOERA"]}
]
}
In cell array first column is not used.
If this column is removed, jqgrid does not render data correctly since this column presence is required as placeholder for actions column.
How to fix this so that jqgrid will accept data without first column:
{"total":1,
"page":1,
"rows":[{"id":"2ARVELDUSARV", "cell":[null,"2ARVELDUSARV"]},
{"id":"ACME","cell":["ACME"]},
{"id":"KAKSKOERA","cell":["KAKSKOERA"]}
]
}
Update
I looked for data format change as recommended in answer.
jqgrid data is created from sql select statement in ASP.NET MVC4 using code below. Web API serializes this to format for json for jqgrid automatically.
How to create result which can serialized to propertyname: value format recommended in answer ?
object GetDataForJqGrid() {
IDbConnection conn;
using (var dataReader = DataAccessBase.ExecuteReader(sql.ToString(), out conn,
CommandBehavior.CloseConnection | CommandBehavior.SingleResult,
sql.GetParameters.ToArray()))
{
var rowList = new List<GridRow>();
var pkeys = DatabasePrimaryKey();
while (dataReader.Read())
{
var pkv = new List<object>();
int offset = 1; // required for actions column
var row = new GridRow
{
id = IdHelper.EncodeId(pkv),
cell = new object[dataReader.FieldCount + offset + imageCount]
};
for (int j = 0; j < dataReader.FieldCount; j++)
row.cell[offset + j] = dataReader.GetValue(j);
rowList.Add(row);
}
return new
{
total = rowList.Count() < rows ? page : page + 1, page,
rows = rowList
};
}
public class GridRow
{
public string id;
public object[] cell;
}

The most easy way would be to chanege the format of data returned from the server to use repeatitems: false style of the data. I mean the usage of
{
"total": 1,
"page": 1,
"rows": [
{ "id": "2ARVELDUSARV", "Nimi": "2ARVELDUSARV" },
{ "id": "ACME", "Nimi": "ACME" },
{ "id": "KAKSKOERA", "Nimi": "KAKSKOERA"}
]
}
or, after adding key: true to the definition of the column Nimi
{
"total": 1,
"page": 1,
"rows": [
{ "Nimi": "2ARVELDUSARV" },
{ "Nimi": "ACME" },
{ "Nimi": "KAKSKOERA"}
]
}
instead of
{
"total": 1,
"page": 1,
"rows": [{
"id": "2ARVELDUSARV",
"cell": ["2ARVELDUSARV"]
}, {
"id": "ACME",
"cell": ["ACME"]
}, {
"id": "KAKSKOERA",
"cell": ["KAKSKOERA"]
}]
}
Alternatively one can use jsonReader: { repeatitems: false } event with your current format of data and add jsonmap: "cell.0" property to, which means getting the first element (index 0) from the array cell:
$("#list").jqGrid({
datatype: "json",
url: "andrus.json",
colModel: [
{ label: "", name: "_actions", template: "actions" },
{ label: "Nimetus", name: "Nimi", jsonmap: "cell.0" }
],
iconSet: "fontAwesome",
jsonReader: { repeatitems: false }
});
see the demo.
I personally would recommend you don't use your original format (cell with array of values) and use just the named property with additional id property (if id value is not included in the item already). If you would do use the solution with jsonmap you should be carefully with changing the order of the columns (using remapColumns) and later reloading of data. You could required to update jsonmap values after the changing the column order. Thus I repeat that I recommend you to change format of data returned from the server.
UPDATED: The Updated part of your question formulate absolutely new question which have no relation with jqGrid. It's pure C# problem. Nevertheless I try to answer, because I use C# too.
What you can do with minimal changes of your code is the following: You should add using System.Dynamic; and using System.Linq; first of all. Then you should replace the code inside of using (...) {...} to about the following
var rowList = new List<dynamic>();
while (dataReader.Read()) {
var row = new ExpandoObject() as IDictionary<string, object>;
for (int j = 0; j < dataReader.FieldCount; j++) {
if (!dataReader.IsDBNull(j)) {
row.Add(dataReader.GetName(j), dataReader.GetValue(j));
}
}
rowList.Add(row);
}
Serializing of rowList will produce the names properties. If you know the primary key of the data, then you can add id property with the corresponding value in the same way (using row.Add("id", IdHelper.EncodeId(pkv))). I don't included the part because the code which you posted is not full and pkv is currently always new List<object>(), which is wrong. If the data have composed key (multiple value set is unique) then you can make string concatenation of the keys using '_' (underscore) as the separator.

Related

Restructuring a large amount of values in a JSON file

I have a JSON file with a large amount of the following values:
"values": [
"Foo": 1,
"Bar": 2,
"Baz": 3,
...
],
How do I efficiently convert this into:
"values": [
{
"name": "Foo",
"value": 1
},
{
"name": "Bar",
"value": 2
},
{
"name": "Baz",
"value": 3
},
...
],
Any help would be appreciated!
Okay, so there are two problems with your input. The first is the fact that the given JSON is invalid, so can't directly be parsed. The square brackets after "values" should be curly brackets, to allow for a hash instead of an array:
let raw_old_data =
// Read the old file
fs.readFileSync('./input_data.json').toString()
// Remove all newlines which could interfere with the regex
.replace(/[\r\n]/g, '')
// Replace the square brackets after `"values"` with curly braces
.replace(/"values": \[(.+?)\]/g, '"values": { $1 }');
To convert this (now valid) string to a JSON object, you use JSON.parse:
let old_data = JSON.parse(raw_old_data);
The second problem is that the format in which the values are stored doesn't match your needs. You want to convert from { key: "value" } to [ name: "key", value: "value" ]. The following function can do that, assuming your version of Node supports ES6 (If not, look at Murillo's answer):
function fix_format(obj) {
// This is where we keep the new items in the correct format
let res = [];
// Loop over all values
Object.keys(obj.values).forEach(name => {
let value = obj.values[name];
// Change the format and add to resulting array
res.push({
// If the variable is the same as the key of the hash, it doesn't have to be specified
name,
value,
});
});
return res;
}
All that's then left to do is loop all data from the old object through that function with the Array.map function:
let new_data = old_data.map(fix_format);
And optionally write it back to a file to use with a different program:
fs.writeFileSync('./formatted_data.json', JSON.stringify(data, null, 2));
Note: The 2 in the JSON.stringify function indicates that the resulting JSON should be padded with 2 spaces, to keep it readable.
With ES6:
Object.keys(values).map(name => ({
name,
value: values[name]
}))
Without ES6:
var keys = Object.keys(values);
var newValues = [];
for(var i = 0; i < keys.length; i++){
newValues.push({
name: keys[i],
value: values[keys[i]]
})
}
If your intention is to use the received data i.e obtain data from DB (e.g MSSql, MySql...) using the connection.query(your_custom_sql_query, (err, rows, fields)
for more info:Node.js MySQL Select From Table
I'll recommend you to use:
const myJson = JSON.stringify(rows[0]);

Need help in Append JSON file using ContentMerge in Apache NIFi

I'm trying to merge a JSON file which has multiple objects. Below is my Oringinal JSON file.
{
"applicant": {
"full-name": "Tyrion Lannister",
"mobile-number" : "8435739739",
"email-id" : "tyrionlannister_casterlyrock#gmail.com"
},
"product": {
"product-category" : "Credit Card",
"product-type" : "Super Value Card - Titanium"
}
}
I will get some more JSON data as below from other source.
{
"flags": {
"duplicate-flag" : "No"
"contact-flag" : "Yes"
}
}
My task is to append the new JSON in the old JSON recods as a new object as below.
{
"applicant": {
"full-name": "Tyrion Lannister",
"mobile-number" : "8435739739",
"email-id" : "tyrionlannister_casterlyrock#gmail.com"
},
"product": {
"product-category" : "Credit Card",
"product-type" : "Super Value Card - Titanium"
},
"flags": {
"duplicate-flag" : "No"
"contact-flag" : "Yes"
}
}
Can someone help to guide, how it can be achieved in NiFi ?
I recommend accumulating your components as flowfile attributes, then forming a merged object with an ExecuteScript processor using JavaScript/ECMAScript. Sometimes there's just no substitute for JavaScript. Something like the following might work:
flowFile = session.get();
if (flowFile != null) {
var OutputStreamCallback = Java.type("org.apache.nifi.processor.io.OutputStreamCallback");
var StandardCharsets = Java.type("java.nio.charset.StandardCharsets");
// Get attributes
var applicant = JSON.parse(flowFile.getAttribute("applicant"));
var product = JSON.parse(flowFile.getAttribute("product"));
var flags = JSON.parse(flowFile.getAttribute("flags"));
// Combine
var merged = {
"applicant": applicant,
"product": product,
"flags": flags
};
// Write output content
flowFile = session.write(flowFile, new OutputStreamCallback(function(outputStream) {
outputStream.write(JSON.stringify(merged, null, "\t").getBytes(StandardCharsets.UTF_8));
}));
session.transfer(flowFile, REL_SUCCESS);
}

kendoAutoComplete expects that JSON reponse contains the same propertyname as the search filter

Datasource is defined as:
var KendoDataSource_EmployeeAutoCompleteByFirstName = {
serverFiltering: true,
serverPaging: true,
serverSorting: true,
pageSize: 10,
transport: {
read: {
url: '#Url.Action("GetEmployeesByFirstName", "Employee")',
dataType: "json"
}
}
};
AutoComplete is defined as:
function KendoGridFilterAutoComplete(element, kendoDataSource, textField) {
element.kendoAutoComplete({
minLength: 3,
filter: "startswith",
dataSource: kendoDataSource,
dataTextField: textField
});
}
When using a kendoAutoComplete widget, the filter which is send by the datasource is like:
filter[logic]=and&
filter[filters][0][value]=smith&
filter[filters][0][operator]=startswith&
filter[filters][0][field]=LastName&
filter[filters][0][ignoreCase]=true
The JSON response from the server looks like:
[
{"First":"Bill","LastName":"Smith"},
{"First":"Jack","LastName":"Smith"},
{"First":"ABC","LastName":"Smithy"}
]
This works fine, however as you can see I return multiple entries, so the kendoAutoComplete shows two the same entries (Smith) because the first-name differs.
So what I actually want is do distinct on the server, and return only the possible LastName, as an array of strings like this:
[
"Smith",
"Smithy"
]
However the kendoAutoComplete cannot handle this. It shows "undefined" or an error.
How to solve this ?
I've create the following code:
#region AutoComplete
public virtual IQueryable GetAutoComplete(KendoGridRequest request)
{
// Get filter from KendoGridRequest (in case of kendoAutoComplete there is only 1 filter)
var filter = request.FilterObjectWrapper.FilterObjects.First();
// Change the field-name in the filter from ViewModel to Entity
string fieldOriginal = filter.Field1;
filter.Field1 = MapFieldfromViewModeltoEntity(filter.Field1);
// Query the database with the filter
var query = Service.AsQueryable().Where(filter.GetExpression1<TEntity>());
// Apply paging if needed
if (request.PageSize != null)
{
query = query.Take(request.PageSize.Value);
}
// Do a linq dynamic query GroupBy to get only unique results
var groupingQuery = query.GroupBy(string.Format("it.{0}", filter.Field1), string.Format("new (it.{0} as Key)", filter.Field1));
// Make sure to return new objects which are defined as { "FieldName" : "Value" }, { "FieldName" : "Value" } else the kendoAutoComplete will not display search results.
return groupingQuery.Select(string.Format("new (Key as {0})", fieldOriginal));
}
public virtual JsonResult GetAutoCompleteAsJson(KendoGridRequest request)
{
var results = GetAutoComplete(request);
return Json(results, JsonRequestBehavior.AllowGet);
}
#endregion
Which returns a unique list of anonymous objects which look like { "LastName" : "a" }.

How to create DataTable filled by JSON when number of columns is unknown

I have a table in my web-application, which is created like this:
<table id="mygrid">
<thead>
<th>Col1</th>
<th>Col2</th>
<th>Col3</th>
</thead>
</table>
<script type="text/javascript">
$(document).ready(function() {
window.oTable = $("#mygrid").dataTable({
"bServerSide": true,
"bSort": false,
"sAjaxSource": "#Url.Action("MyFunction", "Events")",
"fnServerParams": function(aoData) {
aoData.push({ name: "arg1", value: "#Model.arg1" });
},
"aoColumns": [
{ "mDataProp": "Column1" },
{ "mDataProp": "Column2" },
{ "mDataProp": "Column3" }
],
"bJQueryUI": true,
"sPaginationType": "full_numbers",
"bProcessing": false
});
I fill it with function that returns JSON result, like this:
ActionResult MyFunction(string arg1, ...)
{
List<string> someList = ...;
object array = eventsList.Select(str => new
{
Column1 = str + "1",
Column2 = str + "2",
Column3 = str + "3"
});
var result = new
{
sEcho = ...,
iTotalRecords = ...,
iTotalDisplayRecords = ...,
aaData = array
};
return Json(result, JsonRequestBehavior.AllowGet);
}
Now I want to generate table dynamically, so I don't know the number of columns in design time. For example:
<table id="mygrid"><thead>
#{
foreach (string col in colNames)
<th>#col</th>
}
</thead></table>
Could you please advise how should I change my Javascript and C# code to fill the table in similar way? Is it possible? I can generate "mDataProp" lines in Javascript code like I generate columns, but how can I create JSON result in C# code?
Added:
I have solved the problem with controller. As I discovered, the List of Dictionaries is serialized to exactly the same JSON as the list of anonymous objects, so I wrote this:
for (int i = 0; i < n; ++i)
{
list.Add(new Dictionary<string, string>());
foreach (int colName in colNames) events[i][colName] = cellValue;
}
var result = new
{
...,
aaData = list
};
return Json(result, JsonRequestBehavior.AllowGet);
Now I have new question. I cannot generate "aoColumns" array using C# loop like I generated tags in HTML code. I tried like this:
"aoColumns": [
#{
for (int i = 0; i < n; ++i)
{
string colName = "Column" + i.ToString();
{ "mDataProp": "#colName" },
}
}
],
but it does not work. How can I do it?
DataTables does not allow to change column dynamically but you can get columns before data and load datatable on its callback function...
$.ajax('url/columns', function(data){
//write table header
var options =
"bProcessing": true,
"bServerSide": true,
"sAjaxSource": getGridDataUrl,
"iDisplayLength": 10,
"aoColumns": //column settingssss
};
$('#myDataTable').dataTable(options);
});
I solved my problems. Generating JavaScript is easier that I thought. The problem was that when I generated the aoColumns array as the C# string and then assigned it to "aoColumns" property like this:
"aoColumns": [
#propertyStr
],
the compiler somehow hid the quotes and braces.
The right way was to do this:
"aoColumns": [
#Html.Raw(propertyStr)
],

Creating CSV view from CouchDB

I know this should be easy, but I just can't work out how to do it despite having spent several hours looking at it today. There doesn't appear to be a straightforward example or tutorial online as far as I can tell.
I've got several "tables" of documents in a CouchDB database, with each "table" having a different value in a "schema" field in the document. All documents with the same schema contain an identical set of fields. All I want to do is be able to view the different "tables" in CSV format, and I don't want to have to specify the list of fieldnames in each schema.
The CSV output is going to be consumed by an R script, so I don't want any additional headers in the output if I can avoid them; just the list of fieldnames, comma separated, with the values in CSV format.
For example, two records in the "table1" format might look like:
{
"schema": "table1",
"field1": 17,
"field2": "abc",
...
"fieldN": "abc",
"timestamp": "2012-03-30T18:00:00Z"
}
and
{
"schema": "table1",
"field1": 193,
"field2": "xyz",
...
"fieldN": "ijk",
"timestamp": "2012-03-30T19:01:00Z"
}
My view is pretty simple:
"all": "function(doc) {
if (doc.schema == "table1") {
emit(doc.timestamp, doc)
}
}"
as I want to sort my records in timestamp order.
Presumably the list function will be something like:
"csv": "function(head, req) {
var row;
...
// Something here to iterate through the list of fieldnames and print them
// comma separated
for (row in getRow) {
// Something here to iterate through each row and print the field values
// comma separated
}
}"
but I just can't get my head around the rest of it.
If I want to get CSV output looking like
"timestamp", "field1", "field2", ..., "fieldN"
"2012-03-30T18:00:00Z", 17, "abc", ..., "abc"
"2012-03-30T19:01:00Z", 193, "xyz", ..., "ijk"
what should my CouchDB list function look like?
Thanks in advance
The list function that works with your given map should look something like this:
function(head,req) {
var headers;
start({'headers':{'Content-Type' : 'text/csv; charset=utf-8; header=present'}});
while(r = getRow()) {
if(!headers) {
headers = Object.keys(r.value);
send('"' + headers.join('","') + '"\n');
}
headers.forEach(function(v,i) {
send(String(r.value[v]).replace(/\"/g,'""').replace(/^|$/g,'"'));
(i + 1 < headers.length) ? send(',') : send('\n');
});
}
}
Unlike Ryan's suggestion, the fields to include in the list are not configurable in this function, and any changes in order or included fields would have to be written in. You would also have to rewrite any quoting logic needed.
Here some generic code that Max Ogden has written. While it is in node-couchapp form, you probably can get the idea:
var couchapp = require('couchapp')
, path = require('path')
;
ddoc = { _id:'_design/csvexport' };
ddoc.views = {
headers: {
map: function(doc) {
var keys = [];
for (var key in doc) {
emit(key, 1);
}
},
reduce: "_sum"
}
};
ddoc.lists = {
/**
* Generates a CSV from all the rows in the view.
*
* Takes in a url encoded array of headers as an argument. You can
* generate this by querying /_list/urlencode/headers. Pass it in
* as the headers get parameter, e.g.: ?headers=%5B%22_id%22%2C%22_rev%5D
*
* #author Max Ogden
*/
csv: function(head, req) {
if ('headers' in req.query) {
var headers = JSON.parse(unescape(req.query.headers));
var row, sep = '\n', headerSent = false, startedOutput = false;
start({"headers":{"Content-Type" : "text/csv; charset=utf-8"}});
send('"' + headers.join('","') + '"\n');
while (row = getRow()) {
for (var header in headers) {
if (row.value[headers[header]]) {
if (startedOutput) send(",");
var value = row.value[headers[header]];
if (typeof(value) == "object") value = JSON.stringify(value);
if (typeof(value) == "string") value = value.replace(/\"/g, '""');
send("\"" + value + "\"");
} else {
if (startedOutput) send(",");
}
startedOutput = true;
}
startedOutput = false;
send('\n');
}
} else {
send("You must pass in the urlencoded headers you wish to build the CSV from. Query /_list/urlencode/headers?group=true");
}
}
}
module.exports = ddoc;
Source:
https://github.com/kanso/kanso/issues/336