Loop over tfilefetch while passing different paging token in Talend - json

Well, this scenario may be quite familiar but I couldn't find the solution.
Scenario:
I am making a REST API call through tFileFetch and I get a json out of it. I parse it to get paging token and more result through tflowtoiterate. Now if more result is equals true, I have to call the same tFileFetch component to get the new set of json using new pagination result.
I have to loop through tFileFetch until the 'more result' is false.
My approach:
Access token-pagination-tFileFetch_1->JSON->tflowtoIterate->more result=true->IF [moreresult=true]->tFileFetch_2->JSON->tFlowtoIterate->more result=true->tLOOP [moreresult.equals{true)->tFileFetch_2->
After tFileFetch2 I have used tSetGlobalVar to put pagination as common var to pass to tFileFetch2
I am not sure whether this approach is appreciable or not, please suggest any improvements if any?

I have actually covered it by myself. 'code' Add SubJob Ok from tLoop to tFileFetch and add condition in tLoop which say until your condition matches 'more result' is false. Finally added delimited output with append mode

Related

Getting the value of a particular cell with AJAX

My goal is very simple and I would guess it is a very common goal among web developers.
I am creating a Rails (5.1) application, and I simply want to use AJAX to get the value of a specific cell in a specific row of a specific table in my database (later I am going to use that value to highlight some text on the current page in the user's browser).
I have not been able to find any documentation online explaining how to do this. As I said, it seems like a basic task to ask of jquery and ajax, so I'm confused as to why I'm having so much trouble figuring it out.
For concreteness, say I have a table called 'animals', and I want to get the value of the column 'species' for the animal with 'id' = 99.
How can I construct an AJAX call to query the database for the value of 'species' for the 'animal' with 'id' = 99 .
Though some DBs provide a REST API, what we commonly do is define a route in the app to pull and return data from the DB.
So:
Add a route
Add a controller/action for that route
In that action, fetch the data from the DB and render it in your preferred format
On the client-side, make the AJAX call to that controller/action and do something with the response.

MySQL - get data from custom field with read-only access to db

I have a text field with data, something like:
[{"id":10001,"timeStarted":1355729600733,"projectId":10002,"issueId":"29732,","userName":"tester","assignee":"test","status":"STARTED","shared":True,"name":"Session 4","projectName":"IDS","assigneeDisplayName":"First1 Last1"},
{"id":10002,"timeStarted":1358354188010,"projectId":10002,"issueId":"","userName":"tester","assignee":"test","status":"CREATED","shared":True,"name":"asdf98798","projectName":"IDS","assigneeDisplayName":"First Last"}]
but with much more rows, it may be 30-40, and may be 2 more different statuses (total 4).
Is it possible to extract some data from here having read-only access to DB and only using MySQL query?
For example to count number of items with status "Stated" and with status "created".
Additional conditions may apply, e.g. where id is in definite interval.
Assuming you're using PHP, first you're better off with correcting those unrecognized booleans. You have True where it should have been true (alternatively TRUE for PHP) for it to evaluate the data right.
$jsStr = preg_replace_callback(
'~(?<=[,{[])(".+?"\s*:\s*)(true|false)(?=\s*[,}\]])~i',
create_function('$m','return $m[1].strtolower($m[2]);'),
$jsStr);
Then to be able to process it you want to use the json_decode() function.
$parsed = json_decode($jsStr);
// see the result if you like:
// print_r($parsed);
Ultimately if you want to extract some specific information on the client side (using Javascript) you can use the Array filter() function or a loop if you're not using jQuery. Otherwise you can use the jQuery filter() function with necessary conditions.
If you want to do this in PHP, after the string is parsed into JSON you can use the solutions that apply to Javascript.

sfPropelPager reduce queries

i'm working in a symfony project and using sfPropelPager to show a paged list of elements.
The problem is that with a great amount of data to list (i.e. thousands of registers) it makes a query to the database for each page to show!!!! That means about 100 extra queries in my case, and that is unacceptable.
Showing some of my code: the function that returns the pager object
$pager = new sfPropelPager('MyTable',sfConfig::get('sfPropelPagerLines'));
$c = new Criteria();
$c->add('my_table_field',$value);
$c->addDescendingOrderByColumn('date');
$pager->setCriteria($c);
$pager->init();
return $pager;
So, please, if you know a way to get all the results with only one query, it would be a great solution for my problem. Otherwise i must implement that list with an ajax call for every page the user wants to see
Thank you very much for your time.
I'm not sure to get your problem but, anyway, avoid the use of Criteria. Try to make queries with the ModelCriteria API: http://www.propelorm.org/reference/model-criteria.html.
For each paginated page, a query to the database will be done, this is the standard behavior for all pagers I know. If it's related to related objects (assuming you want to display information from relations), you may want to create a query that links those objects before to paginate, that way you'll get one query per page for all your data to display.
Read this doc for instance: http://www.propelorm.org/documentation/03-basic-crud.html#query_termination_methods
At last i did'nt get a solution for the problem, i had to implement the list via AJAX call, calling to a function that returns the requested page, so at the load of the page, no query for this list is slowing the user experience.
Thank you anyway to help me :)

Does calling ToArray on IQueryable de-attach the entity in LinqToSql?

I have a LinqToSql query that returns an array of Article objects like so:
return db.Articles.ToArray();
I then loop over this array and start to delete some items that meet a certain criteria, for simplicity let's say I delete them all, like so:
foreach (var item in array)
db.articles.DeleteOnSubmit(item);
The call to DeleteOnSubmit(entity) throws an invalid operation exception, it's message says "Can not delete an entity that has not been attached". I modified the code to get the entity first then delete it and it worked just fine. Here's the working code:
db.DeleteOnSubmit(db.Articles.Where(c=>c.Id == item.Id))
Now, I know it would work if I modified the repository to return IQueryable instead of a native array, I just don't understand why? Does ToArray has anything to do with this invalid operation exception?
Thanks.
ps: db is a reference to a DataContext object.
I suspect your using different DataContexts when selecting entities and when submitting changes. If this is the case, the error is natural and would still happen if you returned an IQueryable instead of a native array. Either you Attach an entity to the new data context or you use the same where you selected the initial entities.
Can you put it all in one method and try?
The answer is "No", it doesn't.
Unless you are using differen't db for delete than select (could happenw ithout you realize it) or db.ObjectTrackingEnabled is set to false somewhere.

Custom code in Reporting Services report

In Reporting Services I would like to add a parameter that contains data from a custom code block. Ideally, I would be able to run the following code (this is a simple testing example):
Function GetPeriods() As String()
Dim values As System.Collections.ArrayList =
New System.Collections.ArrayList()
For i as integer = 1 to 24
values.Add(i)
Next
Return values.ToArray()
End Function
and put the following in the "Text Field" of the parameter:
=Code.GetPeriods()
However, when I run the report, the parameter I apply this to is disabled and empty. Is there a different technique that should be used? Or am I doing something wrong?
If you're using SQL 2008 Reporting Services then you can have a look at this page which introduces the concept of using custom assemblies.
If you're using SQL 2005 Reporting Services then this link is the one you want.
It's a mostly trivial thing, simply compile your code into a class library and follow the instructions provided to allow your report to reference it.
You are returning an array item (an array of strings) into a text field. Instead, try returning a plain string. That should work. If you would still like to return an array list, you must basically bind it to a list control in your RDL. You can definitely do that with dataset extensions. However, I am not sure if there is any other easy way. Check the proprties of the list control and see if it allows you to directly bind to an array list.
You can create the same stored procedure on SQL Server and load parameter values from that procedure.
To access your members/functions implemented in custom code of SSRS report you should set the access modifier to "Public":
Public Function GetPeriods() As String
...
see article Writing Custom Code in SQL Server Reporting Services
I've been trying to do this same thing, set a simple list of parameter values from report code. None of the links in any of these answers shows how to do this and after quite a bit of digging around I don't think it's even possible. Yes it is possible to get the values from a database query, from a web service, or from a custom assembly, but each of these creates a lot of overhead compared to getting the list from a simple function call like =Code.GetValues(), where the function uses a For loop to create the values.
msvcyc is correct in pointing out that the parameter is expecting a string value, but the function is returning an array. I changed the return type to Array as suggested by prashant sable, but the select list is still grayed out, it does not work. And coldice is correct in saying that the access modifier should be Public.
In my digging around I found an article by James Kovac from 2005 that pointed out why this is not possible. The Parameters class has a get method, but no set method. In the VS 2008 object browser for SSRS 2008 the object name has changed, but it still does not contain a set method (see Microsoft.ReportingServices.Interfaces.IParameter.Name or .Value).
My current workaround is to just hard code the list of values, but if your value list needs to be dynamic then your only choices are database queries, web services, or custom assemblies. I think the easiest workaround of these three is to get the values from the database engine, as suggested by oleksiy.t, as long as you can write a query to return the value list you want. Your list of integers, or my list of time intervals, would both be easy queries to write. Otherwise you will need to use one of the other two workarounds.
I checked your code. The only thing that's wrong is your function returns String(). When I changed your method signature to return Array, it worked fine, in my report.
Change the signature to Function GetPeriods() As Array
Everything I've seen requires parameters and their respective settings to be part of the RDL.
That being said, if you're going to "hardcode" the values, you could create a dataset just for the report, perhaps in XML, or if it needs to be programmatically driven, do it in a web service.