Well,
I am trying to use remote object (Amfphp) in a project which is using httpservice. I heard it will make my application faster. But when i tried Amfphp in a datagrid for testing purpose i found it takes even more time then httpservice. Here is what i have done so far.
AS-3 code to call php function:
public function init():void{
var params:Array = new Array();
params.push("1234");
_amf = new RemoteObject;
_amf.destination = "dummyDestination";
_amf.endpoint = "http://insight2.ultralysis.com/Amfphp/Amfphp/";//http://insight2.ultralysis.com
_amf.source = "manager1";
_amf.addEventListener(ResultEvent.RESULT, handleResult);
_amf.addEventListener(FaultEvent.FAULT, handleFault);
_amf.init(params);
}
public function handleResult(event:ResultEvent):void{
myGrid.dataProvider = event.result.grid;
}
And the php function to fetch data from mysql database:
class output{
public $grid;
public $week;
}
function form()
{
$arrayOut = new output();
$arrayOut->grid = $this->gridValue();
$arrayOut->week= $this->getAllWeek($this->ThisYear);
return $arrayOut;
}
Everything works fine. But it takes almost 5 seconds to fetch and render 280 rows of data.
Can anyone please help me make it as fast as it should ? I've already tried the optimization tips of silexlabs
I used packet sniffer and stats are the following. It says Latency is consuming most of the time about 5 sec. What's that latency? Need help guys. Please:
Try to use amfphp 1.9.
amfphp 2.x version is unfortunately slower than 1.9
Related
I have a large json file (not in size, but in elements). It has 30000 JSON elements and I am trying to produce Entities from it as it reads it.
So far I have it reading the file with Guzzle, and it ends up producing about 1500 entities before it crashes. I feel that I must be doing this the wrong way.
Here is my code:
public function generateEntities(Request $request, $number)
{
$client = new \GuzzleHttp\Client();
$request = new \GuzzleHttp\Psr7\Request('GET', 'http://www.example.com/file.json');
$promise = $client->sendAsync($request)->then(function ($response) {
$batchSize = 20;
$i = 0;
foreach (json_decode($response->getBody()) as $entityItem) {
$entity = new Entity();
$entity->setEntityItem($entityItem->string);
$em = $this->getDoctrine()->getManager();
$em->persist($entity);
if (($i % $batchSize) === 0) {
$em->flush(); // Executes all updates
}
$i++;
}
$em->flush();
});
$promise->wait();
return $this->redirect($this->generateUrl('show_entities'));
}
I worked out from research that I should be clearing the Entity Manager frequently, so I added in batch sizing etc to flush it every 20 entities created. This did help but is not enough to load the full 30000 files.
Maybe I am completely wrong and should be handling it a different way?
Is it possible for someone to please point me in the right direction, I am happy to work it out on my own I am just a little unsure where to proceed from here.
Thank you!
You could improve your process act in two ways:
1) increment the time limit of the execution of the controller action with the function set_time_limit, so put this as first line of the controller:
public function generateEntities(Request $request, $number)
{
set_time_limit(0); // set to zero, no time limit is imposed
2) Free much memory is possible for each interaction flush the data to the database and detach/free memory as follow:
$em->persist($entity);
$em->flush($entity);
$em->detach($entity);
$em->clear($entity);
unset($entity);
Hope this help
The script is running out of memory because every one of the 30000 entities is managed in memory. You need to detach the entities from the manager periodically to make sure they are "garbage collected". Use $em->clear(); in your batch flushing block to ensure memory isn't exhausted. See the Doctrine page on batch operations for more information.
Keep in mind though, that $em->clear() will detach all entities from the manager, not just those you are using in this loop.
I am new to LDA and mallet. I have the following query
I tried running Mallet-LDA with the command line and by setting the --random-seed to a fixed value, I was able to get consistent results for multiple runs of the algorithm
However, I did try with the Mallet-Java-API and everytime I run the program I get different output.
I did google around and found out that random-seed needs to be fixed and I have it fixed in my java code. I still am getting different results.
Could anyone let me know what other parameters do I need to consider for consistent results (when run multiple times)
I might want to add that train-topics when ran multiple times(command line) yields same result. However, when I rerun import-dir and then run train-topics, the results do not match with previous one. (Probably as expected).
I am ok with running import-dir just once and then experiment with different number of topics and iterations by running train-topics.
Similarly, what needs to be changed/ kept constant if I want to replicate the same when I use Java-Api.
I was able to solve this.
I will respond in detail here:
There are two ways in which Mallet could be run.
a. Command mode
b. Using Java API
To get consistent results for different runs, we need to fix the 'random seed' and in the command line we have an option of setting it. We have no surprises there.
However, while using APIs, though we have an option of setting 'random seed', we need to know that it needs to be done at proper point, else it does not work. (see code)
I have pasted the code here which would create a model(read InstanceList) file from the data
and then we could use the same model file and set the random seed and see to it that we get consistent(read same) results every time we run.
Creating and saving model for later use.
Note: Follow this link to know the format of input file.
http://mallet.cs.umass.edu/ap.txt
public void getModelReady(String inputFile) throws IOException {
if(inputFile != null && (! inputFile.isEmpty())) {
List<Pipe> pipeList = new ArrayList<Pipe>();
pipeList.add(new Target2Label());
pipeList.add(new Input2CharSequence("UTF-8"));
pipeList.add(new CharSequence2TokenSequence());
pipeList.add(new TokenSequenceLowercase());
pipeList.add(new TokenSequenceRemoveStopwords());
pipeList.add(new TokenSequence2FeatureSequence());
Reader fileReader = new InputStreamReader(new FileInputStream(new File(inputFile)), "UTF-8");
CsvIterator ci = new CsvIterator (fileReader, Pattern.compile("^(\\S*)[\\s,]*(\\S*)[\\s,]*(.*)$"),
3, 2, 1); // data, label, name fields
InstanceList instances = new InstanceList(new SerialPipes(pipeList));
instances.addThruPipe(ci);
ObjectOutputStream oos;
oos = new ObjectOutputStream(new FileOutputStream("Resources\\Input\\Model\\Model.vectors"));
oos.writeObject(instances);
oos.close();
}
}
Once model file is saved, this uses the above saved file to generate topics
public void applyLDA(ParallelTopicModel model) throws IOException {
InstanceList training = InstanceList.load (new File("Resources\\Input\\Model\\Model.vectors"));
logger.debug("InstanceList Data loaded.");
if (training.size() > 0 &&
training.get(0) != null) {
Object data = training.get(0).getData();
if (! (data instanceof FeatureSequence)) {
logger.error("Topic modeling currently only supports feature sequences.");
System.exit(1);
}
}
// IT HAS TO BE SET HERE, BEFORE CALLING ADDINSTANCE METHOD.
model.setRandomSeed(5);
model.addInstances(training);
model.estimate();
model.printTopWords(new File("Resources\\Output\\OutputFile\\topic_keys_java.txt"), 25,
false);
model.printDocumentTopics(new File ("Resources\\Output\\OutputFile\\document_topicssplit_java.txt"));
}
I'm using HttpClient from WP8 and do a Post request. I know the call may take long time as I'm actually simulating slow network scenarios. Therefore I set the HttpClient.Timeout accordingly to 5 minutes.
However, I get a Timeout at around 60s. I believe the Timeout is not working.
I believe there is an issue with this for WP as stated in this question:
HttpClient Portable returns 404 notfound on WP8.
They use a workaround but that does not applies to my scenario. I do actually want to wait for long time.
My questions:
1) Is it a bug/issue of HttpClient for WP8 or I'm not setting it properly?
2) Do you think of a workaround still using HttpClient?
I've read that maybe HttpWebRequest is an option. However, I believe HttpClient should be ideal for this 'simple' scenario.
My code is simple:
private static async Task<HttpResponseMessage> PostAsync(Uri serverUri, HttpContent httpContent)
{
var client = new HttpClient();
client.Timeout = TimeSpan.FromMinutes(5);
return await client.PostAsync(serverUri, httpContent).ConfigureAwait(false);
}
The server receives the request and while is processing it, the client aborts.
UPDATE: The HttpResponseMessage returned by HttpClient.PostAsyn is this "{StatusCode: 404, ReasonPhrase: '', Version: 0.0, Content: System.Net.Http.StreamContent, Headers: { Content-Length: 0 }}". As I said, the server is found and is receiving the data and processing it.
After some search and some tests I've came to the conclusion that the problem is Windows Phone itself and that it has a 60 seconds timeout (irrespective of the HttpClient) and that cannot be changed to my knowledge. See http://social.msdn.microsoft.com/Forums/en-US/faf00a04-8a2e-4a64-b1c1-74c52cf685d3/httpwebrequest-60-seconds-timeout.
As I'm programming the server as well, I will try the advice by Darin Rousseau in the link provided above, specifically to send an OK and then do some more processing.
UPDATE: The problem seems to be the Windows Phone emulator as stated here:
http://social.msdn.microsoft.com/forums/wpapps/en-us/6c114ae9-4dc1-4e1f-afb2-a6b9004bf0c6/httpclient-doesnt-work-on-windows-phone?forum=wpdevelop. In my experience the tcp connection times-out if it doesn't hear anything for 60s.
Therefore my solution is to use the Http header characters as a way of keep alive. The first line Http header response always starts with HTTP/1.0. So I send the characters one by one with a delay <60s between them. Of course, if the response gets ready, everything that is left is sent right away. This buys some time, for instance if using a delay of 50s per 9 character we get about 450s.
This is a project for my degree so I wouldn't recommend it for production.
By the way, I also tried with other characters instead the sub string of the header, for instance space character, but that results in a http protocol violation.
This is the main part of the code:
private const string Header1 = #"HTTP/1.0 ";
private int _keepAliveCounter = 0;
private readonly object _sendingLock = new object();
private bool _keepAliveDone = true;
private void StartKeepAlive()
{
Task.Run(() => KeepAlive());
}
/// <summary>
/// Keeps the connection alive sending the first characters of the http response with an interval.
/// This is a hack for Windows Phone 8 that need reponses within 60s interval.
/// </summary>
private void KeepAlive()
{
try
{
_keepAliveDone = false;
_keepAliveCounter = 0;
while (!_keepAliveDone && _keepAliveCounter < Header1.Length)
{
Task.Delay(TimeSpan.FromSeconds(50)).Wait();
lock (_sendingLock)
{
if (!_keepAliveDone)
{
var sw = new StreamWriter(OutputStream);
sw.Write(Header1[_keepAliveCounter]);
Console.Out.WriteLine("Wrote keep alive char '{0}'", Header1[_keepAliveCounter]);
_keepAliveCounter++;
sw.Flush();
}
}
}
_keepAliveCounter = 0;
_keepAliveDone = true;
}
catch (Exception e)
{
// log the exception
Console.Out.WriteLine("Error while sending keepalive: " + e.Message);
}
}
Then, the actual processing happens in a different thread.
Comments and critics are appreciated.
It is possible that you are hitting the timeout of the network stream. You can change this by doing,
var handler = new WebRequestHandler();
handler.ReadWriteTimeout= 5 * 60 * 1000;
var client = new HttpClient(handler);
client.Timeout = TimeSpan.FromMinutes(5);
return await client.PostAsync(serverUri, httpContent).ConfigureAwait(false);
The default on the desktop OS is already 5mins. However, it is possible that on Windows Phone it has been reduced by default.
I am trying to port some code from a Windows form application to WP8, and have run into some issues regarding asynchronous calls.
The basic idea is to do some UAG authentication. In the Windows form code, I do a GET on the portal homepage and wait for the cookies. I then pass these cookies into a POST request to the validation URL the UAG server. It all works fine in the form, since all the steps are sequential and synchronous.
Now, when I started porting this to WP8, first thing I noticed was that GetResponse() wasn't available, instead I had to use BeginGetResponse(), which is asynchronous and calls a callback function. This is no good for me, since I need to ensure this step finishes before I do the POST
My Windows form code looks like this (taken from http://usingnat.net/sharepoint/2011/2/23/how-to-programmatically-authenticate-to-uag-protected-sharep.html):
private void Connect()
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(this.Url);
request.CookieContainer = new CookieContainer();
request.UserAgent = this.UserAgent;
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
//Get the UAG generated cookies from the response
this.Cookies = response.Cookies;
}
}
private void ValidateCredentials()
{
//Some code to construct the headers goes here...
HttpWebRequest postRequest = (HttpWebRequest)WebRequest.Create(this.ValidationUrl);
postRequest.ContentType = "application/x-www-form-urlencoded";
postRequest.CookieContainer = new CookieContainer();
foreach (Cookie cookie in this.Cookies)
{
postRequest.CookieContainer.Add(cookie);
}
postRequest.Method = "POST";
postRequest.AllowAutoRedirect = true;
using (Stream newStream = postRequest.GetRequestStream())
{
newStream.Write(data, 0, data.Length);
}
using (HttpWebResponse response = (HttpWebResponse)postRequest.GetResponse())
{
this.Cookies = response.Cookies;
}
public CookieCollection Authenticate()
{
this.Connect();
this.ValidateCredentials();
return this.Cookies;
}
The thing is this code relies on synchronous operation (first call Connect(), then ValidateCredentials() ), and it seems WP8 does not support that for Web requests. I could combine the two functions into one, but that won't solve my problem fully since later on this needs to be expanded to access resources behind the UAG, so it would need a modular design.
Is there a way to "force" synchronization?
Thanks
You can still continue your steps in the call back function using the asynchronous model. Or you can use the new HttpClient which can be used with the await keyword so you can program your stuff in a synchronous way.
You can get HttpClient through nuget
install-package Microsoft.Net.Http
I am trying to render emails in a windows service host.
I use RazorEngine 3 forked by coxp which has support for Razor 2.
https://github.com/coxp/RazorEngine/tree/release-3.0/src
This works fine for a couple of emailtemplates but there is one causing me problems.
#model string
Click here to enter a new password for your account.
This throws a CompilationException: The name 'WriteAttribute' does not exist in the current context. So passing in a string as model and putting it in the href-attribute causes problems.
I can make it work by changing this line by:
#Raw(string.Format("Klik hier.", #Model))
but this makes the template very unreadable and harder to pass along to a marketing department for further styling.
I like to add that referencing the RazorEngine by using a Nuget package is not a solution since it is based on Razor 1 and somewhere along the process the DLL for system.web.razor gets replaced by version 2 which breaks any code using RazorEngine. It seems more interesting to use Razor 2 to benefit from the new features and to be up to date.
Any suggestions on how to fix this would be great. Sharing your experiences is also very welcome.
UPDATE 1
It seems like calling SetTemplateBaseType might help, but this method does not exist anymore, so I wonder how to be able to bind the templatebasetype?
//Missing method in the new RazorEngine build from coxp.
Razor.SetTemplateBaseType(typeof(HtmlTemplateBase<>));
I use Windsor to inject the template service rather than using the Razor object. Here is a simplified part of the code that shows how to set the base template type.
private static ITemplateService CreateTemplateService()
{
var config = new TemplateServiceConfiguration
{
BaseTemplateType = typeof (HtmlTemplateBase<>),
};
return new TemplateService(config);
}
RazorEngine 3.1.0
Little bit modified example based on coxp answer without the injection:
private static bool _razorInitialized;
private static void InitializeRazor()
{
if (_razorInitialized) return;
_razorInitialized = true;
Razor.SetTemplateService(CreateTemplateService());
}
private static ITemplateService CreateTemplateService()
{
var config = new TemplateServiceConfiguration
{
BaseTemplateType = typeof (HtmlTemplateBase<>),
};
return new TemplateService(config);
}
public static string ParseTemplate(string name, object model)
{
InitializeRazor();
var appFileName = "~/EmailTemplates/" + name + ".cshtml";
var template = File.ReadAllText(HttpContext.Current.Server.MapPath(appFileName));
return RazorEngine.Razor.Parse(template, model);
}