I'm using LDAvis for topic modeling and trying to use the as.gist option to create a gist. When serVis executes there is a timeout in curl::curl_fetch_memory after about 10 seconds. If I immediately execute serVis again I get a different error 'Problems parsing JSON' and from then on whenever serVis is run that same error recurs.
If I start all over with a fresh workspace the same behavior occurs. The first time serVis is run, curl::curl_fetch_memory times out after about 10 seconds. Subsequent executions return 'Problems parsing JSON'.
If I don't use the as.gist option it works fine, but of course doesn't create a gist.
Very rarely, it works and a gist is created. If I change parameters to reduce the size of the JSON object it usually works, which makes me think it may be related to size.
I have explored the various RCurlOptions timeout settings. Currently, they are set as
options(RCurlOptions = list(cainfo = system.file("CurlSSL", "cacert.pem",
package = "RCurl"),
connecttimeout = 300, timeout = 3000,
followlocation = TRUE, dns.cache.timeout = 300))
Below is a console listing with debug set on curl::curl_fetch_memory.
> json <- createJSON(phi = cases$phi,
+ theta = cases$theta,
+ doc.len .... [TRUNCATED]
> serVis(json, open.browser = TRUE, as.gist = TRUE, description = 'APM Community')
debugging in: curl::curl_fetch_memory(url, handle = handle)
debug: {
output <- .Call(R_curl_fetch_memory, url, handle)
res <- handle_response_data(handle)
res$content <- output
res
}
Browse[2]> output <- .Call(R_curl_fetch_memory, url, handle)
Error: Timeout was reached
Browse[2]> output <- .Call(R_curl_fetch_memory, url, handle)
Browse[2]> rawToChar(output)
[1] "{\"message\":\"Problems parsing JSON\",\"documentation_url\":\"https://developer.github.com/v3\"}"
Browse[2]>
.
.
exiting from: curl::curl_fetch_memory(url, handle = handle)
Error: Problems parsing JSON
Any hints on how to debug this problem?
I'm playing with code for supervisor trees taken from http://learnyousomeerlang.com/building-applications-with-otp but I get a noproc exception that I can't figure out when I try to get a supervisor to start the child process. This is my shell interaction:
1> application:start(test).
root supervisor init
ok
2> test_sup:start_service(service_sup,{service_worker, start_link,[]}).
{ok,<0.39.0>}
worker supervisor initialise (M: service_worker,F: start_link,A: [])
3> test_app:run(service_worker,[]).
server run: (name: service_worker args: [])
** exception exit: {noproc,{gen_server,call,[service_worker,{run,[]}]}}
in function gen_server:call/2 (gen_server.erl, line 182)
Code is:
-module(test_app).
-behaviour(application).
-export([start/2, stop/1, run/2]).
start(_StartType, _StartArgs) ->
test_sup:start_link().
run(Name, Args) ->
service_serv:run(Name, Args).
=====
-module(test_sup).
-behaviour(supervisor).
-export([start_link/0, init/1, stop/0, start_service/2, stop_service/1]).
-define(CHILD(I, Type), {I, {I, start_link, []}, permanent, 5000, Type, [I]}).
start_link() ->
supervisor:start_link({local, service}, ?MODULE, []).
init([]) ->
io:format("root supervisor init~n"),
{ok, {{one_for_one, 5, 10},
[]}}.
start_service(Name, MFA) ->
ChildSpec = {Name,
{service_sup, start_link, [Name, MFA]},
permanent, 10500, supervisor, [service_sup]},
io:format("start service supervisor (Name: ~p, MFA: ~p): ", [Name, MFA]),
supervisor:start_child(service, ChildSpec).
[snip]
====
-module(service_sup).
-export([start_link/2, init/1]).
-behaviour(supervisor).
start_link(Name, MFA) ->
supervisor:start_link(?MODULE, {Name, MFA}).
init({Name, MFA}) ->
MaxRestart = 1,
MaxTime = 3600,
{ok, {{one_for_all, MaxRestart, MaxTime},
[{serv,
{service_serv, start_link, [Name, self(), MFA]},
permanent,
5000,
worker, [service_serv]}]}}.
========
-module(worker_sup).
-export([start_link/1, init/1]).
-behaviour(supervisor).
start_link(MFA) ->
supervisor:start_link(?MODULE, MFA).
init({M,F,A}) ->
{ok, {{simple_one_for_one, 5, 3600},
[{service_worker,
{M,F,A},
temporary, 5000, worker, [M]}]}}.
===
-module(service_serv).
-behaviour(gen_server).
-export([start/3, start_link/3, run/2,
status/1, ping/1, stop/1]).
-export([init/1, handle_call/3, handle_cast/2, handle_info/2,
code_change/3, terminate/2]).
-define(WORKER_SUP_SPEC(MFA),
{worker_sup,
{worker_sup, start_link, [MFA]},
permanent,
10000,
supervisor,
[worker_sup]}).
-record(state, {sup,
refs,
queue=queue:new()
}).
start(Name, Sup, MFA) when is_atom(Name) ->
gen_server:start({local, Name}, ?MODULE, {MFA, Sup}, []).
start_link(Name, Sup, MFA) when is_atom(Name) ->
gen_server:start_link({local, Name}, ?MODULE, {MFA, Sup}, []).
init({MFA, Sup}) ->
self() ! {start_worker_supervisor, Sup, MFA},
{ok, #state{}}.
run(Name, Args) ->
io:format("server run: (name: ~p args: ~p) ~n",[Name, Args]),
gen_server:call(Name, {run, Args}).
handle_info({start_worker_supervisor, Sup, MFA}, S = #state{}) ->
{ok, Pid} = supervisor:start_child(Sup, ?WORKER_SUP_SPEC(MFA)),
{noreply, S#state{sup=Pid}};
handle_info({'DOWN', Ref, process, _Pid, _}, S = #state{refs=Refs}) ->
case gb_sets:is_element(Ref, Refs) of
true ->
handle_down_worker(Ref, S);
false -> %% Not our responsibility
{noreply, S}
end;
handle_info(Msg, State) ->
{noreply, State}.
handle_call({run, Args}, _From, S = #state{sup=Sup, refs=R}) ->
io:format("handle run call ~n"),
{ok, Pid} = supervisor:start_child(Sup, Args),
Ref = erlang:monitor(process, Pid),
{reply, {ok, run, Pid}, S#state{refs=gb_sets:add(Ref,R)}};
[snip]
====
-module(service_worker).
-behaviour(gen_server).
-export([start_link/4, stop/1]).
-export([init/0, init/1, handle_call/3, handle_cast/2,
handle_info/2, code_change/3, terminate/2]).
start_link(Task, Delay, Max, SendTo) ->
gen_server:start_link(?MODULE, {Task, Delay, Max, SendTo} , []).
stop(Pid) ->
gen_server:call(Pid, stop).
init({Task, Delay, Max, SendTo}) ->
io:format("initialise worker ~n"),
%% {ok, {Task, Delay, Max, SendTo}}.
{ok, {Task, Delay, Max, SendTo}, Delay}.
[snip]
The volume of code you've produced is a little difficult to parse, but what stands out to me most is that you are apparently using the atom "service_worker" to refer to the process that you start.
This is all well and good if you register a process with that atom (either by calling erlang:register(service_worker, Pid) or starting the process with gen_server:start_link({local, service_worker}, ?MODULE, Args, Opts)). You appear to be doing neither, and the error message you receive supports that assessment.
** exception exit: {noproc,{gen_server,call,[service_worker,{run,[]}]}}
in function gen_server:call/2 (gen_server.erl, line 182)
What this error tells us is that gen_server:call was not able to find the process (noproc). The arguments to gen_server:call are included in the error message, and in the spot where one would expect to find the Pid, we find instead service_worker.
Additionally, your service_worker module appears to be started by a simple_one_for_one supervisor. Such supervisors are used when you need multiple of the same "type" of process (e.g., same callback module). Such supervisors also do not start workers on their own (you must invoke supervisor:start_child(SupPid, ExtraArgs)).
Those are the two main issues I see with your present attempt. For a quick and dirty fix, try adding {local, service_worker} or {local, ?MODULE} as the first argument to the gen_server:start_link call in the service_worker module. Please bear in mind that this will not work if you intend to start multiple service_worker processes (as only one process can be registered to an atom at a time).
I have the following SML program in a file named testexc.sml:
structure TestExc : sig
val main : (string * string list -> int)
end =
struct
exception OhNoes;
fun main(prog_name, args) = (
raise OhNoes
)
end
I build it with smlnj-110.74 like this:
ml-build sources.cm TestExc.main testimg
Where sources.cm contains:
Group is
csx.sml
I invoke the program like so (on Mac OS 10.8):
sml #SMLload testimg.x86-darwin
I expect to see something when I invoke the program, but the only thing I get is a return code of 1:
$ sml #SMLload testimg.x86-darwin
$ echo $?
1
What gives? Why would SML fail silently on this unhandled exception? Is this behavior normal? Is there some generic handler I can put on main that will print the error that occurred? I realize I can match exception OhNoes, but what about larger programs with exceptions I might not know about?
The answer is to handle the exception, call it e, and print the data using a couple functions available in the system:
$ sml
Standard ML of New Jersey v110.74 [built: Tue Jan 31 16:23:10 2012]
- exnName;
val it = fn : exn -> string
- exnMessage;
val it = fn : exn -> string
-
Now, we have our modified program, were we have the generic handler tacked on to main():
structure TestExc : sig
val main : (string * string list -> int)
end =
struct
exception OhNoes;
open List;
fun exnToString(e) =
List.foldr (op ^) "" ["[",
exnName e,
" ",
exnMessage e,
"]"]
fun main(prog_name, args) = (
raise OhNoes
)
handle e => (
print("Grasshopper disassemble: " ^ exnToString(e));
42)
end
I used lists for generating the message, so to make this program build, you'll need a reference to the basis library in sources.cm:
Group is
$/basis.cm
sources.cm
And here's what it looks like when we run it:
$ sml #SMLload testimg.x86-darwin
Grasshopper disassemble: [OhNoes OhNoes (more info unavailable: ExnInfoHook not initialized)]
$ echo $?
42
I don't know what ExnInfoHook is, but I see OhNoes, at least. It's too bad the SML compiler didn't add a basic handler for us, so as to print something when there was an unhandled exception in the compiled program. I suspect ml-build would be responsible for that task.
I test with this code :
get_fee(Transaction,SourceNumber,Amount, Currency) ->
Url = lists:concat(["http://localhost/test.php","?transaction=", Transaction, "&saccount=", SourceNumber,Amount,"¤cy=",Currency]),
inets:start(),
{Flag, Response} = http:request(get, {Url, []}, [], []),
case Flag of
ok ->
{ { _, ReturnCode, _ }, _, Body } = Response,
if ReturnCode =:= 200 ->
{ok,{_,[{_,Code},{_,Permission},{_,Payer},{_,Payee}]}} = json:decode_string(Body),
case Permission of true ->
if Code =:= 200 ->
{ok,{Code, Payer, Payee}};
Code =:= 204 ->
{nok,{Code, not_found}};
true ->
{nok,{Code, parameter_error}}
end;
false ->
{nok,{Code, parameter_error}}
end;
true->
{error, http_error}
end;
error ->
case Response of
nxdomain -> {error, dns_error};
_ -> {error, network_error}
end
end.
the response from the http is : {"code":200,"permission":true,"fee_payer":0,"fee_payee":19}
But now I like to do the same think but the return of http in this case for example is :
{"CIN":"08321224","Name":21}
so I have just CIN and Name in this case
I try to change the previous
get_fee(Num) ->
Url = lists:concat(["http://localhost/GTW/Operation.php","?ACCOUNT_NUM=", Num]),
inets:start(),
{Flag, Response} = http:request(get, {Url, []}, [], []),
case Flag of
ok ->
{ { _, ReturnCode, _ }, _, Body } = Response,
%% for debug
io:format("~p~n",[ReturnCode]),
if ReturnCode =:= "08321224" ->
{ok,{_,[{_,CIN},{_,Name}]}} = json:decode_string(Body),
case Name of 21 ->
io:format(CIN),
io:format(Name),
if CIN =:= "08321224"->
{ok,{CIN, Name}};
CIN =:= 204 ->
{nok,{CIN, not_found}};
true ->
{nok,{CIN, parameter_error}}
end;
false ->
{nok,{CIN, parameter_error}}
end;
true->
{error, http_error}
end;
error ->
case Response of
nxdomain -> {error, dns_error};
_ -> {error, network_error}
%% for debug
%%io:format("pass2~n ~p~n",[Response]),
end
end.
but it displys :
test:get_fee("0001").
200
{error,http_error}
So I'll nitpick on the style here, because you will be far better off if you follow the semantical idea of Erlang:
get_fee(Num) ->
Url = lists:concat(["http://localhost/GTW/Operation.php","?ACCOUNT_NUM=", Num]),
inets:start(),
This is the wrong place to start inets. It should be started outside this function as you only need to
do this one.
{Flag, Response} = http:request(get, {Url, []}, [], []),
This part is better coded with a pattern match. The discrimination of Flag and Response can
be decoded directly with a simple match. Write,
case http:request(get, {Url, []}, [], []) of
{ok, {{_, 200, _}, _, Body}} ->
{ok, R} = json:decode_string(Body),
get_fee_decode_(get_cin(R), get_name(R));
{error, Reason} -> {error, Reason}
end.
I would recommend against changing {error, nxdomain} to {error, dns_error} since nxdomain perfectly
codes this case in any case. Just pass the error tuple to the caller and have him handle it.
get_fee_decode_("08321224" = CIN, 21 = Name) -> {ok, {CIN, Name}};
get_fee_decode_("204" = CIN, 21) -> {nok, {CIN, not_found}};
get_fee_decode_(CIN, _Name) -> {nok, {CIN, parameter_error}};
Introduce a new function like this to handle the inner parts of your code base. And hoist the matching to
top-level. This helps in the long run by decoupling your code into functions.
Do note that in a JSON structure, there is no order on an "object" so you can't assume that the structure is
{"code":200,"permission":true,"fee_payer":0,"fee_payee":19}
But a decode does not have to preserve this structure, according to JSON. So a valid decode may be:
[{"fee_payee", 19}, {"fee_payer", 0}, {"permission", true}, {"code", 200}]
This will fail to match in your code and you are setting yourself up for some nasty errors later on.
You want something along the lines of:
get_fee_payer(PL) -> proplists:get_value("fee_payer", PL).
Another thing which will be a problem with your programming style is that you are hiding error-information cases. In Erlang, you can often get away with only handling the "happy path" through the code and leave all the error handling out until you know what kind of errors are there in the code base. Then you can begin adding in error handling slowly. Defensive programming is not a thing you should be doing if you can avoid it.
You changed:
if ReturnCode =:= 200 ->
to:
if ReturnCode =:= "08321224" ->
However, that needs to stay the same in your version. 200 is the HTTP status code for "OK" - the first step here is to verify that the server actually processed the request and returned a positive reply. You'll find that number only in Body - that is what the if CIN =:= "08321224"-> part is for.