Difficulty unpacking JSON tuple string - json
I figured out how to use rebar. I'm trying to use jsx (jiffy doesn't work properly on Windows) to parse json that I obtained using the openexchangerates.org API, but I can't even figure out how to correctly utilize Erlang's extensive binary functionality in order to unpack the JSON tuple obtained. Using the following code snippet, I managed to get a tuple that has all the data I need:
-module(currency).
-export([start/0]).
start() ->
URL = "http://openexchangerates.org",
Endpoint = "/api/latest.json?app_id=<myprivateid>",
X = string:concat(URL, Endpoint),
% io:format("~p~n",[X]).
inets:start(),
{ok, Req} = httpc:request(X),
Req.
Here is the obtained response:
9> currency:start().
{{"HTTP/1.1",200,"OK"},
[{"cache-control","public"},
{"connection","close"},
{"date","Fri, 15 Aug 2014 01:28:06 GMT"},
{"etag","\"d9ad180d4af1caaedab6e622ec0a8a70\""},
{"server","Apache"},
{"content-length","4370"},
{"content-type","application/json; charset=utf-8"},
{"last-modified","Fri, 15 Aug 2014 01:00:56 GMT"},
{"access-control-allow-origin","*"}],
"{\n \"disclaimer\": \"Exchange rates are provided for informational purposes only, and do not constitute financial advice of any kind. Although every attempt is made to ensure quality, NO guarantees are given whatsoever of accuracy, validity, availability, or fitness for any purpose - please use at your own risk. All usage is subject to your acceptance of the Terms and Conditions of Service, available at: https://openexchangerates.org/terms/\",\n \"license\": \"Data sourced from various providers with public-facing APIs; copyright may apply; resale is prohibited; no warranties given of any kind. Bitcoin data provided by http://coindesk.com. All usage is subject to your acceptance of the License Agreement available at: https://openexchangerates.org/license/\",\n \"timestamp\": 1408064456,\n \"base\": \"USD\",\n \"rates\": {\n \"AED\": 3.673128,\n \"AFN\": 56.479925,\n \"ALL\": 104.147599,\n \"AMD\": 413.859001,\n \"ANG\": 1.789,\n \"AOA\": 97.913074,\n \"ARS\": 8.274908,\n \"AUD\": 1.073302,\n \"AWG\": 1.79005,\n \"AZN\": 0.783933,\n \"BAM\": 1.46437,\n \"BBD\": 2,\n \"BDT\": 77.478631,\n \"BGN\": 1.464338,\n \"BHD\": 0.377041,\n \"BIF\": 1546.956667,\n \"BMD\": 1,\n \"BND\": 1.247024,\n \"BOB\": 6.91391,\n \"BRL\": 2.269422,\n \"BSD\": 1,\n \"BTC\": 0.0019571961,\n \"BTN\": 60.843812,\n \"BWP\": 8.833083,\n \"BYR\": 10385.016667,\n \"BZD\": 1.99597,\n \"CAD\": 1.0906,\n \"CDF\": 924.311667,\n \"CHF\": 0.906799,\n \"CLF\": 0.02399,\n \"CLP\": 577.521099,\n \"CNY\": 6.153677,\n \"COP\": 1880.690016,\n \"CRC\": 540.082202,\n \"CUP\": 1.000688,\n \"CVE\": 82.102201,\n \"CZK\": 20.81766,\n \"DJF\": 178.76812,\n \"DKK\": 5.579046,\n \"DOP\": 43.43789,\n \"DZD\": 79.8973,\n \"EEK\": 11.70595,\n \"EGP\": 7.151305,\n \"ERN\": 15.062575,\n \"ETB\": 19.83205,\n \"EUR\": 0.748385,\n \"FJD\": 1.85028,\n \"FKP\": 0.599315,\n \"GBP\": 0.599315,\n \"GEL\": 1.74167,\n \"GGP\": 0.599315,\n \"GHS\": 3.735499,\n \"GIP\": 0.599315,\n \"GMD\": 39.73668,\n \"GNF\": 6995.309935,\n \"GTQ\": 7.839405,\n \"GYD\": 205.351249,\n \"HKD\": 7.750863,\n \"HNL\": 21.04854,\n \"HRK\": 5.708371,\n \"HTG\": 44.66625,\n \"HUF\": 233.847801,\n \"IDR\": 11682.083333,\n \"ILS\": 3.471749,\n \"IMP\": 0.599315,\n \"INR\": 60.81923,\n \"IQD\": 1178.211753,\n \"IRR\": 26354,\n \"ISK\": 115.976,\n \"JEP\": 0.599315,\n \"JMD\": 112.604801,\n \"JOD\": 0.707578,\n \"JPY\": 102.501401,\n \"KES\": 88.106539,\n \"KGS\": 51.96,\n \"KHR\": 4056.578416,\n \"KMF\": 368.149,\n \"KPW\": 900,\n \"KRW\": 1021.166657,\n \"KWD\": 0.283537,\n \"KYD\": 0.826373,\n \"KZT\": 182.076001,\n \"LAK\": 8049.834935,\n \"LBP\": 1509.068333,\n \"LKR\": 130.184301,\n \"LRD\": 91.49085,\n \"LSL\": 10.56165,\n \"LTL\": 2.583284,\n \"LVL\": 0.521303,\n \"LYD\": 1.244127,\n \"MAD\": 8.372529,\n \"MDL\": 13.7178,\n \"MGA\": 2495.605,\n \"MKD\": 45.99967,\n \"MMK\": 972.1784,\n \"MNT\": 1884.666667,\n \"MOP\": 7.986251,\n \"MRO\": 292.0081,\n \"MTL\": 0.683602,\n \"MUR\": 30.61708,\n \"MVR\": 15.37833,\n \"MWK\": 392.9201,\n \"MXN\": 13.07888,\n \"MYR\": 3.175156,\n \"MZN\": 30.3522,\n \"NAD\": 10.56145,\n \"NGN\": 162.303701,\n \"NIO\": 26.07651,\n \"NOK\": 6.157432,\n \"NPR\": 97.66846,\n \"NZD\": 1.179688,\n \"OMR\": 0.38501,\n \"PAB\": 1,\n \"PEN\": 2.795018,\n \"PGK\": 2.464545,\n \"PHP\": 43.66429,\n \"PKR\": 99.5662,\n \"PLN\": 3.126223,\n \"PYG\": 4272.421673,\n \"QAR\": 3.641137,\n \"RON\": 3.320192,\n \"RSD\": 87.82784,\n \"RUB\": 36.00216,\n \"RWF\": 690.269,\n \"SAR\": 3.750523,\n \"SBD\": 7.269337,\n \"SCR\": 12.40801,\n \"SDG\": 5.699103,\n \"SEK\": 6.86018,\n \"SGD\": 1.246263,\n \"SHP\": 0.599315,\n \"SLL\": 4372.166667,\n \"SOS\": 841.5678,\n \"SRD\": 3.275,\n \"STD\": 18316.816667,\n \"SVC\": 8.745567,\n \"SYP\": 150.751249,\n \"SZL\": 10.56279,\n \"THB\": 31.86192,\n \"TJS\": 4.9856,\n \"TMT\": 2.8501,\n \"TND\": 1.719658,\n \"TOP\": 1.8861,\n \"TRY\": 2.15338,\n \"TTD\": 6.343484,\n \"TWD\": 30.00481,\n \"TZS\": 1661.865,\n \"UAH\": 13.02466,\n \"UGX\": 2614.28,\n \"USD\": 1,\n \"UYU\": 23.70693,\n \"UZS\": 2337.106637,\n \"VEF\": 6.295009,\n \"VND\": 21191.15,\n \"VUV\": 94.6,\n \"WST\": 2.301222,\n \"XAF\": 491.286739,\n \"XAG\": 0.05031657,\n \"XAU\": 0.00076203,\n \"XCD\": 2.70154,\n \"XDR\": 0.654135,\n \"XOF\": 491.394602,\n \"XPF\": 89.414091,\n \"YER\": 214.985901,\n \"ZAR\": 10.55678,\n \"ZMK\": 5253.075255,\n \"ZMW\": 6.169833,\n \"ZWL\": 322.355006\n }\n}"}
I don't understand why this code oesn't work:
X = "Arthur".
B = <<X>>.
JSX allows a lot of parsing functionality but only if I have a binary as my representation of JSON, and this JSON I'm getting from the currency API is a string in a tuple... I'm a bit lost as to where to start to research. Unpacking a tuple using pattern matching is supposedly quite simple (I've done some Prolog programming and I can see that erlang has similar behavior) but is there a another, better, Erlang-appropriate way to grab the "rates" part of the JSON I'm receiving as a response?
Thank you! I'm working on a cool web app to learn erlang and this is a good first step. I have three Erlang books and I'm reading through them diligently but the problem is that I want as much practical exposure as early on as possible. I love this language but I want to get a solid grounding as fast as possible.
Thank you!
get_currencies() ->
URL = "http://openexchangerates.org",
Endpoint = "/api/latest.json?app_id=<myprivateid>",
X = string:concat(URL, Endpoint),
% io:format("~p~n",[X]).
inets:start(),
{ok, {_,_,R}} = httpc:request(X),
E = jsx:decode(lists_to_binary(R)),
Base = proplists:get_value(<<"base">>,E),
Sec = proplists:get_value(<<"timestamp">>,E),
{Days,Time} = calendar:seconds_to_daystime(Sec),
Date = calendar:gregorian_days_to_date(Days+719528),
Currencies = proplists:get_value(<<"rates">>,E),
fun(C) -> V = proplists:get_value(C,Currencies),
{change,Date,Time,Base,C,V}
end.
and somewhere in your code:
GC = get_currencies(), %% you can store GC in an ets, a server state...
%% but don't forget to refresh it :o)
and use it later
{change,D,T,B,C,V} = GC(<<"ZWL">>),
%% should return {change,{2014,8,15},{2,0,12},<<"USD">>,<<"ZWL">>,322.355006}
[edit]
When I use an external application such as jsx (using rebar itself), I use also rebar and its dependency mechanism to create my own application, in my opinion it is the most convenient way. (In other cases, I use also rebar :o)
Then I build my application using the OTP behaviors (application, supervisor, gen_server...). It is a lot of modules to write, but some of them are very very short (application and supervisors) and they facilitate the application structure (see What is OTP if you are not familiar with this).
In your case, my first idea is to have a gen server that build and store the GC anonymous function in its state, each time it get a cast message such as update_currencies, and provide the answer each time it get a call message such as {get_change,Cur} (and maybe refresh GC if it is undefined or out dated).
You will also have to decide where the errors will be managed - it may be nowhere: if the gen_server does nothing else but answer to this currency query: if something wrong appears it will crash and be restarted by its supervisor - because this code has many interfaces with the out world and so subject to numerous failures (no Internet access, structure of answer change from site, bad currency request from user...)
I figured out my problem.
So first of all, I wasn't thinking of how simple it is to simply count how many elements there are in the tuple. That being said, I realized there were only three.
So the line I needed was
{A,B,C} = Req.
After that, I only wanted to look at the last one (C, the JSON payload), which was a string.
I found out through another source (not to disregard what you told me, Kay!) that you need to use the list functions, since strings and just lists of integers within an ASCII range (I think), in this case list_to_binary.
Once I used this line:
D = list_to_binary(C), and subsequently
E = jsx:decode(D), I got this output:
[{<<"disclaimer">>,
<<"Exchange rates are provided for infor
attempt is made to ensure quality, NO gua
se - please use at your own risk. All usag
s://openexchangerates.org/terms/">>},
{<<"license">>,
<<"Data sourced from various providers w
any kind. Bitcoin data provided by http://
t: https://openexchangerates.org/license/"
{<<"timestamp">>,1408068012},
{<<"base">>,<<"USD">>},
{<<"rates">>,
[{<<"AED">>,3.673128},
{<<"AFN">>,56.479925},
{<<"ALL">>,104.1526},
{<<"AMD">>,413.859001},
{<<"ANG">>,1.789},
{<<"AOA">>,97.913949},
{<<"ARS">>,8.274608},
{<<"AUD">>,1.073236},
{<<"AWG">>,1.79005},
{<<"AZN">>,0.783933},
{<<"BAM">>,1.46437},
{<<"BBD">>,2},
{<<"BDT">>,77.478631},
{<<"BGN">>,1.464358},
{<<"BHD">>,0.377041},
{<<"BIF">>,1546.956667},
{<<"BMD">>,1},
{<<"BND">>,1.246774},
{<<"BOB">>,6.91391},
{<<"BRL">>,2.269462},
{<<"BSD">>,1},
{<<"BTC">>,0.0019393375},
{<<"BTN">>,60.843812},
{<<"BWP">>,8.833083},
{<<"BYR">>,10385.016667},
{<<"BZD">>,1.99597},
{<<"CAD">>,1.090486},
{<<"CDF">>,924.311667},
{<<"CHF">>,0.906833},
{<<"CLF">>,0.02399},
{<<"CLP">>,577.521099},
{<<"CNY">>,6.151637},
{<<"COP">>,1880.690016},
{<<"CRC">>,540.082202},
{<<"CUP">>,1.000688},
{<<"CVE">>,82.049699},
{<<"CZK">>,20.818},
{<<"DJF">>,179.084119},
{<<"DKK">>,5.579049},
{<<"DOP">>,43.43789},
{<<"DZD">>,79.8641},
{<<"EEK">>,11.7064},
{<<"EGP">>,7.150475},
{<<"ERN">>,15.062575},
{<<"ETB">>,19.83205},
{<<"EUR">>,0.748419},
{<<"FJD">>,1.850441},
{<<"FKP">>,0.599402},
{<<"GBP">>,0.599402},
{<<"GEL">>,1.74167},
{<<"GGP">>,0.599402},
{<<"GHS">>,3.735499},
{<<"GIP">>,0.599402},
{<<"GMD">>,39.73668},
{<<"GNF">>,6995.309935},
{<<"GTQ">>,7.839405},
{<<"GYD">>,205.351249},
{<<"HKD">>,7.750754},
{<<"HNL">>,21.04854},
{<<"HRK">>,5.708511},
{<<"HTG">>,44.66625},
{<<"HUF">>,233.8448},
{<<"IDR">>,11685.75},
{<<"ILS">>,3.471469},
{<<"IMP">>,0.599402},
{<<"INR">>,60.82523},
{<<"IQD">>,1178.211753},
{<<"IRR">>,26355.666667},
{<<"ISK">>,115.96},
{<<"JEP">>,0.599402},
{<<"JMD">>,112.604801},
{<<"JOD">>,0.707778},
{<<"JPY">>,102.495401},
{<<"KES">>,88.107639},
{<<"KGS">>,51.991},
{<<"KHR">>,4056.578416},
{<<"KMF">>,368.142141},
{<<"KPW">>,900},
{<<"KRW">>,1021.353328},
{<<"KWD">>,0.283537},
{<<"KYD">>,0.826373},
{<<"KZT">>,182.076001},
{<<"LAK">>,8049.834935},
{<<"LBP">>,1509.068333},
{<<"LKR">>,130.184301},
{<<"LRD">>,91.49085},
{<<"LSL">>,10.56165},
{<<"LTL">>,2.583364},
{<<"LVL">>,0.521328},
{<<"LYD">>,1.244147},
{<<"MAD">>,8.372619},
{<<"MDL">>,13.7178},
{<<"MGA">>,2495.605},
{<<"MKD">>,46.00037},
{<<"MMK">>,972.1784},
{<<"MNT">>,1885},
{<<"MOP">>,7.986291},
{<<"MRO">>,292.0081},
{<<"MTL">>,0.683738},
{<<"MUR">>,30.61748},
{<<"MVR">>,15.37833},
{<<"MWK">>,392.9201},
{<<"MXN">>,13.07883},
{<<"MYR">>,3.175406},
{<<"MZN">>,30.3272},
{<<"NAD">>,10.56145},
{<<"NGN">>,162.303701},
{<<"NIO">>,26.07651},
{<<"NOK">>,6.156902},
{<<"NPR">>,97.66846},
{<<"NZD">>,1.179692},
{<<"OMR">>,0.38501},
{<<"PAB">>,1},
{<<"PEN">>,2.795018},
{<<"PGK">>,2.464545},
{<<"PHP">>,43.68439},
{<<"PKR">>,99.5642},
{<<"PLN">>,3.126203},
{<<"PYG">>,4272.421673},
{<<"QAR">>,3.641297},
{<<"RON">>,3.319212},
{<<"RSD">>,87.8205},
{<<"RUB">>,36.00206},
{<<"RWF">>,690.088},
{<<"SAR">>,3.750583},
{<<"SBD">>,7.258136},
{<<"SCR">>,12.40829},
{<<"SDG">>,5.697837},
{<<"SEK">>,6.857347},
{<<"SGD">>,1.246447},
{<<"SHP">>,0.599402},
{<<"SLL">>,4360},
{<<"SOS">>,841.5678},
{<<"SRD">>,3.275},
{<<"STD">>,18322.733333},
{<<"SVC">>,8.745567},
{<<"SYP">>,150.793749},
{<<"SZL">>,10.56279},
{<<"THB">>,31.87122},
{<<"TJS">>,4.985575},
{<<"TMT">>,2.8501},
{<<"TND">>,1.719698},
{<<"TOP">>,1.889033},
{<<"TRY">>,2.15342},
{<<"TTD">>,6.343484},
{<<"TWD">>,29.99281},
{<<"TZS">>,1661.865},
{<<"UAH">>,13.02466},
{<<"UGX">>,2614.28},
{<<"USD">>,1},
{<<"UYU">>,23.70693},
{<<"UZS">>,2337.773304},
{<<"VEF">>,6.295009},
{<<"VND">>,21191.15},
{<<"VUV">>,94.4875},
{<<"WST">>,2.301222},
{<<"XAF">>,491.283058},
{<<"XAG">>,0.05031404},
{<<"XAU">>,7.6211e-4},
{<<"XCD">>,2.70154},
{<<"XDR">>,0.654135},
{<<"XOF">>,491.394602},
{<<"XPF">>,89.416091},
{<<"YER">>,214.985901},
{<<"ZAR">>,10.55649},
{<<"ZMK">>,5252.024745},
{<<"ZMW">>,6.169833},
{<<"ZWL">>,322.355006}]}]
ok
So this is closer to what I want, but how do I extract a specific currency easily? Like ZWL, for example.
Related
Function - Compilation error. Line 9: no viable alternative at character '{'
I am trying to write a trading strategy in Pine Script, but I am getting a compilation error on line 9. The error message states that there is "no viable alternative at character '{'". I have tried adding the keyword "function" before the function name, but the error still persists. The goal is to have the code look at the volatility of the market with MACD, 200 EMA and overall volume with in an hour to determine a safe amount to invest during that period. Can someone please help me fix this error and make my code compile successfully?
function calcTradeAmount(v) => {
tradeAmount = money * (v / 100)
return tradeAmount
}
If the function works properly my if(long) and if(shorts) should be able to pull a different number each time. I don't want to have 1 trade be the entire portfolio each time.
Sample:
if (long)
tradeAmount = calcTradeAmount(volatility)
entryPrice = low
calcTradeAmount(v) => tradeAmount = money * (v / 100) tradeAmount
Result of user-defined function not displaying on web app
I am trying to define a function that divides Amount of money by Number of days.
So far the user can submit values, but I don't know how to make the result display on my Streamlit web app.
I copied part of my code below.
P.S. I am also a complete beginner in Python.
Thanks for any help
#HOW OFTEN EAT OUT
st.write("2. How often do you eat out?")
form04 = st.form(key='form04')
days = form04.text_input('Please enter average number of days')
submit04 = form04.form_submit_button('Submit')
#HOW MUCH INCOME
if submit04:
st.write('3. What is your monthly income?')
form05 = st.form(key='form05')
income = form05.text_input('Please enter monthly income')
submit05 = form05.form_submit_button('Submit')
if submit05:
def idealbudget(days, income):
budget=float(income)/float(days)
return float(budget)
st.write('Result is', budget)
In this code snippet, you define a function but you never actually call it:
if submit05:
def idealbudget(days, income):
budget=float(income)/float(days)
return float(budget)
st.write('Result is', budget)
Additionally, your st.write call is tabbed incorrectly, it should be at the same level as the def statement. A working solution probably looks like the following (untested):
if submit05:
def idealbudget(days, income):
budget=float(income)/float(days)
return float(budget)
st.write('Result is', idealbudget(days, income))
Equivalent of Platform::IBoxArray in C++/WinRT
I am currently porting an UWP application from C++/CX to C++/WinRT. I encountered a safe_cast<Platform::IBoxArray<byte>^>(data) where data is of type Windows::Foundation::IInspectable ^.
I know that the safe_cast is represented by the as<T> method, and I know there are functions for boxing (winrt::box_value) and unboxing (winrt::unbox_value) in WinRT/C++.
However, I need to know the equivalent of Platform::IBoxArray in order to perform the cast (QueryInterface). According to https://learn.microsoft.com/de-de/cpp/cppcx/platform-iboxarray-interface?view=vs-2017, IBoxArray is the C++/CX equivalent of Windows::Foundation::IReferenceArray, but there is no winrt::Windows::Foundation::IReferenceArray...
Update for nackground: What I am trying to achieve is retrieving the view transform attached by the HoloLens to every Media Foundation sample from its camera. My code is based on https://github.com/Microsoft/HoloLensForCV, and I got really everything working except for this last step. The problem is located around this piece of code:
static const GUID MF_EXTENSION_VIEW_TRANSFORM = {
0x4e251fa4, 0x830f, 0x4770, 0x85, 0x9a, 0x4b, 0x8d, 0x99, 0xaa, 0x80, 0x9b
};
// ...
// In the event handler, which receives const winrt::Windows::Media::Capture::Frames::MediaFrameReader& sender:
auto frame = sender.TryAcquireLatestFrame();
// ...
if (frame.Properties().HasKey(MF_EXTENSION_VIEW_TRANSFORM)) {
auto /* IInspectable */ userData = frame.Properties().Lookup(MF_EXTENSION_VIEW_TRANSFORM);
// Now I would have to do the following:
// auto userBytes = safe_cast<Platform::IBoxArray<Byte> ^>(userData)->Value;
//viewTransform = *reinterpret_cast<float4x4 *>(userBytes.Data);
}
I'm also working on porting some code from HoloLensForCV to C++/WinRT. I came up with the following solution for a very similar case (but not the exact same line of code you ask about):
auto user_data = source.Info().Properties().Lookup(c_MF_MT_USER_DATA); // type documented as 'array of bytes'
auto source_name = user_data.as<Windows::Foundation::IReferenceArray<std::uint8_t>>(); // Trial and error to get the right specialization of IReferenceArray
winrt::com_array<std::uint8_t> arr;
source_name.GetUInt8Array(arr);
winrt::hstring source_name_str{ reinterpret_cast<wchar_t*>(arr.data()) };
Specifically, you can replace the safe_cast with .as<Windows::Foundation::IReferenceArray<std::uint8_t> for a boxed array of bytes. Then, I suspect doing the same cast as me (except to float4x4* instead of wchar_t*) will work for you.
The /ZW flag is not required for my example above.
I can't believe that actually worked, but using information from https://learn.microsoft.com/de-de/windows/uwp/cpp-and-winrt-apis/interop-winrt-cx, I came up with the following solution: Enable "Consume Windows Runtime Extension" via /ZW and use the following conversion: auto abi = reinterpret_cast<Platform::Object ^>(winrt::get_abi(userData)); auto userBytes = safe_cast<Platform::IBoxArray<byte> ^>(abi)->Value; viewTransform = *reinterpret_cast<float4x4 *>(userBytes->Data); Unfortunately, the solution has the drawback of generating warning C4447: 'main' signature found without threading model. Consider using 'int main(Platform::Array^ args)'. But for now, I can live with it ...
aws-sdk-cpp: how to use CurlHttpClient?
I need to make signed requests to AWS ES, but am stuck at the first hurdle in that I cannot seem to be able to use CurlHttpClient. Here is my code (verb, path, and body defined elsewhere):
Aws::Client::ClientConfiguration clientConfiguration;
clientConfiguration.scheme = Aws::Http::Scheme::HTTPS;
clientConfiguration.region = Aws::Region::US_EAST_1;
auto client = Aws::MakeShared<Aws::Http::CurlHttpClient>(ALLOCATION_TAG, clientConfiguration);
Aws::Http::URI uri;
uri.SetScheme(Aws::Http::Scheme::HTTPS);
uri.SetAuthority(ELASTIC_SEARCH_DOMAIN);
uri.SetPath(path);
Aws::Http::Standard::StandardHttpRequest req(uri, verb);
req.AddContentBody(body);
auto res = client->MakeRequest(req);
Aws::Http::HttpResponseCode resCode = res->GetResponseCode();
if (resCode == Aws::Http::HttpResponseCode::OK) {
Aws::IOStream &body = res->GetResponseBody();
rejoiceAndBeMerry();
}
else {
gotoPanicStations();
}
When executed, the code throws a bad_function_call deep from within the sdk mixed up with a lot of shared_ptr this and allocate that. My guess is that I am just using the SDK wrong, but I've been unable to find any examples that use the CurlHttpClient directly such as I need to do here.
How can I use CurlHttpClient?
You shouldn't be using the HTTP client directly, but the supplied wrappers with the aws-cpp-sdk-es package. Like previous answer(s), I would recommend evaluating the test cases shipped with the library to see how the original authors intended to implement the API (at least until the documents catch-up).
How can I use CurlHttpClient?
Your on the right track with managed shared resources and helper functions. Just need to create a static factory/client to reference. Here's a generic example.
using namespace Aws::Client;
using namespace Aws::Http;
static std::shared_ptr<HttpClientFactory> MyClientFactory; // My not be needed
static std::shared_ptr<HttpClient> MyHttpClient;
// ... jump ahead to method body ...
ClientConfiguration clientConfiguration;
MyHttpClient = CreateHttpClient(clientConfiguration);
Aws::String uri("https://example.org");
std::shared_ptr<HttpRequest> req(
CreateHttpRequest(uri,
verb, // i.e. HttpMethod::HTTP_POST
Utils::Stream::DefaultResponseStreamFactoryMethod));
req.AddContentBody(body); //<= remember `body' should be `std::shared_ptr<Aws::IOStream>',
// and can be created with `Aws::MakeShared<Aws::StringStream>("")';
req.SetContentLength(body_size);
req.SetContentType(body_content_type);
std::shared_ptr<HttpResponse> res = MyHttpClient->MakeRequest(*req);
HttpResponseCode resCode = res->GetResponseCode();
if (resCode == HttpResponseCode::OK) {
Aws::StringStream resBody;
resBody << res->GetResponseBody().rdbuf();
rejoiceAndBeMerry();
} else {
gotoPanicStations();
}
I encountered exactly the same error when trying to download from S3 using CurlHttpClient. I fixed it by instead modelling my code after the integration test found in the cpp sdk: aws-sdk-cpp/aws-cpp-sdk-s3-integration-tests/BucketAndObjectOperationTest.cpp Search for the test called TestObjectOperationsWithPresignedUrls.
R - Vectorize a JSON call
Working with Mapquest directions API to plot thousands of routes using ggplot2 in R.
Basic code theory: Have a list of end locations and a single start location. For each end location, a call to fromJSON returns routing coordinates from Mapquest. From there, have already vectorized the assignment of coordinates (read as lists in lists) to the geom_path geom of ggplot2.
Right now, running this on a location set of ~ 1200 records takes ~ 4 minutes. Would love to get that down. Any thoughts on how to vectorize the call to fromJSON (which returns a list of lists)?
Windows 7, 64-bit, R 2.14.2
libraries: plyr, ggplot2, rjson, mapproj, XML
k = 0
start_loc = "263+NORTH+CENTER+ST.,+MESA+ARIZ."
end_loc = funder_trunc[,length(funder_trunc)]
route_urls = paste(mapquest_baseurl, "&from=", start_loc, "&to=", end_loc, "&ambiguities=ignore", sep="")
for (n in route_urls) {
route_legs = fromJSON(file = url(n))$route$legs[[1]]$maneuvers
lats = unlist(lapply(route_legs, function(x) return(x$startPoint[[2]])))
lngs = unlist(lapply(route_legs, function(x) return(x$startPoint[[1]])))
frame = data.frame(cbind(lngs, lats))
path_added = geom_path(aes(lngs, lats), data = frame)
p = p + path_added
k = k + 1
print(paste("Processed ", k, " of ", nrow(funder_trunc), " records in set.", sep=""))
}
Going out on a limb here since I don't use rjson or mapproj, but it seems like calling the server thousands of times is the real culprit. If the mapquest server doesn't have an API that allows you to send multiple requests in one go, you are in trouble. If it does, then you need to find out how to use/modify rjson and/or mapproj to call it... As #Chase said, you might be able to call it in parallel, but the server won't like getting too many parallel requests from the same client - it might ban you. Btw, it might not even like getting thousands of serial requests in rapid succession from the same client either - but apparently your current code works so I guess it doesn't mind.