In my application, I need to do API-call to a client in order to get information about a product (I do it by giving product_id).
Their API only allows 1 ID at a time which would make the job many times harder.
what have in mind is to run the API-call several times and add the results to the view.
I did a small test with merging 2 JSON together, which worked fine. Example:
#products = data_1['data'] + data_2['data']
This was, of course, doing them by hand and simple. What I thought I could do was to do an each do on them and run the code, as below:
### CONTROLLER
product_id = Array(params[:product_id].split(','))
product_id.each do |id|
uri = URI.parse("https://website.com/api?product_id=#{id}")
request = Net::HTTP::Get.new(uri)
request["Accept"] = "application/json"
req_options = { use_ssl: uri.scheme == "https", }
response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
http.request(request)
end
data = JSON.parse(response.body)
#products = data['data']
end
### VIEW
- #products.each do |product|
%p= product.name
%p= product.date
This doesn't give me any error, but in my view, I only see details for last product_id param. Same with if I move #products = data['data'] out of the each do.
In a desperate attempt, I also tried with this with no luck:
#products = id.as_json.merge(data['data'])
Any help is appreciated!
The answer was much closer to what I had in mind. I need to run map on the variable itself, like below:
### CONTROLLER
product_id = Array(params[:product_id].split(','))
#products = product_id.map { |id|
request = Net::HTTP::Get.new(uri)
request["Accept"] = "application/json"
req_options = { use_ssl: uri.scheme == "https", }
response = Net::HTTP.start(uri.hostname, uri.port, req_options) do |http|
http.request(request)
end
data = JSON.parse(response.body)
data['data']
}
hope it helps someone :)
Related
I have a rails app running alongside with a rails API, there is a constant value for DAYS_LIMIT in config/initializers/constants.rb
DAYS_LIMIT = 40
DEFAULT_PRICE = 1.29
but now in the app i added an input field so that the user decide his DAYS_LIMIT.
So i want to fetch that value from the database from inside the API models.
I have placed breakpoints and can see that inside the API controller, the data is transfered from the app but not to the models.
edited as a question requested , it's a React-on-Rails app , here is the code where the new input field is save to the database (i have removed the other fields so the question look shorter)
export const saveChannel = (files) => {
return async (dispatch, getState) => {
const { channel } = getState();
const {rss_podcast_days} = channel;
const { image } = files;
const save = id ? updateChannel : createChannel;
const sub_required = subscription_required !== undefined ? subscription_required : false;
const formData = new FormData();
formData.append('channel[rss_podcast_days]', rss_podcast_days || '');
if (Object.keys(image).length) {
formData.append('channel[image]', image);
}
const channelId = await dispatch(save(formData, id));
dispatch(fetchChannel(id));
return id;
};
};
from the app controller
podcast_list = RestClient.get("#{ENV['URL_API']}/api/#{#channel.id.as_json}/podcast/list")
#podcasts = JSON.parse(podcast_list.body)
#podcasts = #podcasts.sort.reverse.to_h
this is from the API controller witch the data is transfered from the app
def index
podcasts = #channel.podcasts.published.list(params[:page], params[:items_per_page], params[:ordered_in])
render json: Podcasts::Normalizer.normalize(podcasts, #channel.station.default_podcast_price)
end
and here from the API model that i want to fetch data instead of the constants.
scope :by_days_limit, -> {with_tags.more_recent_than(Date.today - DAYS_LIMIT.days).ordered}
it should take today date minus the value (DAYS_LIMIT) from user input, but for now i get undefined local variable or method if i try to fetch directly
Bro if your class has constant like DAYS_LIMIT you can access it using that class itself for example,
class Demo
DAYS_LIMIT = 5
end
you can access that constant by Demo.DAYS_LIMIT in controller or else wherever you need it.
good luck!
ok , so i finally got it, i don't know if i should delete this thread or just tell how i did it. If it's inapropriate just tell me and i will delete this entire thread.
So here is how i did it, in the API controller i had to add my fetch so that the arguments (list) knows what i am talking about. #channel.days_limit
def index
podcasts = #channel.podcasts.published.list(params[:page], params[:items_per_page], params[:ordered_in], #channel.days_limit)
render json: Podcasts::Normalizer.normalize(podcasts, #channel.station.default_podcast_price)
end
then in the def list of the models, i added days_limit has argument
def list(page = nil, nb_items_per_page = 40, ordered_in = 'desc', days_limit)
ordered_in = ordered_in.in?(['asc', 'desc']) ? ordered_in : 'desc'
page.blank? ? by_days_limit(days_limit) : by_page(page, nb_items_per_page, ordered_in)
end
and finally in the scope of the models, i pass in the new argument
scope :by_days_limit, -> (days_limit) {with_tags.more_recent_than(Date.today - days_limit.days).ordered}
Now the user input from the app is passing to the models via the controller.
I want to get 10 rows from database and count them all for pagination. It works when I don't use ajax however there is a problem in encoding jason and sending it to the view.
My code in controller:
$datas = model::take(10)->get();
$dataCount = model::all()->count();
$datas = $datas->put('dataCount', $dataCount);
return json_encode($datas);
And my code in view:
.complete(function(datas)
{
data = JSON.parse(datas.responseText);
data_count = data.dataCount;
delete data['dataCount'];
count = data.length;
for(key=0; key<count; key++)
{
//do something
}
});
When I delete data['dataCount'], the data object gets undefined, but when I delete $datas = $datas->put('dataCount', $dataCount), it works however I want to use count of all rows to create a pagination.
Thanks
Probably something to do with the Eloquent collection serialization. Try not including arbitrary data to the collection. Try something like this:
$datas = model::take(10)->get();
$dataCount = model::count();
$resp['data'] = $datas->toArray();
$resp['dataCount'] = $dataCount;
return json_encode($resp);
It will return your data array in a json attribute and your count in other.
I am currently working on a html scraper that takes a list of anime-planet url's from a text file and then loops through them, parses and stores the data in a database.
The scraper is working nicely however if I put in a large list then the chances of the url not linking to a series properly and throwing an error is quite high. I want to try make it so that IF the url does not work then it notes down the url in an array named 'error-urls' and just skips the record.
The end result being that the script finishes all working url's and returns a list of non working urls i can work with later (maybe in a text file, or just display in console).
I am currently using a rake task for this which is working quite nicely. If anyone could help me with implementing the error handling functionality it would be much appreciated. Cheers!
scrape.rake:
task :scrape => :environment do
require 'nokogiri'
require 'open-uri'
text = []
File.read("text.txt").each_line do |line|
text << line.chop
end
text.each do |series|
url = "http://www.anime-planet.com/anime/" + series
data = Nokogiri::HTML(open(url))
title = data.at_css('.theme').text
synopsis = data.at_css('.synopsis').text.strip
synopsis.slice! "Synopsis:\r\n\t\t\t\t\t"
eps = data.at_css('.type').text
year = data.at_css('.year').text
rating = data.at_css('.avgRating').text
categories = data.at_css('.categories')
genre = categories.css('li').text.to_s
image = data.at_css('#screenshots img')
imagePath = "http://www.anime-planet.com" + image['src']
anime = Series.create({:title => title, :image => imagePath, :description => synopsis, :eps => eps, :year => year, :rating => rating})
anime.tag_list = genre
anime.save()
end
end
Small example of list.txt
5-Centimeters-Per-Second
11Eyes
A-Channel
Air
Air-Gear
Aishiteru-Ze-Baby
You can use open-uri's error handling. See this for more details.
url = "http://www.anime-planet.com/anime/" + series
begin
doc = open(url)
rescue OpenURI::HTTPError => http_error
# bad status code returned
// do something here
status = http_error.io.status[0].to_i # => 3xx, 4xx, or 5xx
puts "Got a bad status code #{status}"
# http_error.message is the numeric code and text in a string
end
data = Nokogiri::HTML(doc)
I'm trying to find out how rails converts a hash such as (This is an example please do not take this literally I threw something together to get the concept by I know this query is the same as User.find(1)):
{
:select => "users.*",
:conditions => "users.id = 1",
:order => "username"
}
Into:
SELECT users.* FROM users where users.id = 1 ORDER BY username
The closest thing I can find is ActiveRecord::Base#find_every
def find_every(options)
begin
case from = options[:from]
when Symbol
instantiate_collection(get(from, options[:params]))
when String
path = "#{from}#{query_string(options[:params])}"
instantiate_collection(format.decode(connection.get(path, headers).body) || [])
else
prefix_options, query_options = split_options(options[:params])
path = collection_path(prefix_options, query_options)
instantiate_collection( (format.decode(connection.get(path, headers).body) || []), prefix_options )
end
rescue ActiveResource::ResourceNotFound
# Swallowing ResourceNotFound exceptions and return nil - as per
# ActiveRecord.
nil
end
end
I'm unsure as to how to modify this to just return what the raw mysql statement would be.
So after a few hours of digging I came up with an answer although its not great.
class ActiveRecord::Base
def self._get_finder_options options
_get_construct_finder_sql(options)
end
private
def self._get_construct_finder_sql(options)
return (construct_finder_sql(options).inspect)
end
end
adding this as an extension gives you a publicly accessible method _get_finder_options which returns the raw sql statement.
In my case this is for a complex query to be wrapped as so
SELECT COUNT(*) as count FROM (INSERT_QUERY) as count_table
So that I could still use this with the will_paginate gem. This has only been tested in my current project so if you are trying to replicate please keep that in mind.
I am using the code below to post a status and an image to twitter. The library is tmhOAuth, the image is coming from a blob in my mysql db as is the mime type and file name.
Basically if an image is found stored in the db row then it sends to twitter using a post to "update_with_media" if no image is found then it just sends the status update alone using "update". the "update" works fine, but I can't get the update with media working. I've tried several variations from various sites including Twitter's dev site but no joy. I'd really appreciate any help with this.
if(!empty($imgid)) {
$getimg = mysql_query("SELECT filedata, filename, mime_type FROM post_images WHERE id = $imgid");
$img = #mysql_fetch_array($getimg);
$imagedata = $img['filedata'];
$imagename = $img['filename'];
$imagetype = $img['mime_type'];
$theimg = "#{$imagedata};type=$imagetype;filename={$imagename}";
$hasimg = "y";
}
if($hasimg == "y") {
$code = $connection->request( 'POST',
$connection->url('https://upload.twitter.com/1/statuses/update_with_media.json'),
array(
'media[]' => $theimg,
'status' => $thetweet),
true, // use auth
true // multipart
);
} else {
$code = $connection->request('POST',
$connection->url('1.1/statuses/update'),
array('status' => $thetweet));
}
// A response code of 200 is a success
if ($code == 200) {
print "Tweet sent";
} else {
print "Error: $code";
}
in case anyone else has a similar issue my problem was formatting. Instead of using the variable "$theimg", I replaced
'media[]' => $theimg, with 'media[]' => "$imagedata; type=$imagetype; filename=$imagename"
Hope that helps someone...
Peace!