I have retrieve this key/value from a hash using the facebook api
"message":"Next Practice:\n\nDate: 04.05.2014\nTime: 10:00-12:00\nVenue: Llandaff Fields\n\nAll welcome
but when i save it to my model i seem to lose all the special characters, i.e \n. Is there a way to save the value as it is returned so that i can use the \n when outputting to my view using .html_safe
This is how i am retrieving the data
def get_feed
fb_access_token = access_token
uri = URI(URI.escape "https://graph.facebook.com/#{VANDALS_ID}/posts/?#{fb_access_token}")
response = HTTParty.get(uri)
results = JSON.parse(response.body)
formatted_data(results)
end
anything i need to be doing to keep that string with \n left in it
Thanks
When I run the following code:
raw_json = '{"message":"Next Practice:\n\nDate: 04.05.2014\nTime: 10:00-12:00\nVenue: Llandaff Fields\n\nAll welcome"}'
parsed_json = JSON.parse(raw_json)
puts parsed_json['message']
# => Next Practice:
# => Date: 04.05.2014
# => Time: 10:00-12:00
# => Venue: Llandaff Fields
# => All welcome
So the \n is kept (it is parsed, and shown as real new-line). I also don't believe that saving this to your model erased the new lines.
Where I think your real problem lies is that in HTML new lines (\n) are not rendered as new lines at all, but as spaces. To render them as new lines, you need to replace them with breaks (<br>).
So you can try using the following on your ERB:
<div class=message><%= feed.message.gsub("\n", "<br>").html_safe %></div>
Your new-lines will now be rendered on the page.
Related
i'm working on a scrapping project but i got a problem:
I wanna get all the data of https://coinmarketcap.com/all/views/all/ with nokigiri
but i only get 20 crypto name on the 200 loaded with nokogiri
the code:
ruby
require 'nokogiri'
require 'open-uri'
require 'rubygems'
def scrapper
return doc = Nokogiri::HTML(URI.open('https://coinmarketcap.com/all/views/all/'))
end
def fusiontab(tab1,tab2)
return Hash[tab1.zip(tab2)]
end
def crypto(page)
array_name=[]
array_value=[]
name_of_crypto=page.xpath('//tr//td[3]')
value_of_crypto=page.xpath('//tr//td[5]')
hash={}
name_of_crypto.each{ |name|
array_name<<name.text
}
value_of_crypto.each{|price|
array_value << price.text
}
hash=fusiontab(array_name,array_value)
return hash
end
puts crypto(scrapper)
can you help me to get all the cryptocurrencies ?
The URL you're using does not generate all the data as HTML; a lot of it is rendered after the page has been loaded.
Looking at the source code for the page, it appears that the data is rendered from a JSON script, embedded in the page.
it took quite some time to find the objects in order to work out what part of the JSON data has the contents that you want to work with:
The JSON object within the HTML, as a String object
page.css('script[type="application/json"]').first.inner_html
The JSON String converted to a real JSON Hash
JSON.parse(page.css('script[type="application/json"]').first.inner_html)
the position inside the JSON or the Array of Crypto Hashes
my_json["props"]["initialState"]["cryptocurrency"]["listingLatest"]["data"]
pretty print the first "crypto"
2.7.2 :142 > pp cryptos.first
{"id"=>1,
"name"=>"Bitcoin",
"symbol"=>"BTC",
"slug"=>"bitcoin",
"tags"=>
["mineable",
"pow",
"sha-256",
"store-of-value",
"state-channel",
"coinbase-ventures-portfolio",
"three-arrows-capital-portfolio",
"polychain-capital-portfolio",
"binance-labs-portfolio",
"blockchain-capital-portfolio",
"boostvc-portfolio",
"cms-holdings-portfolio",
"dcg-portfolio",
"dragonfly-capital-portfolio",
"electric-capital-portfolio",
"fabric-ventures-portfolio",
"framework-ventures-portfolio",
"galaxy-digital-portfolio",
"huobi-capital-portfolio",
"alameda-research-portfolio",
"a16z-portfolio",
"1confirmation-portfolio",
"winklevoss-capital-portfolio",
"usv-portfolio",
"placeholder-ventures-portfolio",
"pantera-capital-portfolio",
"multicoin-capital-portfolio",
"paradigm-portfolio"],
"cmcRank"=>1,
"marketPairCount"=>9158,
"circulatingSupply"=>18960043,
"selfReportedCirculatingSupply"=>0,
"totalSupply"=>18960043,
"maxSupply"=>21000000,
"isActive"=>1,
"lastUpdated"=>"2022-02-16T14:26:00.000Z",
"dateAdded"=>"2013-04-28T00:00:00.000Z",
"quotes"=>
[{"name"=>"USD",
"price"=>43646.858047604175,
"volume24h"=>20633664171.70021,
"marketCap"=>827546305397.4712,
"percentChange1h"=>-0.86544168,
"percentChange24h"=>-1.6482985,
"percentChange7d"=>-0.73945082,
"lastUpdated"=>"2022-02-16T14:26:00.000Z",
"percentChange30d"=>2.18336134,
"percentChange60d"=>-6.84146969,
"percentChange90d"=>-26.08073361,
"fullyDilluttedMarketCap"=>916584018999.69,
"marketCapByTotalSupply"=>827546305397.4712,
"dominance"=>42.1276,
"turnover"=>0.02493355,
"ytdPriceChangePercentage"=>-8.4718}],
"isAudited"=>false,
"rank"=>1,
"hasFilters"=>false,
"quote"=>
{"USD"=>
{"name"=>"USD",
"price"=>43646.858047604175,
"volume24h"=>20633664171.70021,
"marketCap"=>827546305397.4712,
"percentChange1h"=>-0.86544168,
"percentChange24h"=>-1.6482985,
"percentChange7d"=>-0.73945082,
"lastUpdated"=>"2022-02-16T14:26:00.000Z",
"percentChange30d"=>2.18336134,
"percentChange60d"=>-6.84146969,
"percentChange90d"=>-26.08073361,
"fullyDilluttedMarketCap"=>916584018999.69,
"marketCapByTotalSupply"=>827546305397.4712,
"dominance"=>42.1276,
"turnover"=>0.02493355,
"ytdPriceChangePercentage"=>-8.4718}}
}
the value of the first "crypto"
crypto.first["quote"]["USD"]["price"]
the key that you use in your Hash for the first "crypto"
crypto.first["symbol"]
put it all together and you get the following code (looping through each "crypto" with each_with_object)
require `json`
require 'nokogiri'
require 'open-uri'
...
def crypto(page)
my_json = JSON.parse(page.css('script[type="application/json"]').first.inner_html)
cryptos = my_json["props"]["initialState"]["cryptocurrency"]["listingLatest"]["data"]
hash = cryptos.each_with_object({}) do |crypto, hsh|
hsh[crypto["name"]] = crypto["quote"]["USD"]["price"]
end
return hash
end
puts crypto(scrapper);
I want to append multiple list items to a JSON file, but it creates a list within a list, and therefore I cannot acces the list from python. Since the code is overwriting existing data in the JSON file, there should not be any list there. I also tried it by having just an text in the file without brackets. It just creates a list within a list so [["x", "y","z"]] instead of ["x", "y","z"]
import json
filename = 'vocabulary.json'
print("Reading %s" % filename)
try:
with open(filename, "rt") as fp:
data = json.load(fp)
print("Data: %s" % data)#check
except IOError:
print("Could not read file, starting from scratch")
data = []
# Add some data
TEMPORARY_LIST = []
new_word = input("give new word: ")
TEMPORARY_LIST.append(new_word.split())
print(TEMPORARY_LIST)#check
data = TEMPORARY_LIST
print("Overwriting %s" % filename)
with open(filename, "wt") as fp:
json.dump(data, fp)
example and output with appending list with split words:
Reading vocabulary.json
Data: [['my', 'dads', 'house', 'is', 'nice']]
give new word: but my house is nicer
[['but', 'my', 'house', 'is', 'nicer']]
Overwriting vocabulary.json
So, if I understand what you are trying to accomplish correctly, it looks like you are trying to overwrite a list in a JSON file with a new list created from user input. For easiest data manipulation, set up your JSON file in dictionary form:
{
"words": [
"my",
"dad's",
"house",
"is",
"nice"
]
}
You should then set up functions to separate your functionality to make it more manageable:
def load_json(filename):
with open(filename, "r") as f:
return json.load(f)
Now, we can use those functions to load the JSON, access the words list, and overwrite it with the new word.
data = load_json("vocabulary.json")
new_word = input("Give new word: ").split()
data["words"] = new_word
write_json("vocabulary.json", data)
If the user inputs "but my house is nicer", the JSON file will look like this:
{
"words": [
"but",
"my",
"house",
"is",
"nicer"
]
}
Edit
Okay, I have a few suggestions to make before I get into solving the issue. Firstly, it's great that you have delegated much of the functionality of the program over to respective functions. However, using global variables is generally discouraged because it makes things extremely difficult to debug as any of the functions that use that variable could have mutated it by accident. To fix this, use method parameters and pass around the data accordingly. With small programs like this, you can think of the main() method as the point in which all data comes to and from. This means that the main() function will pass data to other functions and receive new or edited data back. One final recommendation, you should only be using all capital letters for variable names if they are going to be constant. For example, PI = 3.14159 is a constant, so it is conventional to make "pi" all caps.
Without using global, main() will look much cleaner:
def main():
choice = input("Do you want to start or manage the list? (start/manage)")
if choice == "start":
data = load_json()
words = data["words"]
dictee(words)
elif choice == "manage":
manage_list()
You can use the load_json() function from earlier (notice that I deleted write_json(), more on that later) if the user chooses to start the game. If the user chooses to manage the file, we can write something like this:
def manage_list():
choice = input("Do you want to add or clear the list? (add/clear)")
if choice == "add":
words_to_add = get_new_words()
add_words("vocabulary.json", words_to_add)
elif choice == "clear":
clear_words("vocabulary.json")
We get the user input first and then we can call two other functions, add_words() and clear_words():
def add_words(filename, words):
with open(filename, "r+") as f:
data = json.load(f)
data["words"].extend(words)
f.seek(0)
json.dump(data, f, indent=4)
def clear_words(filename):
with open(filename, "w+") as f:
data = {"words":[]}
json.dump(data, f, indent=4)
I did not utilize the load_json() function in the two functions above. My reasoning for this is because it would call for opening the file more times than needed, which would hurt performance. Furthermore, in these two functions, we already need to open the file, so it is okayt to load the JSON data here because it can be done with only one line: data = json.load(f). You may also notice that in add_words(), the file mode is "r+". This is the basic mode for reading and writing. "w+" is used in clear_words(), because "w+" not only opens the file for reading and writing, it overwrites the file if the file exists (that is also why we don't need to load the JSON data in clear_words()). Because we have these two functions for writing and/or overwriting data, we don't need the write_json() function that I had initially suggested.
We can then add to the list like so:
>>> Do you want to start or manage the list? (start/manage)manage
>>> Do you want to add or clear the list? (add/clear)add
>>> Please enter the words you want to add, separated by spaces: these are new words
And the JSON file becomes:
{
"words": [
"but",
"my",
"house",
"is",
"nicer",
"these",
"are",
"new",
"words"
]
}
We can then clear the list like so:
>>> Do you want to start or manage the list? (start/manage)manage
>>> Do you want to add or clear the list? (add/clear)clear
And the JSON file becomes:
{
"words": []
}
Great! Now, we implemented the ability for the user to manage the list. Let's move on to creating the functionality for the game: dictee()
You mentioned that you want to randomly select an item from a list and remove it from that list so it doesn't get asked twice. There are a multitude of ways you can accomplish this. For example, you could use random.shuffle:
def dictee(words):
correct = 0
incorrect = 0
random.shuffle(words)
for word in words:
# ask word
# evaluate response
# increment correct/incorrect
# ask if you want to play again
pass
random.shuffle randomly shuffles the list around. Then, you can iterate throught the list using for word in words: and start the game. You don't necessarily need to use random.choice here because when using random.shuffle and iterating through it, you are essentially selecting random values.
I hope this helped illustrate how powerful functions and function parameters are. They not only help you separate your code, but also make it easier to manage, understand, and write cleaner code.
I need to send an mail through my perl script:
use MIME::Lite;
GetOptions(
'mail|m:s' =>\$recipients
)
my #recipients = split(/,/, $recipients);
sub sendmail {
chomp #recipients;
$to = "#recipients";
$from = 'xyz#gmail.com';
$subject = 'Output';
$msg = MIME::Lite->new(
From => $from,
To => $to,
Subject => $subject,
Data => $mailbody
);
$msg->attr("content-type" => "text/html");
$msg->send;
print "Email Sent Successfully\n";
}
Here, I am appending output to mailbody like:
mailbody.=qq(Welcome\n);
which are statements containing output which has to be emailed.
How can I format this output to include additional lines and/or spaces? I think \n or even lots of spaces are also not accepted by mailbody.=qq(Welcome\n);, which results in a single line of content.
You've said:
"content-type" => "text/html"
This means you are writing HTML (or at least telling the email client that you are).
If you want to send plain text and format it using literal whitespace, then don't claim to send HTML!
"content-type" => "text/plain"
If you want to send HTML, then write HTML which has the formatting you want (e.g. use <p>...</p> to indicate paragraphs, and style="text-indent: 1em;" to indent the first line of a block).
If you want to send HTML (as your content-type implies) then you need to use HTML tags (<br>, <p>...</p>, etc) to format your text.
If you want to use newlines to format your text, then you need to change your content-type to text/plain.
A few other suggestions:
I wouldn't recommend MIME::Lite. These days, you should use something from the Email::* namespace - perhaps Email::Sender or Email::Stuffer (I think I mentioned this yesterday).
You should chomp() $recipients before splitting it into #recipients.
You use #recipients inside sendmail(), but you don't pass the array to the subroutine. It's bad practice to use global variables within a subroutine - it renders your subroutine less portable and harder to maintain.
MIME::Lite expects multiple recipients in a comma-separated string. So splitting $recipients into #recipients is pointless.
So I have a JSON file of a somewhat known format { String => JSON::Type, ... }. So it is basically of type Hash(String, JSON::Type). But when I try and read it from file to memory like so: JSON.parse(File.read(#cache_file)).as(Hash(String, JSON::Type)) I always get an exception: can't cast JSON::Any to Hash(String, JSON::Type)
I'm not sure how I am supposed to handle the data if I can't cast it.
What I basically want to do is the following:
save JSON::Type data under a String key
replace JSON::Type data with other JSON::Type data under a String key
And of course read from / write to file...
Here's the whole thing I've got so far:
class Cache
def initialize(#cache_file = "/tmp/cache_file.tmp")
end
def cache(cache_key : (String | Symbol))
mutable_cache_data = data
value = mutable_cache_data[cache_key.to_s] ||= yield.as(JSON::Type)
File.write #cache_file, mutable_cache_data
value
end
def clear
File.delete #cache_file
end
def data
unless File.exists? #cache_file
File.write #cache_file, {} of String => JSON::Type
end
JSON.parse(File.read(#cache_file)).as(Hash(String, JSON::Type))
end
end
puts Cache.new.cache(:something) { 10 } # => 10
puts Cache.new.cache(:something) { 'a' } # => 10
TL;DR I want to read a JSON file into a Hash(String => i_dont_care), replace a value under a given key name and serialize it to file again. How do I do that?
JSON.parse returns an JSON::Any, not a Hash so you can't cast it. You can however access the underlying raw value as JSON.parse(file).raw and cast this as hash.
Then your code is basically working (I've fixed a few error): https://carc.in/#/r/28c1
You can use use Hash(String, JSON::Type).from_json(File.read(#cache_file)). Hopefully you can restrict the type of JSON::Type down to something more sensible too. JSON::Any and JSON.parse_raw are very much a last resort compared to simply representing your schema using Hash, Array and custom types using JSON.mapping.
I think I've written myself in a corner. Basically, I have an array of hashes, like so.
my_hashes = [{"colorName"=>"first", "hexValue"=>"#f00"}, {"colorName"=>"green", "hexValue"=>"#0f0"},
{"colorName"=>"blue", "hexValue"=>"#00f"}, {"colorName"=>"cyan", "hexValue"=>"#0ff"},
{"colorName"=>"magenta", "hexValue"=>"#f0f"}, {"colorName"=>"yellow", "hexValue"=>"#ff0"},
{"colorName"=>"black", "hexValue"=>"#000"}]
I need to use JSON.parse to eventually be able to transform these hashes into CSV format. The only problem is I can't get JSON.parse to work as long as the "=>" symbol is present. I've tried just doing a regular gsub('=>', ':') but it appears that I cannot use it as this is an array of hashes. I've tried variations of the following method:
my_hashes.each do |hash|
hash.each do |key, value|
key.gsub!('=>', ':')
value.gsub!('=>', ':')
end
end
I need these hash values to stay intact, so even if I transform them intro strings, if I transform them back they'll still have the '=>' symbol available. Any advice?
Changing => to : wouldn't make a Ruby hash to a JSON object. And in fact you cannot just change a hash like that at all. Because the written representation of a hash is not the same as the interpreted version in memory.
But that doesn't solve your problem: You need a JSON representation of a Ruby hash, just use to_json:
my_hashes = [
{"colorName"=>"first", "hexValue"=>"#f00"},
{"colorName"=>"green", "hexValue"=>"#0f0"},
{"colorName"=>"blue", "hexValue"=>"#00f"},
{"colorName"=>"cyan", "hexValue"=>"#0ff"},
{"colorName"=>"magenta", "hexValue"=>"#f0f"},
{"colorName"=>"yellow", "hexValue"=>"#ff0"},
{"colorName"=>"black", "hexValue"=>"#000"}
]
require 'json'
my_hashes.to_json
#=> "[{"colorName":"first","hexValue":"#f00"},{"colorName":"green","hexValue":"#0f0"},{"colorName":"blue","hexValue":"#00f"},{"colorName":"cyan","hexValue":"#0ff"},{"colorName":"magenta","hexValue":"#f0f"},{"colorName":"yellow","hexValue":"#ff0"},{"colorName":"black","hexValue":"#000"}]"
my_hashes=[{"colorName"=>"first", "hexValue"=>"#f00"}]
new_data = my_hashes.to_json.gsub(/\=\>/, ':')
data = Json.parse new_data