Fetching Reddit posts in React using Axios, weird JSON path - json

I'm trying to get the titles of reddit posts from a subreddit in React, using Axios for fetching.
I can fetch the JSON from here, and would like to get the data of each object (post/comment) separately (so I can show titles, post text and the like in the render() part of the component).
Here's the first lines of that JSON, pretty-printed:
{
"kind": "Listing",
"data": {
"modhash": "",
"dist": 27,
"children": [
{
"kind": "t3",
"data": {
"approved_at_utc": null,
"subreddit": "reactjs",
"selftext": "Previous two threads - [June 2019](https:\/\/www.reddit.com\/r\/reactjs\/comments\/bvxng8\/beginners_thread_easy_questions_june_2019\/) and [May 2019](https:\/\/www.reddit.com\/r\/reactjs\/comments\/bjgval\/beginners_thread_easy_questions_may_2019\/). \n\nGot questions about React or anything else in its ecosystem? Stuck making progress on your app? Ask away! We\u2019re a friendly bunch. \n\nNo question is too simple. \ud83e\udd14\n\n---------------------------------------------\n\n\ud83c\udd98 **Want Help with your Code?** \ud83c\udd98\n\n* **Improve your chances** by putting a minimal example to either [JSFiddle](https:\/\/jsfiddle.net\/Luktwrdm\/) or [Code Sandbox](https:\/\/codesandbox.io\/s\/new). Describe what you want it to do, and things you've tried. Don't just post big blocks of code!\n\n* **Pay it forward!** Answer questions even if there is already an answer - multiple perspectives can be very helpful to beginners. Also there's no quicker way to learn than [being wrong on the Internet](https:\/\/xkcd.com\/386\/). \n\n**Have a question regarding code \/ repository organization?**\n\nIt's most likely answered within this [tweet](https:\/\/twitter.com\/dan_abramov\/status\/1027245759232651270?lang=en).\n\n---------------------------------------------------\n\n**New to React?**\n\n**Check out the sub's sidebar!**\n\n\ud83c\udd93 Here are great, **free** resources! \ud83c\udd93\n\n* [Create React App](https:\/\/facebook.github.io\/create-react-app\/)\n* [Read the **official** Getting Started page](https:\/\/reactjs.org\/docs\/getting-started.html) on the docs.\n* [\/u\/acemarke](https:\/\/www.reddit.com\/u\/acemarke)'s [suggested resources for learning React](http:\/\/blog.isquaredsoftware.com\/2017\/12\/blogged-answers-learn-react\/)\n* [Kent Dodd's Egghead.io course](http:\/\/kcd.im\/beginner-react)\n* [Tyler McGinnis' 2018 Guide](https:\/\/medium.freecodecamp.org\/a-comprehensive-guide-to-react-js-in-2018-ba8bb6975597)\n* [Codecademy's React courses](https:\/\/www.codecademy.com\/catalog\/language\/javascript)\n* [Scrimba's React Course](https:\/\/scrimba.com\/g\/glearnreact)\n* [Robin Wieruch's Road to React](https:\/\/roadtoreact.com)\n\n-----\n\nAny ideas\/suggestions to improve this thread - feel free to comment here!\n\n----\n_Finally, an ongoing thank you to all who post questions and those who answer them. We're a growing community and helping each other only strengthens it!_",
"author_fullname": "t2_2aun3ozb",
"saved": false,
"mod_reason_title": null,
"gilded": 0,
"clicked": false,
"title": "Beginner's Thread \/ Easy Questions (July 2019)",
"link_flair_richtext": [
],
Here's my component (without the render part, just what happens after that fetch is complete - i.e. the componentDidMount() function):
componentDidMount() {
axios.get(`https://www.reddit.com/r/reactjs.json`)
.then (
res => {
const posts = res.data.data.children.map(obj => obj.data);
this.setState({posts});
}
);
}
This works perfectly.
My question is - why?
My logic would go - get res -> look for data -> look for children -> look for data, then map() that data object into the data object for posts.
Instead, what seems to work is - get res -> look for data -> look for data -> look for children, then do the mapping.

That's how axios parses the response json object. They store it under their own personal data key. It just so happens that within the json object provided by Reddit, they also have a field called data that holds the children array you want.
It might be more helpful to name your variables like so:
componentDidMount() {
axios.get(`https://www.reddit.com/r/reactjs.json`)
.then((res) => {
const redditJson = res.data
const posts = redditJson.data.children.map(obj => obj.data)
this.setState({
posts: posts
})
})
}

Related

Google books api returns missing parameters

I am making a react app that searches for a book by title and returns the results.
It's mostly working fine, but for some titles searched (such as "hello") it can't get the results because the parameters are missing.
Specially, the "amount" value is missing, and it can get me e-books that are not for sale even if I add the filter=paid-ebooks param while fetching the api. Using projection=full doesn't help either.
For example, when I call the api with
https://www.googleapis.com/books/v1/volumes?printType=books&filter=paid-ebooks&key=${APIKEY}
and use the fetched data inside books array in reactjs:
this.props.books.map((book, index) => {
return (
<CardItem
key={index}
title={book.volumeInfo.title}
authors={book.volumeInfo.authors ?
book.volumeInfo.authors.join(', ') :
"Not provided"}
price={book.saleInfo.listPrice.amount}
publisher={book.volumeInfo.publisher}
addToCart={() =>
this.props.addItem(this.props.books[index])}
/>
)
})
One of the results it gets is like this:
"saleInfo": {
"country": "TR",
"saleability": "NOT_FOR_SALE",
"isEbook": false
}
While it should be like, what's expected is :
"saleInfo": {
"country": "TR",
"saleability": "FOR_SALE",
"isEbook": true,
"listPrice": {
"amount": 17.23,
"currencyCode": "TRY"
}
And trying to search with this api answer throws the error :
TypeError: Cannot read property 'amount' of undefined
price={book.saleInfo.listPrice.amount}
As you can see in react code's authors, this issue comes up with authors parameter too, which I've bypassed as seen in the code. But I cannot do the same with amount. Is this a known error in Google Books API or is there a way to prevent this? I don't understand why it still returns me e-books that are not for sale even with filter=paid-ebooks param.
I have not dug into the API documentation. An ideal solution would be a query param that only sends back books with a list price (like you tried with filter=paid-ebooks). Because that's not working, a simple fix would be to filter your results once you get them.
Assuming the response contains an array of book objects, it would look something like this:
const paidBooks = apiResponse.data.filter(book => book.listPrice)
This code will take the response from the API, and filter out all books that do not contain a truthy value for listPrice
That totally right, actually i never used react but the same logic try using try{ }catch(error){} for those missing data

Weird Json in Elasticsearch

I am learning the ELK Stack for log analysis and one thing I can't digest is whenever we have to populate index, the JSON format we have to use is not really a JSON.
For Eg:
Following is from tutorialspoint website, to populate a "Schools" index with
"School" type for bulk population :-
{ "index":{ "_index":"schools", "_type":"school", "_id":"1" }}
{"name":"Central School", "description":"CBSE Affiliation", "street":"Nagan","city":"paprola", "state":"HP", "zip":"176115", "location"[31.8955385, 76.8380405],"fees":2000, "tags":["Senior Secondary", "beautiful campus"], "rating":"3.5"}
{ "index":{ "_index":"schools", "_type":"school", "_id":"2" }}
{ "name":"Saint Paul School", "description":"ICSE Afiliation", "street":"Dawarka", "city":"Delhi", "state":"Delhi", "zip":"110075","location":[28.5733056, 77.0122136], "fees":5000,"tags":["Good Faculty", "Great Sports"], "rating":"4.5"}
Now I do understand space is the delimiter here and without giving one entity (index info or school info) in one line,it will return with parsing error. But according to my understanding and this (for reference), I think we do need a top level Key(Schools in below) with array ([ ]) of set of data(index info and School info) in above maybe like below :
{"Schools" : [
{
{"index":{ "_index":"schools", "_type":"school", "_id":"1" }},
{"name":"Central School", "description":"CBSE Affiliation", "street":"Nagan","city":"paprola", "state":"HP", "zip":"176115", "location"[31.8955385, 76.8380405],"fees":2000, "tags":["Senior Secondary", "beautiful campus"], "rating":"3.5"}
},
{
{ "index":{ "_index":"schools", "_type":"school", "_id":"2" }},
{ "name":"Saint Paul School", "description":"ICSE Afiliation", "street":"Dawarka", "city":"Delhi", "state":"Delhi", "zip":"110075","location":[28.5733056, 77.0122136], "fees":5000,"tags":["Good Faculty", "Great Sports"], "rating":"4.5"}
}
]}
I also tried changing the type of payload in POSTMAN from JSON(applicaton/json) to TEXT but it gives back an error stating that former payload is required.
I am clearly missing something here. It will be very helpful if someone can reason above behaviour.

How to parse the external json in gulp-jade?

I'm using jade templates for my templating system, passing a json file in as the jade locals via my gulpfile.js, but I can't seem to deep dive into the json. I feel like I'm overlooking something basic, but can't find an example online anywhere.
gulpfile.js:
Passes the json file into jade
gulp.task('html', function() {
gulp.src('./markup/*.jade')
.pipe(jade({
pretty: true,
locals: JSON.parse( fs.readFileSync('./markup/data/website_data.json', { encoding: 'utf8' }) )
}).on('error', gutil.log))
.pipe(gulp.dest('../'))
});
Then in my jade, I just pass the locals into a variable for the sake of readability.
- var employees = locals
And I can loop through json that is one level deep:
jade:
for employee in employees
if employee.Tier === 'Founder'
li
button(data-bio="#{employee.LastName.toLowerCase()}")
img(src="/public/img/employees/#{employee.FirstName.toLowerCase()}-#{employee.LastName.toLowerCase()}.jpg", alt="#{employee.FirstName} #{employee.LastName} | #{employee.Title}")
strong #{employee.FirstName} #{employee.LastName}
| #{employee.Title}
json:
[
{
"FirstName":"John",
"LastName":"Doe",
"Title":"Strategist",
"Tier":"Founder",
"Description":"",
"Email":"",
"Links":""
},
...
]
But that has only worked for me if the items I loop through are in the root, as soon as I make the json one level deeper, I can't get it to work based on the key. I want to make the json deeper so I can different sections of a site in it instead of just the employees.
[{
"employees": [
{
"FirstName":"Jason",
"LastName":"Bellinger",
"Title":"Lorem Ipsum",
"Tier":"",
"Description":"",
"Email":"",
"Links":""
},
...
]
}]
I tried a few different approaches to to dig into the json and have failed thus far.
ATTEMPT 1: adjust the variable call and keep the same loop
- var employees = locals.employees
And I get 'Cannot read property 'length' of undefined' in the terminal running $gulp watch
Also try:
- var employees = locals['employees']
to the same result.
ATTEMPT 2: don't use the var and call locals directly in my loop
for employee in locals.employees
AND
for employee in locals["employees"]
And I end up with the same error.
ATTEMPT 3:
keep the var and adjust the loop
- var employees = locals
...
for employee in employees
li #{employee.LastName}
Then I don't get an error in Terminal, but I don't get any content. It produces one empty li.
So then, I try to go a layer deeper in the loop with:
for employee in employees[0]
li #{employee.LastName}
AND
for employee in employees['employees']
li #{employee.LastName}
AND I still get no error and one empty li
I've parsed enough json in my day and jade seems simple enough, I have to be overlooking something basic. Someone please humble me.
I also dabbled in gulp-data, but I'm getting the data into jade with my approach, so I think it's my approach in jade...
You need to access the array inside you locals variable.
The length of local = 1 and that is the entire array of employees.
You'll need to set employees = to the array inside of the locals variable with:
"- var employees = locals[0].employees"
I knew it was something basic. I reverted everything back to the original setup and changed the var and this is working.
- var employees = locals[0]['employees']
Truth be told, I thought I already tried this, but went back and tried again...

Parsing Google Custom Search API for Elasticsearch Documents

After retrieving results from the Google Custom Search API and writing it to JSON, I want to parse that JSON to make valid Elasticsearch documents. You can configure a parent - child relationship for nested results. However, this relationship seems to not be inferred by the data structure itself. I've tried automatically loading, but not results.
Below is some example input that doesn't include things like id or index. I'm trying to focus on creating the correct data structure. I've tried modifying graph algorithms like depth-first-search but am running into problems with the different data structures.
Here's some example input:
# mock data structure
google = {"content": "foo",
"results": {"result_one": {"persona": "phone",
"personb": "phone",
"personc": "phone"
},
"result_two": ["thing1",
"thing2",
"thing3"
],
"result_three": "none"
},
"query": ["Taylor Swift", "Bob Dole", "Rocketman"]
}
# correctly formatted documents for _source of elasticsearch entry
correct_documents = [
{"content":"foo"},
{"results": ["result_one", "result_two", "result_three"]},
{"result_one": ["persona", "personb", "personc"]},
{"persona": "phone"},
{"personb": "phone"},
{"personc": "phone"},
{"result_two":["thing1","thing2","thing3"]},
{"result_three": "none"},
{"query": ["Taylor Swift", "Bob Dole", "Rocketman"]}
]
Here is my current approach this is still a work in progress:
def recursive_dfs(graph, start, path=[]):
'''recursive depth first search from start'''
path=path+[start]
for node in graph[start]:
if not node in path:
path=recursive_dfs(graph, node, path)
return path
def branching(google):
""" Get branches as a starting point for dfs"""
branch = 0
while branch < len(google):
if google[google.keys()[branch]] is dict:
#recursive_dfs(google, google[google.keys()[branch]])
pass
else:
print("branch {}: result {}\n".format(branch, google[google.keys()[branch]]))
branch += 1
branching(google)
You can see that recursive_dfs() still needs to be modified to handle string, and list data structures.
I'll keep going at this but if you have thoughts, suggestions, or solutions then I would very much appreciate it. Thanks for your time.
here is a possible answer to your problem.
def myfunk( inHole, outHole):
for keys in inHole.keys():
is_list = isinstance(inHole[keys],list);
is_dict = isinstance(inHole[keys],dict);
if is_list:
element = inHole[keys];
new_element = {keys:element};
outHole.append(new_element);
if is_dict:
element = inHole[keys].keys();
new_element = {keys:element};
outHole.append(new_element);
myfunk(inHole[keys], outHole);
if not(is_list or is_dict):
new_element = {keys:inHole[keys]};
outHole.append(new_element);
return outHole.sort();

Ember many-to-many Ids are not included in JSON payload on post?

I have a meeting and a sales-rep models, The relation is ManyToMany.
The problem is, When I want to create a New meeting, and assign existing salesReps to it (They are already saved to the store), But The salesReps IDS are not included in The post action caused by model.save() (not even an empty array), To make it more clear, Here is what my code looks like:
meeting.coffee:
Meeting = DS.Model.extend
client: DS.belongsTo('client')
salesReps: DS.hasMany('sales-rep')
memo: DS.attr('string')
startDate: DS.attr('date')
duration: DS.attr()
sales-rep.coffee:
SalesRep = DS.Model.extend
meetings: DS.hasMany('meeting')
firstName: DS.attr('string')
lastName: DS.attr('string')
title: DS.attr('string')
meetings/new.coffee (the save action am using inside new meeting controller):
save: ->
meeting = #get('model')
meeting.set('client', #get('client'))
meeting.get('salesReps').pushObjects(#get('salesReps.content'))
meeting.save().then =>
#transitionToRoute 'meetings'
the JSON payload: ( POST http://localhost:4200/api/meetings)
meeting: {memo: null, start_date: null, duration: "00:15", client_id: null}
client_id: null
duration: "00:15"
memo: null
start_date: null
No matter what, There is no ANY trace of the salesReps ids in the payload!!
What I tried so far:
Setting the hasMany relation in the meeting model only.
Setting {async: true}, and then {async: false}, on both SalesRep, And
then on one of them
spending almost 2 days googling and reading all related posts in here
with no luck
Any Help/hints/Advice, Is highly appreciated
I will write the solution I found after endless reading and researching, I will write the full details, and trial/failure i've been through, Because no one, no one EVER should have to spend more than 3 days trying to fix something like that!!
I am using Ember-cli, So, there are a files/directories structure am following:
First attempt:
Trying all combinations of async: true, embedded true and what not.
Result, No luck
Second attempt:
in app/serializers/ I added the following serializer file:
meeting.coffee
`import DS from "ember-data"`
`import Ember from "ember"`
`import config from '../config/environment'`
get = Ember.get
serializer = DS.RESTSerializer.extend
serializeHasMany: (record, json, relationship) ->
rel_ids = get(record, relationship.key).map (rel) -> get(rel, 'id') || []
json["#{relationship.key.underscore().singularize()}_ids"] = rel_ids
json
`export default serializer`
result:
Adding this serializer, And I finally was able to send sales_rep_ids:[] array to the controller! and I could confirm that the server is saving the accociations as required.
But, When listing meetings, I was not able to list the associated salesReps, So, I checked the JSON am getting from the server, and it was correct (salesReps Ids were included!) But still not listed in Ember
Third Attempt:
After more reading and endless head-banging-against-the-wall, Changing ONE line fixed the problem!:
in app/serializers/meeting.coffee
change serializer = DS.RESTSerializer.extend to
serializer = DS.ActiveModelSerializer.extend
And Voila! Saved to the back-end, And listed correctley as association in ember!
This solution is a result of 3+ of constant headache, Am posting it here hopefully it might be helpful to someone facing the same problem, I can't claim that it's my own solution, but, It's the result of reading many people's code.
am not sure if it's the Ember way to do so, So, Any suggestions, Improvements Ideas and thought are welcome.