What is the least resource intensive to perform "findManyOrCreate" in Laravel eloquent? - mysql

I have an array of post codes coming from an input:
$postCodes = collect(["BK13TV", "BK14TV", "BK15TV", "BK16TV"]);
In my database I already have two of the post codes - "BK13TV", "BK16TV".
So I would like to run something like this:
$postCodeModels = PostCode::findManyOrCreate($postCodes->map($postCode) {
return ['code' => $postCode]
})
My initial approach was to load all the post codes, then diff them against the postCodes from the input like so:
PostCode::createMany($postCodes->diff(PostCode::all()->pluck('code')))
However, here it means that I am loading the entire content of post_codes table, which just seems wrong.
In the ideal case, this would return all post code models matching the passed post codes as well as would create entries for post codes that did not exist in the database.

First I need to retrieve existing postcodes:
$existingPostCodes = PostCode::whereIn('code', $postCodes)->get();
The find all the post codes in the input, that are not stored yet in database:
$newPostCodes = $postCodes->diff($existingPostCode->pluck('code'));
Finally retrieve all the new post codes as models:
$postCodeModels = PostCode::whereIn('code', $postCodes)->get();
Admittedly, this still takes three queries, but does eliminate the crazy stuff of loading an entire table worth of data.

Related

How to count total amount of comments per posts in nodejs, react js and sequelize?

I am working on a project where a user can ask and answer a question so in the comments filed I have trouble with it.
the comments table is working well and displaying the data and can delete the data, but I want to count the total number of comments per post.
These blooks of codes are working well but it counts all the comments in the table, that's what I don't want, I want to display the total number of each post not all the amount of the comments in one post.
router.get("/commentsCounter/:id", async (req, res) => {
const id = req.params.id;
const commentsCounter = await Comments.count(id);
res.json({commentsCounter});
console.log('commentsCounter', commentsCounter)
});
in the comment table, I have commentbody,author name, and postId.
Do any suggestions, please?
Sequelize count method takes an object as its argument which you can specify your post ID in its where attribute. So in your case you want to change your query to
const commentsCounter = await Comments.count({ where: { postId: id} });
Without seeing the full code (DB call, what the select statement looks like etc) it's hard to say..
But, typically if you DB is setup to have say 'posts' and 'comments' tables and the comment table has something like id AND parentID (the ID of the parent POST in the 'posts' table) you should be able to do it fairly easily using javascript OR the DB query itself
If you are doing it on the DB side, you would typically add a COUNT to the select statement OR once you have the results of the query you can just do a 'length' of the returned object to get the total returned.
*** Edit ***
As #Pouya had pointed out, you can doing via the SQL query as show in his post, this will count the number of responses based on the select, this does require a query per though, again without knowing the full flow of this this may or may not be what you are looking for but would give you the count based on the specific query.
Now, if you are say, returning a large object of ALL replies and want to get just the responses for a specific subset you will want to look at filtering your return object by the right key/value
something like
let myArr = [{id:'123',name:'item1'},{id:'234',name:"item2"},
{id:'123',name:"item3"}]
//filter just id 123
// filter syntax is [array].filter(variable => [key you want to filter on].includes("[value to filter on]")
let newArray = myArr.filter(p => p.id.includes('123'));
//our new array
console.log(newArray)
Should get you just a subset of the array, then you can just do
let replyCount = newArray.length

What is the URL when I DELETE an object

I'm running a local server playing around with an API using Django. I have a model called 'Users' populated with a few objects, and am using DefaultRouter.
I want to know what the URL would be if I were to DELETE a specific object from this model. For example, if I wanted to GET a user with an ID of 1 in this model, the URL would be: "localhost:8000/Users/1/". What would the equivalent be to DELETE this user?
I found an explanation of this on the REST API website (below), however, I don't understand what any of the syntaxes means.
What is {prefix}, {url_path}, {lookup} and [.format]? If anyone could provide an example of what this might be using a localhost that would be really helpful.
Thanks
Let us take an example of an API (URL) to update book data with id (pk) being 10. It would look something like this:
URL: http://www.example.com/api/v1/book/10/
Method: PUT/PATCH
With some data associated.
If you want to delete you just need to change method to DELETE instead of put or patch.
Regarding your second question lets compare the url with the parameters.
prefix: http://www.example.com/api/v1/book
lookup: 10
format: It specifies what type of data do you expect when you hit the API. Generally it is considered to be json.
url_path: In general, every thing after look up except query string is considered to be url_path.

Rails massive data upload and nested records

I have to update a lot of data to mysql (~100Mio records!). Some records already exists, some have to be created. I also have to create some nested resources for each record.
I know the activerecord-import gem but as far as i know it can't handle nested records (or only with ugly workarounds). The issue is that I dont know the ID's for all nested records before they are created - and creating them in single queries takes time.
So lets say there is a model called Post and can have many Comments. My current code looks like this:
Post.transaction do
import_posts.each do |import_post|
post = Post.find_or_initialize_by(somevalue: import_post['somevalue']
post.text = import_post['text']
import_post['comments'].each do |import_comment|
comment = post.comments.find_or_initialize_by(someothervalue: import_comment['someothervalue'])
comment.text = import_comment['text']
end
post.save(validate: false) #Dont need validation - saves some time
end
end
This is just an example and it works but its far away from 'damn fast'. Are there any ideas how to speed up the data upload? Am I totally wrong?
Im working with Rails5 and ruby 2.4.
Thanks in advance!

What is the correct way to handle a POST request in Django class based view?

In my class based view I am handling a post request (which is an AJAX call).I am inserting some data in a database table and returning some json to the frontend.
def post(self,request,*args,**kwargs):
result_data = {}
doc = kwargs['doc']
doc_obj = Document.objects.get_document(doc)
doc_id = doc_obj.doc_id
reference_obj = Reference.objects.save_to_db(request,doc_id)
friendly_name = reference_obj.friendly_name
result_data['friendly_name'] = friendly_name
return HttpResponse(json.dumps(result_data),content_type='application/json')
My question is that is this the right way of handling a post request?My doubt arises here - I am writing some data into database and returning a json of properties of the same data I have written to the database.
Should I do it differently, ie first storing some data in database and returning just id of the row and again making GET request on the same URL?
I would say if it is an API, it is best to minimize the number of queries by returning the whole object the way you did. In case it is to serve a web user interface, it is simpler to return the id and get the rest of the data, if it needs to be displayed, from the client itself.
Either way is fine. The gain is negligible. I have also seen applications give a local positive feedback and then replace with a negative one if the request was unsuccessful. The technique can improve responsiveness, which is always appreciated.
Django-rest-framework returns the json of the inserted object, so i would think it reasonable to copy that behaviour.

Valid to return different json-response depending on list or retrieve?

I am currently designing a Rest API and is a little stuck on performance matters for 2 of the use cases in the system:
List all campaigns (api/campaigns) - needs to return campaign data needed for listing and paging campaigns. Maybe return up to 1000 records and would take ages to retreive and return detailed data. The needed data can be returned in a single DB call.
Retrieve campaign item (api/campaigns/id) - need to return all data about the campaign and may take up to a second to run. Multiple DB calls is needed to get all campaign data for a single campaign.
My question is: Is it valid to return different json-responses to those 2 calls (if well documented) even if it regards the same resource? I am thinking that the list response is a sub set of the retreive-response. The reason for this is to make to save DB calls and bandwitdh + parsing.
Thanks in advance!
I think it's both fine and expected for /campaigns and /campaigns/{id} to return different information. I would suggest using query parameters to limit the amount of information you need to return. For instance, only return a URI to each player unless you see a ?expand=players query parameter, in which case you return detailed player information.