When I fire User.first.email in Rails - 4 console it gives me query like
SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT 1
By this query structure how it can get email column from database just only firing SELECT "users".* FROM "users" in rails.
Please provide needful description for this.
If you want a query equivalent to SELECT "users".* FROM "users" you should be using the following in your console:
User.all
If you want to just select the email field, generating the query SELECT "users"."email" FROM "users" you would use the pluck modifier, which will return you an array of emails:
User.pluck(:email)
Notice we've dropped the all query modifier. It isn't required in this instance, but you can add it for clarity if you like:
User.all.pluck(:email)
Be aware though that this will return you an array just containing your requested attribute, not instances of the model they belong to.
For that, you would use the select modifier:
User.select(:email)
#or
User.all.select(:email)
This will return you instances of your User class with email populated with the data from the DB. This is useful if you need to chain this query with others, but will be less performant than the pluck alternative with a large dataset.
Try this
#users = User.select("email")
Related
I'm trying to make a N1QL based query on Spring Data Couchbase. The documentation says
#n1ql.fields will be replaced by the list of fields (eg. for a SELECT clause) necessary to reconstruct the entity.
My repository implementation is this one:
#Query("#{#n1ql.fields} WHERE #{#n1ql.filter}")
List<User> findAllByFields(String fields);
And I'm calling this query as follows:
this.userRepository.findAllByFields("SELECT firstName FROM default");
I'm getting this error:
Caused by: org.springframework.data.couchbase.core.CouchbaseQueryExecutionException: Unable to execute query due to the following n1ql errors:
{"msg":"syntax error - at AS","code":3000}
After a little bit of researching, I also tryed:
#Query("SELECT #{#n1ql.fields} FROM #{#n1ql.bucket} WHERE #{#n1ql.filter}")
With this query, I don't get an error, I get all the documents stored but only the ID the other fields are set to null, when my query tries to get the firstName field.
this.userRepository.findAllByFields("firstName");
Anyone knows how to do such a query?
Thank you in advance.
You're misunderstanding the concept, I encourage you to give the documentation more time and see more examples. I'm not sure what exactly you're trying to achieve but I'll throw some examples.
Find all users (with all of their stored data)
#Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter}")
List<User> findAllUsers();
This will basically generate SELECT meta().id,_cas,* FROM bucket WHERE type='com.example.User'
Notice findAllUsers() does not take any parameters because there are no param placeholders defined in the #Query above.
Find all users where firstName like
#Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter} AND firstName like $1")
List<User> findByFirstNameLike(String keyword);
This will generate something like the above query but with an extra where condition firstName like
Notice this method takes a keyword because there is a param placeholder defined $1.
Notice in the documentation it says
#{#n1ql.selectEntity} WHERE #{#n1ql.filter} AND test = $1
is equivalent to
SELECT #{#n1ql.fields} FROM #{#n1ql.bucket} WHERE
#{#n1ql.filter} AND test = $1
Now if you don't want to fetch all the data for user(s), you'll need to specify the fields being selected, read following links for more info
How to fetch a field from document using n1ql with spring-data-couchbase
https://docs.spring.io/spring-data/couchbase/docs/2.2.4.RELEASE/reference/html/#_dto_projections
I think you should try below query, that should resolve the issue to get fields based parameter you have sent as arguments.
Please refer blow query.
#Query("SELECT $1 FROM #{#n1q1.bucket} WHERE #{#n1ql.filter}")
List findByFirstName(String fieldName);
Here, bucket name resolve to the User entity and and n1ql.filter would be a default filter.
I want to clean robots entries from log file. One of the way to identify crawlers by the user agent field in weblog.I've stored raw logs in one folder and token of the popular crawlers in crawler table. TO clean logs those have user agent matched with token i made this query
CREATE TABLE temp
AS
SELECT host,time,method,url,protocol,status,size,referer,agent
FROM raw_logs
WHERE
agent NOT RLIKE (SELECT concat_ws("|",collect_set(concat("(.*",token,".*)"))) FROM crawler) ;
It gives me parseException cannot recognize input near 'SELECT' 'concat_ws' '(' in expression specification
If i replace result of sub query manually then it works perfect.
CREATE TABLE temp
AS
SELECT host,time,method,url,protocol,status,size,referer,agent
FROM raw_logs
WHERE agent NOT RLIKE '(.*Googlebot.*)|(.*bingbot.*)' ;
So sub query in LIKE clause not supported by hive 1.0.1 ?
Similar query in mysql works perfect.
So, I have a problem. I have a query which returns ids from one table (say table1) and I have to pass those ids to another query which uses table2. (Writing inner selects or joins is not an option due to some certain reasons).
Query:
client = Mysql2::Client.new(:host => "localhost", :username => "", :password => "", :database =>"test")
query1 = %Q{select id from table1 where code='ABC123'}
ids = client.query(query1)
query2 = %Q{select * from table2 where `table2`.`table1_id` IN (#{ids}) and status="rejected"}
table2_data = client.query(query2)
ids is Mysql2::Result type
Also, when I do ids.to_a, the resulting array has data something like this: [{"id"=>1}, {"id"=>2}]
I need some feasible way to pass ids to the second query. I tried ids.to_a, but it gives error due to the braces [ ]. I have also tried concatenating, say the MySQL result is:
array = ids.to_a # [1,2,3]
id_new = "("+#{array.join(',')}+")"
id_new becomes "(1,2,3)" which is a string and hence IN doesn't work.
Can anyone please suggest something how to pass ids array in the raw MySQL query?
I have banged my head finding the answer, but couldn't find an appropriate one.
Edit: I can use Active Record only for query1 and if that is the case and ids is an Active Record object, can anyone suggest how to pass it to query2 in the IN clause which is supposed to be a raw SQL query?
Edit2: I can't use Active Record (for query2) or join because it's making the query heavy and taking long time (>10s) to fetch the result (indices are present). So, I am using raw query to optimise it.
When I ran similar queries to try to mimic your problem I saw that I'm getting an array of array for ids, like [["1"], ["2"], ["3"]].
If this is also what you're getting then you should call ids.flatten before calling join:
query2 = %Q{select * from table2 where `table2`.`table1_id` IN (#{ids.flatten.join(',')}) and status="rejected"}
array.flatten removes extra braces, so:
[[1], [2], [3]].flatten
# => [1,2,3]
[[1], [2], [3]].flatten.join(',')
# => "1,2,3"
EDIT
Since you reported you are receiving a Mysql2::Result object, do this:
ids.to_a.map(&:values).flatten.join(',')
The to_a first converts the Mysql2::Result to an array of hashes that looks like this:
[{"id"=>"1"}, {"id"=>"2"}]
Then using map(&:values) we convert it to an array that looks like this:
[["1"], ["2"]]
This array is similar to the above (before the edit), so running flatten.join(',') converts it to the string you are looking for.
Note that instead of doing map(&:values).flatten you could use the common shortcut flat_map(&:values) which results in the same thing.
Are you sure it doesn't work because it is a string. I think it doesn't work because of duplicate brackets. Please try this:
array = ids.flat_map(&:values).join(',')
query2 = %Q{select * from table2 where `table2`.`table1_id` IN (#{array}) and status="rejected"}
I suggest to use a ORM (object-relational mapping) like the ActiveRecord or Sequel gems - especially because building database queries manually by string concatination is error prone and leads to vulnerabilities like sql injections.
If the main reason you posted was to learn how to extract data from an array of hashes, then you can ignore this answer.
However, if you wanted the best way to get the data from the database, I'd suggest you use ActiveRecord to do the donkey work for you:
class Table1 < ActiveRecord::Base
self.table_name = :table1
has_many :table2s
end
class Table2 < ActiveRecord::Base
self.table_name = :table2
belongs_to :table1
end
table2_data = Table2.joins(:table1).where(table1: {code: 'ABC123'}, status: 'rejected')
A key point is that a SQL join, will effectively do the processing of the IDs for you. You could code up the SQL join yourself, but ActiveRecord will do that for you, and allow you to add the additional queries, such that you can gather the data you want in one query.
You can join array with comma, like following code.
ids = ids.to_a.map{|h| h['id']}
query2 = %Q{select * from table2 where `table2`.`table1_id` IN (#{ids.join(',')}) and status="rejected"}
table2_data = client.query(query2)
It will work fine.
I am doing something like this:
data = Model.where('something="something"')
random_data = data.rand(100..200)
returns:
NoMethodError (private method `rand' called for #<User::ActiveRecord_Relation:0x007fbab27d7ea8>):
Once I get this random data, I need to iterate through that data, like this:
random_data.each do |rd|
...
I know there's a way to fetch random data in MySQL, but I need to pick the random data like 400 times, so I think to load data once from database and 400 times to pick random number is more efficient than to run the query 400 times on MySQL.
But - how to get rid of that error?
NoMethodError (private method `rand' called for #<User::ActiveRecord_Relation:0x007fbab27d7ea8>):
Thank you in advance
I would add the following scope to the model (depends on the database you are using):
# to model/model.rb
# 'RANDOM' works with postgresql and sqlite, whereas mysql uses 'RAND'
scope :random, -> { order('RAND()') }
Then the following query would load a random number (in the range of 200-400) of objects in one query:
Model.random.limit(rand(200...400))
If you really want to do that in Rails and not in the database, then load all records and use sample:
Model.all.sample(rand(200..400))
But that to be slower (depending on the number of entries in the database), because Rails would load all records from the database and instantiate them what might take loads of memory.
It really depends how much effort you want to put into optimizing this, because there's more than one solution. Here's 2 options..
Something simple is to use ORDER BY RAND() LIMIT 400 to randomly select 400 items.
Alternatively, just select everything under the moon and then use Ruby to randomly pick 400 out of the total result set, ex:
data = Model.where(something: 'something').all # all is necessary to exec query
400.times do
data.sample # returns a random model
end
I wouldn't recommend the second method, but it should work.
Another way, which is not DB specific is :
def self.random_record
self.where('something = ? and id = ?', "something", rand(self.count))
end
The only things here is - 2 queries are being performed. self.count is doing one query - SELECT COUNT(*) FROM models and the other is your actual query to get a random record.
Well, now suppose you want n random records. Then write it like :
def self.random_records n
records = self.count
rand_ids = Array.new(n) { rand(records) }
self.where('something = ? and id IN (?)',
"something", rand_ids )
end
Use data.sample(rand(100..200))
for more info why rand is not working, read here https://rails.lighthouseapp.com/projects/8994-ruby-on-rails/tickets/4555
I have the following call in my RoR script:
Foo.where("event = 'Login'").group('user_id').order('user_id ASC').count()
This gives me a list of all users and how much they have logged in in the form of:
<userid1> => <count>, <userid2> => <count>, ...}
This is great and very close to what I wan but I've been unable to convince it to sort by the count of logins instead, what I'd really like to have it do. There is also a column that has some info about the login session in the form of a character delimited string. I'd like to get at certain parts of that information.
To achieve this I've tried using find_by_sql and when I make the following call:
Foo.find_by_sql("SELECT userid, COUNT(*) AS number, SUBSTRING_INDEX(stuff, ',', 1) AS info FROM <table> WHERE event = 'Login' GROUP BY userid")
What I get is a ilst of Foo entries that contain the userids but not the count or the info. When I run this in the MySQL workbench it works like a charm. Is there something else I need to do to get this to work? Also, would there be a way to just do this using Foo.select or Foo.where? Thanks.
Update I have also tried this format, as demonstrated here:
Foo.find(:all, :select => 'count(*) count, userid', :group =>'userid')
But this too merely responds with the userids and does not spit out the count.
Update 2 Looking at the output a bit more i can see now that when i do the find_by_fql call everything is being found in the correct way and even being sorted. It just isn't actually selecting the COUNT(*) or the SUBSTRING_INDEX.
Update 3 I also tried out this SO tip but when I tell it:
Foo.find(:all, :select => 'userid, count(*) as cnt', :group => 'userid')
It doesn't print or find anything related to the var cnt. I'm totally baffled here because I've seen more than one example now that does it this ^^ way and I've yet to get it to succeed.
Actually, your problem is not an SQL problem. To generate the correct SQL you would just need this:
Foo.where("event = 'Login'").group('user_id').order('count_all').count()
Take a look in your log and you'll find that this generates the following SQL:
SELECT COUNT(*) AS count_all, user_id AS school_id FROM `foos` GROUP BY user_id ORDER BY count_all
...and if you run that in your SQL console you'll get what you want.
The problem is that Rails doesn't return them in this order, Rails always returns these special group/count results in the order of the GROUP BY field. So, if you want them in a different order then you'll need to do it in Ruby after getting the hash back.
Code below returns an array of foos, checking any element inside foo will return userid/cnt
foos = Foo.find(:all, :select => 'userid, count(*) as cnt', :group => 'userid')
Is this what you're looking for?
foos.first.userid # will show userid
foos.first.cnt # will show count